Possible memory leak of cx_Oracle/pyodbc/Oracle Instant Client?

Let me first list all the relevant information…

  • Oracle database Version 19.4.0.0.0
  • Oracle instantclient version: 21.3.0.0.0
  • OS version: RHEL 7.9 x64
  • cx_Oracle version: 8.2
  • pyodbc version: 4.0.32
  • unixODBC driver version: 2.3.8

I tried both pyodbc and cx_Oracle, the issues are essentially the same. Here is a simplified version of my code:

# cx_Oracle version of connection init
dsn = cx_Oracle.makedsn(host=(host), port=(port), sid=(sid))
con = cx_Oracle.connect(user=(username), password=(password), dsn=dsn)
# pyodbc version of connection init
con = pyodbc.connect('DRIVER=(driver-definition);DBQ=(host):(port)/(service-name);UID=(username);PWD=(password)')

with con.cursor() as cursor:
    cursor.arraysize = 100_000 # Only effective when using cx_Oracle
    sql = """
            SELECT "Date", "ID", "KeyName", "ValueName"
            FROM "A_Table"
            WHERE ("A_Table"."Date" >= '07-SEP-20' AND 
                   "A_Table"."Date" <= '07-SEP-21')
    """
    # So I just to get everything from the last year
    cursor.execute(sql)
    while True:
        for fetched_rows in cursor.fetchmany():
            if not fetched_rows :
                break   
            rows.append(row)

The issue is, the code can run, just it uses a LOT of memory. My guesstimate of the final resultant rows object should be no more than 2GB in size but the script uses up all my 30GB memory and gets terminated by OS (i.e., I dont know exactly how much memory the script would like to allocate since it is not given the chance). Before the script gets killed, it asks for about 200 Mbytes of memory each second so my 30GB memory does not last for too long…At the moment it is killed, usually it fetches no more than 50% of the rows needed.

Here comes the really strange thing–I can avoid the above issue by commenting out rows.append(row). That is, if I don’t save the fetched rows and just keeps reading and discarding, the script becomes totally fine…

One may say: would it be possible that there is a mismatch between rows’ actual size and cx_Oracle/pyodbc‘s idea of their size? This is unlikely, I tried printing cursor.description and the result is the following:
(('Date', <class 'cx_Oracle.DATETIME'>, 23, None, None, None, 0), ('ID', <class 'cx_Oracle.STRING'>, 9, 9, None, None, 0), ('KeyName', <class 'cx_Oracle.STRING'>, 53, 53, None, None, 0), ('ValueName', <class 'cx_Oracle.NUMBER'>, 127, None, 126, -127, 0)). Looks normal.

What if I do this instead:

    while True:
        for fetched_rows in cursor.fetchmany():
            if not fetched_rows :
                break
            dcfr = copy.deepcopy(fetch_rows) 
            fetch_rows = None # we keep a deepcopy of fetched_rows, hoping it can release the object reference and space.
            rows.append(dcfr)
            gc.collect() # Manual garbage collection to avoid any delay

No, it won’t solve the issue…Any idea?…