Hello,
I'm trying to understand how Psycopg behaves memory-wise.
Basically, in my use case, it looks like memory is not released as I thought it would be.
and run on my system (Ubuntu 12.04/x64, PostgreSQL 9.3, Python 2.7.3, Psycopg 2.5.1).
Using the original query:
SELECT pagecontent FROM test WHERE siteid = 3;
loaded 0
VmSize: 3251264 kB
cleared 0
VmSize: 112924 kB
loaded 1
VmSize: 3250496 kB
cleared 1
VmSize: 112928 kB
So far, so good -- incidentally, the lowest memory usage seems to 112 MB, where it is reported at 78 MB in bug report #78. Any idea why?
Now, using this very simple query (column1 being a NUMBER, around 2 millions rows):
SELECT column1 FROM table1
loaded 0
VmSize: 259392 kB
cleared 0
VmSize: 232960 kB
loaded 1
VmSize: 259392 kB
cleared 1
VmSize: 226676 kB
So why isn't the memory reclaimed as in the first case?
I've also done some tests where the fetchall() call is commented altogether. The results are :
SELECT pagecontent FROM test WHERE siteid = 3;
loaded 0
VmSize: 2226860 kB
cleared 0
VmSize: 112920 kB
loaded 1
VmSize: 2226488 kB
cleared 1
VmSize: 112924 kB
SELECT column1 FROM table1
loaded 0
VmSize: 159772 kB
cleared 0
VmSize: 113036 kB
loaded 1
VmSize: 162356 kB
cleared 1
VmSize: 112920 kB
In that case, the memory is effectively reclaimed for the second query -- getting to the lowest 112 MB number I have mentioned before.
But what is the reason why Psycopg needs memory to just run a query, without fetching results?
Thanks for your help!