Re: Understanding memory usage

Поиск
Список
Период
Сортировка
От Damiano Albani
Тема Re: Understanding memory usage
Дата
Msg-id CAKys9521rbsAVDk+h2xc2fdNs9-XXRuFJJEGh7ozc6MELU+m1w@mail.gmail.com
обсуждение исходный текст
Ответ на Re: Understanding memory usage  (Daniele Varrazzo <daniele.varrazzo@gmail.com>)
Ответы Re: Understanding memory usage
Список psycopg
Дерево обсуждения
Hello,

On Tue, Oct 29, 2013 at 12:23 AM, Daniele Varrazzo <daniele.varrazzo@gmail.com> wrote:

Because the result is returned to the client as the response for the
query and is stored inside the cursor. fetch*() only return it to
Python.

So why does calling "fetch*()" uses additional memory then? Does it copy the data returned from the database?


By the way, I've re-run my tests but focused on the VmRSS metric, which represents how much actual physical memory is used by the process.

And I got the same behavior, that is almost no memory is reclaimed after having fetched a large number of rows.
For instance, if I fetch 2 millions small rows, memory usage peaks around 500 MB and then only lowers to ~ 450 MB after data is freed.
On the other hand, fetching 100 large rows amounts to a 3 GB peak, which subsequently falls back to 10 MB.

So is it a problem related to Psycopg itself or rather how Python handles memory in general?

Regards,

--
Damiano Albani

В списке psycopg по дате отправления:

Предыдущее
От: Daniele Varrazzo
Дата:
Сообщение: Re: Understanding memory usage
Следующее
От: Daniele Varrazzo
Дата:
Сообщение: Re: Understanding memory usage