Re: [ADMIN] Processing very large TEXT columns (300MB+) using C/libpq

Поиск
Список
Период
Сортировка
От Bear Giles
Тема Re: [ADMIN] Processing very large TEXT columns (300MB+) using C/libpq
Дата
Msg-id CALBNtw4rATXoUhT1CSH7ecdezvkkEAfdT4f=nH3s973ATDOLeQ@mail.gmail.com
обсуждение исходный текст
Ответ на Re: [ADMIN] Processing very large TEXT columns (300MB+) using C/libpq  (Tom Lane <tgl@sss.pgh.pa.us>)
Ответы Re: [ADMIN] Processing very large TEXT columns (300MB+) using C/libpq  (Geoff Winkless <pgsqladmin@geoff.dj>)
Список pgsql-admin
Дерево обсуждения
As an aside any halfway decent optimizer would realize that the results of strlen() are unchanging as long as the contents of what it's passed isn't modified. That's a common enough pattern that it should be checked.

What about buffer size? Are you using a smaller fetch size that results in lots of little packets?

Is there a way to specify a connection that uses compression?

On Fri, Oct 20, 2017 at 10:55 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Cory Nemelka <cnemelka@gmail.com> writes:
> Yes, but I should be able to read them much faster.  The psql client can
> display an 11MB column in a little over a minute, while in C using libpg
> library, it takes over an hour.

Well, of course psql relies on libpq, so it seems unlikely that libpq
itself is where the time is going.  Have you tried applying a profiler?
"perf" or "oprofile" or similar tool ought to pinpoint the culprit
pretty easily.

                        regards, tom lane


--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

В списке pgsql-admin по дате отправления:

Предыдущее
От: Tom Lane
Дата:
Сообщение: Re: [ADMIN] Processing very large TEXT columns (300MB+) using C/libpq
Следующее
От: Geoff Winkless
Дата:
Сообщение: Re: [ADMIN] Processing very large TEXT columns (300MB+) using C/libpq