Re: pgsql: Refactor pg_get_line() to expose an alternative StringInfo-based

Поиск
Список
Период
Сортировка
От Pavel Stehule
Тема Re: pgsql: Refactor pg_get_line() to expose an alternative StringInfo-based
Дата
Msg-id CAFj8pRC2ixEeNK-a9XQwVrP6Vf3KhEG42ckq+gZZvmDTnnGQxA@mail.gmail.com
обсуждение исходный текст
Ответ на pgsql: Refactor pg_get_line() to expose an alternative StringInfo-based  (Tom Lane <tgl@sss.pgh.pa.us>)
Ответы Re: pgsql: Refactor pg_get_line() to expose an alternative StringInfo-based  (Alvaro Herrera <alvherre@2ndquadrant.com>)
Список pgsql-committers
Hi

ne 6. 9. 2020 v 20:13 odesílatel Tom Lane <tgl@sss.pgh.pa.us> napsal:
Refactor pg_get_line() to expose an alternative StringInfo-based API.

Letting the caller provide a StringInfo to read into is helpful when
the caller needs to merge lines or otherwise modify the data after
it's been read.  Notably, now the code added by commit 8f8154a50
can use pg_get_line_append() instead of having its own copy of that
logic.  A follow-on commit will also make use of this.

Also, since StringInfo buffers are a minimum of 1KB long, blindly
using pg_get_line() in a loop can eat a lot more memory than one would
expect.  I discovered for instance that commit e0f05cd5b caused initdb
to consume circa 10MB to read postgres.bki, even though that's under
1MB worth of data.  A less memory-hungry alternative is to re-use the
same StringInfo for all lines and pg_strdup the results.


I tried  to reuse this new API in pg_dump.c, and I had a problem with private struct StringInfo.

Inside initdb.c there is a access to this structure via #include "access/xlog_internal.h" and it is little bit unreadable

maybe there should be included directly with #include "lib/stringinfo.h" ?

Regards

Pavel


 
Discussion: https://postgr.es/m/1315832.1599345736@sss.pgh.pa.us

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/8e3c58e6e459b285d37edb6129e412ed25cd90c1

Modified Files
--------------
src/backend/libpq/hba.c     | 40 +++++++++-----------------
src/bin/initdb/initdb.c     | 12 ++++++--
src/common/pg_get_line.c    | 70 +++++++++++++++++++++++++++++++--------------
src/include/common/string.h |  3 ++
4 files changed, 75 insertions(+), 50 deletions(-)

В списке pgsql-committers по дате отправления:

Предыдущее
От: Amit Kapila
Дата:
Сообщение: pgsql: Add additional tests to test streaming of in-progress transactio
Следующее
От: Michael Paquier
Дата:
Сообщение: pgsql: doc: Tweak sentence for pg_checksums when enabling checksums