Hi,
On 2022-02-15 20:45:46 -0500, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2022-02-15 11:33:26 -0500, Tom Lane wrote:
> >> It might be worth trying to knock that down a bit, but I wouldn't
> >> start with a one-time leak of 28 bytes. It looks like the biggest
> >> offender is that we don't bother trying to reclaim the lines
> >> malloc'd by readfile() and replace_token(). Fixing that is *maybe*
> >> worth the trouble, but TBH no one has complained about initdb's
> >> memory consumption.
>
> > It's a bit insane that we allocate the lines[] quite so many times, when
> > processing the same file.
>
> Yeah, I noticed that --- why don't we reuse the array of pointers?
> Not sure it'd save much compared to freeing the strings, but it is
> mighty low-hanging fruit.
The number of replacements is low enough that the memory for the changed
strings themselves doesn't actually matter much, I think. replace_token()
doesn't allocate memory for unchanged strings...
I think we'd see memory usage of quite different proportions otherwise - my
postgres.bki is 900kB. 9 copies of that would start to add up...
for k in NAMEDATALEN SIZEOF_POINTER ALIGNOF_POINTER FLOAT8PASSBYVAL POSTGRES ENCODING LC_COLLATE LC_CTYPE;do echo $k:
$(grep-c $k ./src/backend/catalog/postgres.bki);done
NAMEDATALEN: 5
SIZEOF_POINTER: 2
ALIGNOF_POINTER: 2
FLOAT8PASSBYVAL: 8
POSTGRES: 1
ENCODING: 1
LC_COLLATE: 1
LC_CTYPE: 1
> > The replacement patterns either are
> > compile time constants which we just should handle in genbki.pl, or have
> > exactly 1 replacement....
>
> Mmm, really? I thought most of them were data that we don't know
> until initdb runs. Anything that really is known at build time,
> sure, genbki.pl ought to take care of.
Only POSTGRES, ENCODING, LC_COLLATE, LC_CTYPE of the above list are runtime
variable, right? And those just affect two rows in total...
I was pondering initdb's design a bunch lately. So I started a -hackers thread:
https://postgr.es/m/20220216021219.ygzrtb3hd5bn7olz%40alap3.anarazel.de
Greetings,
Andres Freund