Andres Freund <andres@anarazel.de> writes:
> On 2022-02-15 23:30:07 -0500, Tom Lane wrote:
>> Hmm, I did it the other way, as attached.
> My goal when I did this was to improve the performance, rather than reduce the
> memory usage (this was a few months back). It's clearly better to perform all
> the replacements of a line soon after each other, rather than iterate over all
> the lines, once for each replacement. The latter is guaranteed to not have the
> data in L2/L1 anymore.
I'm skeptical that that effect is large enough to be interesting ...
> But if we move to not needing replacement for postgres.bki anymore, that's not
> an important goal anymore.
... and as you say, it won't matter if we push this processing to
the backend. So I'd prefer to go with the less-invasive change.
> Not that it matters anymore: At least for postgres.bki, it doesn't seem to
> make sense to use a line based approach in the first place? Seems like it
> should really be a processing the input file as a whole, doing all the
> replacements, into a resizable output buffer.
If we push this to the backend then it'd all have to be rethought.
I kind of like Peter's idea that maybe the run-time-variable items
could be handled by doing UPDATEs after the initial bootstrap.
(Very far down the road, maybe that would put us in a position where
the bootstrap phase could be replaced by copying some prefab data files.)
>> It seems that actually the pointer arrays *are* a big chunk of
>> the leakage, because the individual strings get freed in the
>> output loops!
> Right, isn't that what I had said?
I hadn't absorbed the existence of the per-string free() calls;
the fact that the reported leakage was close to the size of
postgres.bki misled me into thinking we were leaking the
strings too.
regards, tom lane