Bruce Momjian <maillist@candle.pha.pa.us> writes:
> Actually, our problem is not malloc itself. Most Unix OS's have pretty
> good malloc's, tuned to their OS. The problem is the number of times we
> call it.
Well, some malloc libs are noticeably better than others, but as long
as the operating assumption is that any allocated block can be freed
independently of any other one, it's hard to do a *lot* better than
a standard malloc library. You have to keep track of each allocated
chunk and each free area, individually, to meet malloc/free's API.
What we need to do is exploit the notion of pooled allocation
(contexts), wherein the memory management apparatus doesn't keep track
of each allocation individually, but just takes it from a pool of space
that will all be freed at the same time. End of statement, end of
transaction, etc, are good pool lifetimes for Postgres.
We currently have the worst of both worlds: we pay malloc's overhead,
and we have a *separate* bookkeeping layer on top of malloc that links
allocated blocks together to allow everything to be freed at end-of-
context. We should be able to do this more cheaply than malloc, not
more expensively.
BTW, I already did something similar in the frontend libpq, and it
was a considerable win for reading large SELECT results.
regards, tom lane