On Mon, Dec 9, 2019 at 4:37 PM Greg Stark <stark@mit.edu> wrote:
> On Mon, 9 Dec 2019 at 14:03, Ibrar Ahmed <ibrar.ahmad@gmail.com> wrote:
> > I'd
> > actually argue that the segment size should be substantially smaller
> > than 1 GB, like say 64MB; there are still some people running systems
> > which are small enough that allocating 1 GB when we may need only 6
> > bytes can drive the system into OOM."
>
> I don't even see why you would allocated as much as 64MB. I would
> think something around 1MB would be more sensible. So you might need
> an array of segment pointers as long as a few thousand pointers, big
> deal. We can handle repalloc on 8kB arrays pretty easily.
See https://www.postgresql.org/message-id/9bf3fe70-7aac-cbf7-62f7-acdaa4306ccb%40iki.fi
Another consideration is that, if we have parallel VACUUM, this all
needs to be done using DSM or DSA, neither of which is going to do a
fantastic job with lots of 1MB allocations. If you allocate 1MB DSMs,
you'll run out of DSM slots. If you allocate 1MB chunks from DSA,
it'll allocate progressively larger DSMs and give you 1MB chunks from
them. That's probably OK, but you're just wasting whatever memory from
the chunk you don't end up allocating.
I suggested 64MB because I don't think many people these days run out
of memory because VACUUM overshoots its required memory budget by a
few tens of megabytes. The problem is when it overruns by hundreds of
megabytes, and people would like large maintenance_work_mem settings
where the overrun might be gigabytes.
Perhaps there are contrary arguments, but I don't think the cost of
repalloc() is really the issue here.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company