Hi,
On 2023-11-06 13:02:50 -0500, Stephen Frost wrote:
> > > The max_total_memory limit is checked whenever the global counters are
> > > updated. There is no special error handling if a memory allocation exceeds
> > > the global limit. That allocation returns a NULL for malloc style
> > > allocations or an ENOMEM for shared memory allocations. Postgres has
> > > existing mechanisms for dealing with out of memory conditions.
> >
> > I still think it's extremely unwise to do tracking of memory and limiting of
> > memory in one patch. You should work towards and acceptable patch that just
> > tracks memory usage in an as simple and low overhead way as possible. Then we
> > later can build on that.
>
> Frankly, while tracking is interesting, the limiting is the feature
> that's needed more urgently imv.
I agree that we need limiting, but that the tracking needs to be very robust
for that to be usable.
> We could possibly split it up but there's an awful lot of the same code that
> would need to be changed and that seems less than ideal. Still, we'll look
> into this.
Shrug. IMO keeping them together just makes it very likely that neither goes
in.
> > > For sanity checking, pgstat now includes the pg_backend_memory_allocation
> > > view showing memory allocations made by the backend process. This view
> > > includes a scan of the top memory context, so it compares memory allocations
> > > reported through pgstat with actual allocations. The two should match.
> >
> > Can't you just do this using the existing pg_backend_memory_contexts view?
>
> Not and get a number that you can compare to the local backend number
> due to the query itself happening and performing allocations and
> creating new contexts. We wanted to be able to show that we are
> accounting correctly and exactly matching to what the memory context
> system is tracking.
I think creating a separate view for this will be confusing for users, without
really much to show for. Excluding the current query would be useful for other
cases as well, why don't we provide a way to do that with
pg_backend_memory_contexts?
Greetings,
Andres Freund