> Could you adjust your process to log the actual temp table size
> each time, ie "select count(*) from temp_table" in between the
> two steps, and then note whether the failures are correlated
> with unusual temp table sizes?
Thanks, I'll add a count there and come back with a number when we next
hit the "invalid memory alloc".
Stefan
On 11/03/2020 15:37, Tom Lane wrote:
> Stefan Blanke <stefan.blanke@framestore.com> writes:
>> We've upgraded to PostgreSQL 11.5 (postgresql.org rhel 6 rpm) and I have
>> had another occurrence of this invalid alloc of 1GB. Apologies for never
>> providing a query plan when discussing this two years ago; we decided to
>> move to a newer PostgreSQL to see if the issue went away but took a
>> while to complete the move.
>
>> The invalid alloc still only occurs occasionally every few months on a
>> query that we run every minute; so unfortunately we still don't have a
>> contained reproducible test case.
>
> Given the lack of stats, I wonder whether the issue could be related
> to the plan sometimes being horribly bad, eg due to the temp table
> being much larger than expected. (A possible mechanism would be
> hash table bloat, perhaps, but that's getting way ahead of the
> evidence.)
>
> Could you adjust your process to log the actual temp table size
> each time, ie "select count(*) from temp_table" in between the
> two steps, and then note whether the failures are correlated
> with unusual temp table sizes?
>
> regards, tom lane
>