Attached is a patch for a write after allocated memory which we found in testing. Its an obscure case but can happen if the same column is used in different grouping keys, as in the example below, which uses tables from the regress test suite (build with --enable-cassert in order to turn on memory warnings). Patch is against master.
The hashed aggregate state has an array for the column indices that is sized using the number of non-aggregated columns in the set that includes the agg's targetlist, quals and input grouping columns. The duplicate elimination of columns can result in under-allocation, as below. Sizing based on the number of grouping columns and number of quals/targetlists not in the grouping columns avoids this.
Regards,
Colm McHugh (Salesforce)
explain (costs off) select 1 from tenk where (hundred, thousand) in (select twothousand, twothousand from onek);
psql: WARNING: problem in alloc set ExecutorState: detected write past chunk end in block 0x7f8b8901fa00, chunk 0x7f8b89020cd0
psql: WARNING: problem in alloc set ExecutorState: detected write past chunk end in block 0x7f8b8901fa00, chunk 0x7f8b89020cd0
QUERY PLAN
-------------------------------------------------------------
Hash Join
Hash Cond: (tenk.hundred = onek.twothousand)
-> Seq Scan on tenk
Filter: (hundred = thousand)
-> Hash
-> HashAggregate
Group Key: onek.twothousand, onek.twothousand
-> Seq Scan on onek
(8 rows)