Re: BUG #19528: Assert failure in generate_normalized_query() via Squashed Array Literals
Re: BUG #19528: Assert failure in generate_normalized_query() via Squashed Array Literals
On Sat, Jun 20, 2026 at 7:53 PM PG Bug reporting form
wrote:
> `" /*, ... */"` is 11 bytes, so a squashed entry writes `"$N"` (2 bytes)
> plus `" /*, ... */"` (11 bytes) = 13 bytes to replace a 1-byte constant, a
> net expansion of 12 bytes. The budget of 10 bytes is exceeded by 2 bytes per
> squashed location. With N squashed array elements the buffer overflows by 2N
> bytes.
This analysis doesn't seem correct to me.
For "ARRAY[1,2]", the recorded location covers the entire list text,
"1,2" (3 bytes), which is replaced with "$n /*, ... */". For a
single-digit placeholder, "$1 /*, ... */" is 13 bytes, so replacing
"1,2" results in a net growth of 10 bytes.
That still fits within the existing budget. The budget is first
exceeded when the placeholder number reaches two digits:
"$10 /*, ... */" is 14 bytes, giving a net growth of 11 bytes.
Therefore, the smallest reproducer is 10 squashed "ARRAY[1,2]"
entries, where the shortfall seems only 1 byte overall, not 2 bytes per
squashed location.
> ### Fix
>
> Account for the extra 11 bytes appended when a squashed suffix is written.
> One correct approach:
>
> ```c
> norm_query_buflen = query_len + jstate->clocations_count *
> (10 + (jstate->has_squashed_lists ? 11 : 0));
> ```
>
> A simpler conservative fix that covers the worst case (`"$2147483648 /*, ...
> */"` = 22 bytes replacing a 1-byte constant, net 21 bytes):
>
> ```c
> norm_query_buflen = query_len + jstate->clocations_count * 22;
> ```
How about keeping the current "clocations_count * 10" budget for
ordinary replacements, counting squashed entries separately, and
adding only the extra space they may need, for example:
clocations_count * 10 + squashed_count * 9
The existing "+10" already covers ordinary placeholders. For squashed
entries, the worst-case additional overhead is 9 bytes, from replacing
the shortest possible source text "1,2" (3 bytes) with the longest
possible placeholder "$2147483647 /*, ... */" (22 bytes), for a total
growth of 19 bytes. Attached patch implements this approach.
Thoughts?
Regards,
--
Fujii Masao
Re: BUG #19528: Assert failure in generate_normalized_query() via Squashed Array Literals
От:
王跃林 <violin0613@tju.edu.cn>
Дата:
The analysis looks correct to me; clocations_count * 10 + squashed_count * 9 tightly covers the worst case and is cleaner than the alternatives.
王跃林
3020001251@tju.edu.cn
Original:
- From:Fujii Masao <masao.fujii@gmail.com>
- Date:2026-06-20 23:18:19(中国 (GMT+08:00))
- To:3020001251<3020001251@tju.edu.cn> , pgsql-bugs<pgsql-bugs@lists.postgresql.org>
- Cc:
- Subject:Re: BUG #19528: Assert failure in generate_normalized_query() via Squashed Array Literals
On Sat, Jun 20, 2026 at 7:53 PM PG Bug reporting form
<noreply@postgresql.org> wrote:
> `" /*, ... */"` is 11 bytes, so a squashed entry writes `"$N"` (2 bytes)
> plus `" /*, ... */"` (11 bytes) = 13 bytes to replace a 1-byte constant, a
> net expansion of 12 bytes. The budget of 10 bytes is exceeded by 2 bytes per
> squashed location. With N squashed array elements the buffer overflows by 2N
> bytes.
This analysis doesn't seem correct to me.
For "ARRAY[1,2]", the recorded location covers the entire list text,
"1,2" (3 bytes), which is replaced with "$n /*, ... */". For a
single-digit placeholder, "$1 /*, ... */" is 13 bytes, so replacing
"1,2" results in a net growth of 10 bytes.
That still fits within the existing budget. The budget is first
exceeded when the placeholder number reaches two digits:
"$10 /*, ... */" is 14 bytes, giving a net growth of 11 bytes.
Therefore, the smallest reproducer is 10 squashed "ARRAY[1,2]"
entries, where the shortfall seems only 1 byte overall, not 2 bytes per
squashed location.
> ### Fix
>
> Account for the extra 11 bytes appended when a squashed suffix is written.
> One correct approach:
>
> ```c
> norm_query_buflen = query_len + jstate->clocations_count *
> (10 + (jstate->has_squashed_lists ? 11 : 0));
> ```
>
> A simpler conservative fix that covers the worst case (`"$2147483648 /*, ...
> */"` = 22 bytes replacing a 1-byte constant, net 21 bytes):
>
> ```c
> norm_query_buflen = query_len + jstate->clocations_count * 22;
> ```
How about keeping the current "clocations_count * 10" budget for
ordinary replacements, counting squashed entries separately, and
adding only the extra space they may need, for example:
clocations_count * 10 + squashed_count * 9
The existing "+10" already covers ordinary placeholders. For squashed
entries, the worst-case additional overhead is 9 bytes, from replacing
the shortest possible source text "1,2" (3 bytes) with the longest
possible placeholder "$2147483647 /*, ... */" (22 bytes), for a total
growth of 19 bytes. Attached patch implements this approach.
Thoughts?
Regards,
--
Fujii Masao
pgsql: pg_locale.c: comment improvements.
От:
Jeff Davis <jdavis@postgresql.org>
Дата:
pg_locale.c: comment improvements. Add missing comments and fix outdated/incorrect comments in pg_locale.c and related files. Suggested-by: Andres Freund Discussion: https://postgr.es/m/v3nniwcrxejmcfvz56xbd22hphprqleuornd6hqkmw2bl7kgmz@cnytz2ee5ltk Backpatch-through: 18 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/cba80d5e74d0ee5f39874db63348f3b35e55afe3 Modified Files -------------- src/backend/utils/adt/pg_locale.c | 106 ++++++++++++++++++++++++------ src/backend/utils/adt/pg_locale_builtin.c | 4 +- src/backend/utils/adt/pg_locale_icu.c | 19 +++--- src/include/utils/pg_locale.h | 13 +++- 4 files changed, 106 insertions(+), 36 deletions(-)