Re: postgres_fdw: fix cumulative stats after imported foreign-table stats
Re: postgres_fdw: fix cumulative stats after imported foreign-table stats
От:
Chao Li <li.evan.chao@gmail.com>
Дата:
> On Jun 13, 2026, at 17:32, Etsuro Fujita wrote: > > Hi Chao, > > On Fri, Jun 12, 2026 at 12:49 PM Chao Li wrote: >> While testing "[28972b6fc] Add support for importing statistics from remote servers", I found a problem: when remote stats are imported, cumulative stats are not updated. > > Thanks for the report and patch! > > Actually, I have also just noticed this problem in my post-commit > self-review. Will look into the patch. > > Best regards, > Etsuro Fujita Thanks for confirming. Then I think this issue deserves a place on the open items list, so I just added it. Best regards, -- Chao Li (Evan) HighGo Software Co., Ltd. https://www.highgo.com/
Re: postgres_fdw: fix cumulative stats after imported foreign-table stats
От:
Etsuro Fujita <etsuro.fujita@gmail.com>
Дата:
On Mon, Jun 15, 2026 at 4:37 PM Chao Li wrote: > Then I think this issue deserves a place on the open items list, so I just added it. Thanks for that! Best regards, Etsuro Fujita
Re: postgres_fdw: fix cumulative stats after imported foreign-table stats
От:
Heikki Linnakangas <hlinnaka@iki.fi>
Дата:
Hi Etsuro, On 15/06/2026 12:31, Etsuro Fujita wrote: > On Mon, Jun 15, 2026 at 4:37 PM Chao Li wrote: >> Then I think this issue deserves a place on the open items list, so I just added it. > > Thanks for that! Spotted this while going through the open items list. Have you had a chance to look into this? - Heikki
Re: postgres_fdw: fix cumulative stats after imported foreign-table stats
От:
Etsuro Fujita <etsuro.fujita@gmail.com>
Дата:
Hi Heikki, On Wed, Jul 1, 2026 at 11:54 PM Heikki Linnakangas wrote: > On 15/06/2026 12:31, Etsuro Fujita wrote: > > On Mon, Jun 15, 2026 at 4:37 PM Chao Li wrote: > >> Then I think this issue deserves a place on the open items list, so I just added it. > > > > Thanks for that! > > Spotted this while going through the open items list. Have you had a > chance to look into this? Yes, I have. I'll provide my review comments by tomorrow at the latest. Best regards, Etsuro Fujita
Re: postgres_fdw: fix cumulative stats after imported foreign-table stats
Hi Chao, On Fri, Jun 12, 2026 at 12:49 PM Chao Li wrote: > I think the root cause is that, with 28972b6fc, when stats are imported successfully for a foreign table, do_analyze_rel() is skipped. But do_analyze_rel() is the only place that calls pgstat_report_analyze() to update cumulative stats. > > To fix this, I think we need to add an output parameter to ImportForeignStatistics to pass out total live rows. AFAIK, the imported remote relation stats have no dead-tuple estimate, so analyze_rel() can pass 0 as the dead-tuple estimate when calling pgstat_report_analyze(). That gives us the needed data to call pgstat_report_analyze() after a successful import. The root-cause analysis is correct, but I don't think that the fix is the right way to go, because if we modified pgstat_report_analyze() to report more ANALYZE stats, we would need to modify the ImportForeignStatistics API as well, which isn't great. To avoid that, how about calling pgstat_report_analyze() in postgresImportForeignStatistics(), like the attached? (Actually, I designed the API as something we entirely replace do_analyze_rel() with.) I modified the test case as well. Sorry for the delay. Best regards, Etsuro Fujita
Re: postgres_fdw: fix cumulative stats after imported foreign-table stats
> On Jul 3, 2026, at 02:10, Etsuro Fujita wrote: > > Hi Chao, > > On Fri, Jun 12, 2026 at 12:49 PM Chao Li wrote: >> I think the root cause is that, with 28972b6fc, when stats are imported successfully for a foreign table, do_analyze_rel() is skipped. But do_analyze_rel() is the only place that calls pgstat_report_analyze() to update cumulative stats. >> >> To fix this, I think we need to add an output parameter to ImportForeignStatistics to pass out total live rows. AFAIK, the imported remote relation stats have no dead-tuple estimate, so analyze_rel() can pass 0 as the dead-tuple estimate when calling pgstat_report_analyze(). That gives us the needed data to call pgstat_report_analyze() after a successful import. > > The root-cause analysis is correct, but I don't think that the fix is > the right way to go, because if we modified pgstat_report_analyze() to > report more ANALYZE stats, we would need to modify the > ImportForeignStatistics API as well, which isn't great. To avoid > that, how about calling pgstat_report_analyze() in > postgresImportForeignStatistics(), like the attached? (Actually, I > designed the API as something we entirely replace do_analyze_rel() > with.) I modified the test case as well. > > Sorry for the delay. > > Best regards, > Etsuro Fujita > Hi Etsuro-san, Thanks for your review and for proposing an alternative fix. I thought postgres_fdw was where we interact with the remote server, while analyze_rel() is the code deciding that ANALYZE succeeded. Ideally, cumulative ANALYZE reporting would be driven from that level, not from an FDW callback implementation. But I understand your concern about changing the ImportForeignStatistics API, so I’m fine with your proposal. I have integrated your changes into v2. Best regards, -- Chao Li (Evan) HighGo Software Co., Ltd. https://www.highgo.com/
Re: postgres_fdw: fix cumulative stats after imported foreign-table stats
От:
Etsuro Fujita <etsuro.fujita@gmail.com>
Дата:
Hi Chao,
On Sat, Jul 4, 2026 at 10:24 PM Chao Li wrote:
> I thought postgres_fdw was where we interact with the remote server, while analyze_rel() is the code deciding that ANALYZE succeeded. Ideally, cumulative ANALYZE reporting would be driven from that level, not from an FDW callback implementation.
The division of labor would be arbitrary: as mentioned upthread, the
FDW callback is designed as a function corresponding to
do_analyze_rel(), which not only updates the stats system catalogs but
reports to pgstats, so I think it's appropriate for the callback to do
the report as well.
> But I understand your concern about changing the ImportForeignStatistics API, so I’m fine with your proposal. I have integrated your changes into v2.
Ok, thanks for the integration!
@@ -230,7 +230,8 @@ analyze_rel(Oid relid, RangeVar *relation,
if (fdwroutine->ImportForeignStatistics != NULL &&
fdwroutine->ImportForeignStatistics(onerel, va_cols, elevel))
stats_imported = true;
- else
+
+ if (!stats_imported)
Is this a leftover?
Best regards,
Etsuro Fujita
Re: postgres_fdw: fix cumulative stats after imported foreign-table stats
> On Jul 5, 2026, at 21:36, Etsuro Fujita wrote: > > Hi Chao, > > On Sat, Jul 4, 2026 at 10:24 PM Chao Li wrote: >> I thought postgres_fdw was where we interact with the remote server, while analyze_rel() is the code deciding that ANALYZE succeeded. Ideally, cumulative ANALYZE reporting would be driven from that level, not from an FDW callback implementation. > > The division of labor would be arbitrary: as mentioned upthread, the > FDW callback is designed as a function corresponding to > do_analyze_rel(), which not only updates the stats system catalogs but > reports to pgstats, so I think it's appropriate for the callback to do > the report as well. > >> But I understand your concern about changing the ImportForeignStatistics API, so I’m fine with your proposal. I have integrated your changes into v2. > > Ok, thanks for the integration! > > @@ -230,7 +230,8 @@ analyze_rel(Oid relid, RangeVar *relation, > if (fdwroutine->ImportForeignStatistics != NULL && > fdwroutine->ImportForeignStatistics(onerel, va_cols, elevel)) > stats_imported = true; > - else > + > + if (!stats_imported) > > Is this a leftover? > Ah, yes, that’s a leftover. Reverted that in v3. Best regards, -- Chao Li (Evan) HighGo Software Co., Ltd. https://www.highgo.com/
Re: postgres_fdw: fix cumulative stats after imported foreign-table stats
От:
Etsuro Fujita <etsuro.fujita@gmail.com>
Дата:
Hi Chao, On Mon, Jul 6, 2026 at 11:52 PM Chao Li wrote: > > @@ -230,7 +230,8 @@ analyze_rel(Oid relid, RangeVar *relation, > > if (fdwroutine->ImportForeignStatistics != NULL && > > fdwroutine->ImportForeignStatistics(onerel, va_cols, elevel)) > > stats_imported = true; > > - else > > + > > + if (!stats_imported) > > > > Is this a leftover? > > Ah, yes, that’s a leftover. Reverted that in v3. Ok, thanks for updating the patch! Committed and back-patched. Best regards, Etsuro Fujita
Re: postgres_fdw: fix cumulative stats after imported foreign-table stats
От:
Chao Li <li.evan.chao@gmail.com>
Дата:
> On Jul 7, 2026, at 02:50, Etsuro Fujita wrote: > > Hi Chao, > > On Mon, Jul 6, 2026 at 11:52 PM Chao Li wrote: >>> @@ -230,7 +230,8 @@ analyze_rel(Oid relid, RangeVar *relation, >>> if (fdwroutine->ImportForeignStatistics != NULL && >>> fdwroutine->ImportForeignStatistics(onerel, va_cols, elevel)) >>> stats_imported = true; >>> - else >>> + >>> + if (!stats_imported) >>> >>> Is this a leftover? >> >> Ah, yes, that’s a leftover. Reverted that in v3. > > Ok, thanks for updating the patch! Committed and back-patched. > > Best regards, > Etsuro Fujita Thanks for pushing. Best regards, -- Chao Li (Evan) HighGo Software Co., Ltd. https://www.highgo.com/
Re: pgsql: Change wal_compression=on to the first of zstd, lz4, pglz
On 8/12/26 01:52, Tom Lane wrote: > Tomas Vondra writes: >> Initial proposal and patch by wenhui qiu, reviews and patch adjustments >> by Christoph Berg. A number of other people participated in the >> discussion. Benchmarks by me. > >> Backpatch to 19. > > [ raised eyebrow... ] I don't object to this change in master, > but it seems pretty out of scope for v19 at this point. > Yeah. The consensus in the discussion thread was it'd be good to do this in 19, together with the TOAST compression default. I wasn't sure about that, so I consulted with the RMT and the response was it's OK. If that's not sufficient, and it's too late for 19, we can revert it. regards -- Tomas Vondra
Re: pgsql: Change wal_compression=on to the first of zstd, lz4, pglz
От:
Tom Lane <tgl@sss.pgh.pa.us>
Дата:
Tomas Vondra writes: > On 8/12/26 01:52, Tom Lane wrote: >> [ raised eyebrow... ] I don't object to this change in master, >> but it seems pretty out of scope for v19 at this point. > Yeah. The consensus in the discussion thread was it'd be good to do this > in 19, together with the TOAST compression default. I wasn't sure about > that, so I consulted with the RMT and the response was it's OK. Ah. The point about changing this along with the TOAST compression default does make sense. Objection withdrawn ... but this needs to get into the v19 release notes beside the other change. regards, tom lane