Re: Report bytes and transactions actually sent downtream

On Mon, Nov 3, 2025 at 12:23 PM Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:
>
> On Thu, Oct 30, 2025 at 9:08 AM shveta malik <shveta.malik@gmail.com> wrote:
>
> > >
> >
> > > +                * not filtered out, we don't count modify filteredBytes. The filtered
> >
> > Something is wrong in this sentence.
>
> :), here's better one
>
> /*
> * Even if we filtered out some relations, we still send a TRUNCATE
> * message for the remaining relations. Since the change, as a whole, is
> * not filtered out we don't increment filteredBytes. The filtered
> * out relations will be reflected as a smaller sentBytes count.
> */
>
> >
> > Also, regarding "The filtered out relations will be reflected as a
> > smaller sentBytes count."
> > Can you please point me to the code where it happens? From what I have
> > understood, pgoutput_truncate() completely skips the relations which
> > do not support publishing truncate. Then it sends 'BEGIN',  then
> > schema info of non-filtered relations and then TRUNCATE for
> > non-filtered relations (based on nrelids).
>
> Let's take an example. Assume the TRUNCATE WAL record had relids X, Y,
> Z and W. Out of those X and Y were filtered out. Then the message sent
> to the downstream will have only Z, W, let's say "TRUNCATE Z W" - 12
> bytes (hypothetically). So sentBytes will be incremented by 12.
> However, if no relation was filtered, the message would be "TRUNCATE X
> Y Z W" ~ 16 bytes and thus sentBytes will be incremented by 16 bytes.
> Thus when the relations are filtered from the truncate message,
> sentBytes is incremented by a smaller value than those when no
> relations are filtered. So, even if filteredBytes is same in both
> cases (filtered some relations vs no relation was filtered), sentBytes
> indicates the difference.

I understand the point, but I didn’t find the message clearly reflecting it.

> Similarly for column level filtering.
> However, reading this again, it seems adding more confusion than
> reducing it.

Right.

> So I propose to just add comment
>
> in pgoutput_truncate()
> /*
> * Even if we filtered out some relations, we still send a TRUNCATE
> * message for the remaining relations. Since the change, as a whole, is
> * not filtered out we don't increment filteredBytes.
> */
>
> and in pgoutput_change
> /*
> * Send the data. Even if we end up filtering some columns while sending the
> * message, we won't consider the change, as a whole, to be filtered out. Hence
> * won't increment the filteredBytes.
> */
>
> Does that look good?

Yes. Works for me.

thanks
Shveta