Re: Proposal: Deferred Replica Filtering for PostgreSQL Logical Replication

On Sun, Mar 16, 2025 at 12:59 AM Dean <ds.blue797@gmail.com> wrote:
>
> I'd like to propose an enhancement to PostgreSQL's logical replication system: Deferred Replica Filtering (DRF). The
goalof this feature is to provide more granular control over which rows are replicated by applying publication filters
afterthe WAL decoding process, before sending data to subscribers. 
>
> Currently, PostgreSQL's logical replication filters apply deterministically. Deferred filtering, however, operates
afterthe WAL has been decoded, giving it access to the complete row data and making filtering decisions based on
mutablevalues. Additionally, record columns may be omitted by the filter. 
>
> This opens up several possibilities for granular control. Consider the following examples:
> Alice and Bob subscribe to changes on a table with RLS enabled, allowing CRUD operations based on user's IDs.
> 1. Alice needs to know the timestamp at which Bob updated the table. With DRF, we can omit all columns except for the
timestamp.
> 2. Bob wants to track DELETEs on the table. Without DRF, Bob can see all columns on any deleted row, potentially
exposingcomplete records he shouldn't be authorized to view. DRF can filter these rows out. 
>
> Deferred replica filtering allows for session-specific, per-row, and per-column filtering - features currently not
supportedby existing replication filters, enhancing security and data privacy. 
>

We provide column lists [1] and row filters [2]. Doesn't that suffice
the need, if not, kindly let us know what exactly you need with some
examples.

[1] - https://www.postgresql.org/docs/devel/logical-replication-col-lists.html
[2] - https://www.postgresql.org/docs/devel/logical-replication-row-filter.html

--
With Regards,
Amit Kapila.