Обсуждение: Toasted column values during replication
Hi PG Users,
We are running into the postgres toasted values showing up in replication slots. We are using debezium and have gone through the following
1. Various options for handling toasted values during replication at the application level.
2. Postgres option to turn on replica identitiy to full.
While these are solutions to the problem. Wanted to understand if there is any scope for enhancement / improvements in postgres itself to handle such scenarios.
For instance : Can the postgres internal component (wal-sender / *) interpret this special toasted-col-value sitting in the wal-file -> replace it with the actual value and send it to the replication slot. I presume, today the wal-sender does not have this support and it just sends whatever data that the wal log has. So if the wal-sender - can intercept such toasted values and translate it to actual values before being sent - that would solve the problem without every consumer having to implement a solution to handle toasted value. This would also help prevent the unwarranted increase of the wal-log if replica-identity to full is set as a w/a.
Any thoughts?
Thanks.
On Fri, 2023-06-30 at 17:06 +0000, Chandy G wrote: > For instance : Can the postgres internal component (wal-sender / *) interpret this special > toasted-col-value sitting in the wal-file -> replace it with the actual value and send it to > the replication slot. Which logical decoding plugin are you using? "pgoutput", which is provided by PostgreSQL, will surely emit properly detoasted values. Yours, Laurenz Albe
Using pgoutput - aws rds 13* version and seeing this behavior when using it with debezium 2.0.0 version.
Best.
On Friday, 30 June, 2023 at 12:53:20 pm GMT-7, Laurenz Albe <laurenz.albe@cybertec.at> wrote:
On Fri, 2023-06-30 at 17:06 +0000, Chandy G wrote:
> For instance : Can the postgres internal component (wal-sender / *) interpret this special
> toasted-col-value sitting in the wal-file -> replace it with the actual value and send it to
> the replication slot.
Which logical decoding plugin are you using?
"pgoutput", which is provided by PostgreSQL, will surely emit properly detoasted values.
Yours,
Laurenz Albe
> For instance : Can the postgres internal component (wal-sender / *) interpret this special
> toasted-col-value sitting in the wal-file -> replace it with the actual value and send it to
> the replication slot.
Which logical decoding plugin are you using?
"pgoutput", which is provided by PostgreSQL, will surely emit properly detoasted values.
Yours,
Laurenz Albe