Re: RFC: PostgreSQL Storage I/O Transformation Hooks

> Could you point me to where I can access the SMGR extensibility patch?
> I'd like to review it properly before any further discussion.

There's the hadkers discussion thread[1], the PG18 branch of our fork
[2] (it has less than <20 commits on top of PG 18.1, smgr/aio changes
are easy to find), and of course you can look at how pg_tde uses
it[3].

But please note that none of them is 100% up to date. The hackers
thread is for PG17 (no AIO part yet). And we also had some in person
discussions about the patch during PgConf.Eu, which is not yet
reflected even in our fork. We plan to update the mailing list thread
in January.

> Though I wonder if WAL encryption should be part of the same
> discussion, or separate. SMGR handles pages, but WAL has different
> characteristics.

I think we should keep it separate, the SMGR question is much simpler than WAL.

> Do you think this is a reasonable direction? Or would you prefer a
> different approach?

I have no preferred approach for WAL yet. Our solution in pg_tde has
some good and bad points, and the approach you used here similarly has
some good and bad. The main reason why we kept delaying opening a
"let's add WAL hooks" discussion on the mailing list is because we
weren't confident enough in our current approach. Is it good for a
fork? Definitely. Is it good enough for getting it accepted into the
core? Probably not.

Personally I tried to come up with an approach that could be useful
for something else other than tde, including some proof of concept
implementation of that something. (for example wal compression, or
enabling an extension to split wal into separate streams for each
database) But that's not easy to do, I didn't spend too much time on
it so far, and maybe not even necessary, maybe simpler is better in
this case.

Starting a discussion about it is definitely a good idea, but maybe
the focus should be on debating/trying out different approaches
instead of proposing specific solutions to be included in pg? From
this point it is great that your implementation is different, because
we can talk about pros/cons, maybe figure out something even better?

[1]: https://www.postgresql.org/message-id/flat/CAEze2WgMySu2suO_TLvFyGY3URa4mAx22WeoEicnK%3DPCNWEMrA%40mail.gmail.com
[2]: https://github.com/percona/postgres/commits/PSP_REL_18_STABLE/
[3]: https://github.com/percona/pg_tde/blob/main/src/smgr/pg_tde_smgr.c