Re: Proposal: Document ABI Compatibility

On Jun 3, 2024, at 14:58, Andres Freund <andres@anarazel.de> wrote:

> Hi,

Hello Andres.

> Are there notes for the session?

Yes, but not posted yet. Here’s what Andreas 'ads' Scherbaum sent me for that bit of the conversation:

*   Core is focused on core ABI stability
*   David: No "statement of stability" in Core
    *   David/Jeremy/Tom: coding guidelines, style guidelines
    *    useful to have docs in core about what's stable and what's not, what you should compile against or not, and
ABIguarantees 
*   Abigale: there are hooks, but no overall concept for extensions
*   Tom: Peter Eisentraut is working on tests for extensions stability
*   Jeremy: nothing is preventing people from installing incompatible versions


> It'd be nice if the slides for the talk could be uploaded...

He also talked about it at Mini-Summit Five[1], for which I posted slides[2] (slides 11-14) and video[3] (starting at
29:08).

> I don't think we can really make this a hard guarantee. Yes, we try hard to
> avoid ABI breaks, but there IIRC have been a few cases over the years where
> that wasn't practical for some reason. If we have to decide between a bad bug
> and causing an ABI issue that's unlikely to affect anybody, we'll probably
> choose the ABI issue.

We can document that too, and perhaps a policy for letting people know. I thought I recalled something like this in the
past,but Rob Treat did some spelunking through change logs and found only CVEs that needed repairs by manually running
someSQL. So some sense of its rarity would also be useful. 

> Thus I am not really on board with this statement as-is.

That’s fine, we should get it to where there’s consensus on the ordering and agreement on what, exactly, it should be.

> Extensions in general can do lots of stuff, guaranteeing that bug fixes don't
> cause any problems is just not feasible.
>
> It'd be interesting to see a few examples of actual minor-version-upgrade
> extension breakages, so we can judge what caused them.

In the community Slack[4], Matthias van de Meent writes[5]:

> Citus’ pg_version_compat.h[7] where it re-implements a low-level function that was newly introduced in PG14.7. If you
buildagainst PG14.7+ headers, you may get random crashes when running on 14.6. 

I suppose it would work fine on 14.7 if compiled on 14.6 though. I suspect there aren’t many examples, though, mostly
justa lot of anxiety, and some have decided that extensions must be recompiled for every minor release in order to
avoidthe issue. StackGres[7] is one example, but I suspect Omni (Yurii’s company) may follow. 

> I don't think it's common for such new-fields-in-padding to cause problems
> when using an earlier minor PG version. For that the extension would need to
> actually rely on the presence of the new field, but typically that'd not be
> the case when we introduce a new field in a minor version.

That’s what I was thinking, so “compatibility assured only if you don’t use padding yourself” would be super helpful.

>> Unless, that is, we can provide a complete list of things not to do (like
>> make use of padding) to avoid it. Is that feasible?
>
> You can't really rely on the contents of padding, in general. So I don't think
> this is really something that needs to be called out.

Sure, probably not a problem, but if that’s the sole qualifier for making binary changes, I think it’s worth saying, as
opposedto “we don’t make any”. Something like “Only changes to padding, which you never used anyway, right?” :-) 

D

[1]: https://justatheory.com/2024/05/mini-summit-five/
[2]: https://justatheory.com/shared/extension-ecosystem-summit/omni-universally-buildable-extensions.pdf
[3]: https://youtu.be/R5ijx8IJyaM
[4]: https://pgtreats.info/slack-invite
[5]: https://postgresteam.slack.com/archives/C056ZA93H1A/p1716502630690559?thread_ts=1716500801.036709&cid=C056ZA93H1A
[6] https://github.com/citusdata/citus/blob/main/src/include/pg_version_compat.h#L236-L248
[7]: https://stackgres.io/doc/latest/intro/extensions/