Обсуждение: Proposal: Document ABI Compatibility

Hackers,

At the PGConf Unconference session on improving extension support in core, we talked quite a bit about the recent
anxietyamong extension developers about a lack of an ABI compatibility guarantee in Postgres. Yurii Rashkovskii did a
littleheader file spelunking and talked[1] about a few changes in minor version releases, including to apparent field
orderin structs. Jeremy Schneider posted the example on Twitter[2], and Peter G replied[3]: 

> You must be referring to my commit 714780dc. The new field is stored within alignment padding (though only on back
branches).Has this been tied to a known problem? 

At the Unconference, Tom Lane said that this approach is pretty well drilled into the heads of every committer, and new
onespick it up through experience. The goal, IIUC, is to never introduce binary incompatibilities into the C APIs in
minorreleases. This standard would be good to document, to let extension developers know exactly what the guarantees
are.

I’m happy to start a doc patch to add an ABI compatibility guarantee (presumably in xfunc.sgml), but want to be sure
thedetails are right, namely: 

* There are no source or binary compatibility guarantees for major releases.

* The ABI is guaranteed to change only in backward compatible ways in minor releases. If for some reason it doesn’t
it’sa bug that will need to be fixed. 

This ensures that an extension compiled against an earlier minor release will continue to work without recompilation on
laterminor releases of the same major version. 

But if I understand correctly, the use of techniques like adding a new field in padding does not mean that extensions
compiledon later minor releases will work on earlier minor releases of the same major version. Unless, that is, we can
providea complete list of things not to do (like make use of padding) to avoid it. Is that feasible? 

Are there other details it should include?

Thanks,

David

[1]: https://www.pgevents.ca/events/pgconfdev2024/schedule/session/14
[2]: https://x.com/jer_s/status/1785717368804815026
[3]: https://x.com/petervgeoghegan/status/1785720228237717627

Hi,

On 2024-06-03 14:43:17 -0400, David E. Wheeler wrote:
> At the PGConf Unconference session on improving extension support in core,
> we talked quite a bit about the recent anxiety among extension developers
> about a lack of an ABI compatibility guarantee in Postgres.

Are there notes for the session?


> Yurii Rashkovskii did a little header file spelunking and talked[1] about a
> few changes in minor version releases, including to apparent field order in
> structs.

It'd be nice if the slides for the talk could be uploaded...


> > You must be referring to my commit 714780dc. The new field is stored within alignment padding (though only on back
branches).Has this been tied to a known problem?
 
> 
> At the Unconference, Tom Lane said that this approach is pretty well drilled
> into the heads of every committer, and new ones pick it up through
> experience. The goal, IIUC, is to never introduce binary incompatibilities
> into the C APIs in minor releases. This standard would be good to document,
> to let extension developers know exactly what the guarantees are.

I don't think we can really make this a hard guarantee. Yes, we try hard to
avoid ABI breaks, but there IIRC have been a few cases over the years where
that wasn't practical for some reason. If we have to decide between a bad bug
and causing an ABI issue that's unlikely to affect anybody, we'll probably
choose the ABI issue.


> * The ABI is guaranteed to change only in backward compatible ways in minor
> releases. If for some reason it doesn’t it’s a bug that will need to be
> fixed.

Thus I am not really on board with this statement as-is.

Extensions in general can do lots of stuff, guaranteeing that bug fixes don't
cause any problems is just not feasible.

It'd be interesting to see a few examples of actual minor-version-upgrade
extension breakages, so we can judge what caused them.


> But if I understand correctly, the use of techniques like adding a new field
> in padding does not mean that extensions compiled on later minor releases
> will work on earlier minor releases of the same major version.

I don't think it's common for such new-fields-in-padding to cause problems
when using an earlier minor PG version. For that the extension would need to
actually rely on the presence of the new field, but typically that'd not be
the case when we introduce a new field in a minor version.


> Unless, that is, we can provide a complete list of things not to do (like
> make use of padding) to avoid it. Is that feasible?

You can't really rely on the contents of padding, in general. So I don't think
this is really something that needs to be called out.

Greetings,

Andres Freund

On Jun 3, 2024, at 14:58, Andres Freund <andres@anarazel.de> wrote:

> Hi,

Hello Andres.

> Are there notes for the session?

Yes, but not posted yet. Here’s what Andreas 'ads' Scherbaum sent me for that bit of the conversation:

*   Core is focused on core ABI stability
*   David: No "statement of stability" in Core
    *   David/Jeremy/Tom: coding guidelines, style guidelines
    *    useful to have docs in core about what's stable and what's not, what you should compile against or not, and
ABIguarantees 
*   Abigale: there are hooks, but no overall concept for extensions
*   Tom: Peter Eisentraut is working on tests for extensions stability
*   Jeremy: nothing is preventing people from installing incompatible versions


> It'd be nice if the slides for the talk could be uploaded...

He also talked about it at Mini-Summit Five[1], for which I posted slides[2] (slides 11-14) and video[3] (starting at
29:08).

> I don't think we can really make this a hard guarantee. Yes, we try hard to
> avoid ABI breaks, but there IIRC have been a few cases over the years where
> that wasn't practical for some reason. If we have to decide between a bad bug
> and causing an ABI issue that's unlikely to affect anybody, we'll probably
> choose the ABI issue.

We can document that too, and perhaps a policy for letting people know. I thought I recalled something like this in the
past,but Rob Treat did some spelunking through change logs and found only CVEs that needed repairs by manually running
someSQL. So some sense of its rarity would also be useful. 

> Thus I am not really on board with this statement as-is.

That’s fine, we should get it to where there’s consensus on the ordering and agreement on what, exactly, it should be.

> Extensions in general can do lots of stuff, guaranteeing that bug fixes don't
> cause any problems is just not feasible.
>
> It'd be interesting to see a few examples of actual minor-version-upgrade
> extension breakages, so we can judge what caused them.

In the community Slack[4], Matthias van de Meent writes[5]:

> Citus’ pg_version_compat.h[7] where it re-implements a low-level function that was newly introduced in PG14.7. If you
buildagainst PG14.7+ headers, you may get random crashes when running on 14.6. 

I suppose it would work fine on 14.7 if compiled on 14.6 though. I suspect there aren’t many examples, though, mostly
justa lot of anxiety, and some have decided that extensions must be recompiled for every minor release in order to
avoidthe issue. StackGres[7] is one example, but I suspect Omni (Yurii’s company) may follow. 

> I don't think it's common for such new-fields-in-padding to cause problems
> when using an earlier minor PG version. For that the extension would need to
> actually rely on the presence of the new field, but typically that'd not be
> the case when we introduce a new field in a minor version.

That’s what I was thinking, so “compatibility assured only if you don’t use padding yourself” would be super helpful.

>> Unless, that is, we can provide a complete list of things not to do (like
>> make use of padding) to avoid it. Is that feasible?
>
> You can't really rely on the contents of padding, in general. So I don't think
> this is really something that needs to be called out.

Sure, probably not a problem, but if that’s the sole qualifier for making binary changes, I think it’s worth saying, as
opposedto “we don’t make any”. Something like “Only changes to padding, which you never used anyway, right?” :-) 

D

[1]: https://justatheory.com/2024/05/mini-summit-five/
[2]: https://justatheory.com/shared/extension-ecosystem-summit/omni-universally-buildable-extensions.pdf
[3]: https://youtu.be/R5ijx8IJyaM
[4]: https://pgtreats.info/slack-invite
[5]: https://postgresteam.slack.com/archives/C056ZA93H1A/p1716502630690559?thread_ts=1716500801.036709&cid=C056ZA93H1A
[6] https://github.com/citusdata/citus/blob/main/src/include/pg_version_compat.h#L236-L248
[7]: https://stackgres.io/doc/latest/intro/extensions/

Andres Freund <andres@anarazel.de> writes:
> On 2024-06-03 14:43:17 -0400, David E. Wheeler wrote:
>> * The ABI is guaranteed to change only in backward compatible ways in minor
>> releases. If for some reason it doesn’t it’s a bug that will need to be
>> fixed.

> Thus I am not really on board with this statement as-is.

Me either.  There are degrees of ABI compatibility, and we'll choose
the least invasive way, but it's seldom the case that no conceivable
extension will be broken.  For example, if we can't squeeze a new
field into padding space, we'll typically put it at the end of the
struct in existing branches.  That's okay unless some extension has
a dependency on sizeof(the struct), for instance because it's
allocating such structs itself.  Also, for either the padding-space
or add-at-the-end approaches, we're probably in trouble if some
extension is creating its own instances of such structs, because
it will not know how to fill the new field properly.  We try not to
change structs that we think extensions are likely to create ...
but that's a guess not a guarantee.

> It'd be interesting to see a few examples of actual minor-version-upgrade
> extension breakages, so we can judge what caused them.

Yes, that could be a fruitful discussion.

> You can't really rely on the contents of padding, in general. So I don't think
> this is really something that needs to be called out.

For node structs, padding will generally be zero because we memset
them to zeroes.  So to the extent that zero is an okay value for
such a new field, that can help --- but if zero were always okay
then we'd likely not need a new field in the first place.

            regards, tom lane

> > I don't think it's common for such new-fields-in-padding to cause problems
> > when using an earlier minor PG version. For that the extension would need to
> > actually rely on the presence of the new field, but typically that'd not be
> > the case when we introduce a new field in a minor version.
>
> That’s what I was thinking, so “compatibility assured only if you don’t use padding yourself” would be super helpful.

I was under the impression that the (a?) concern was related to
compiling the newer version and using that against an older version,
so if you always compiled the extension against the latest point
release, it wouldn't necessarily be backwards-compatible with older
installations.  (While probably allocated padding is zero'd out in the
old version, I'd not guarantee that that's the case.)

> >> Unless, that is, we can provide a complete list of things not to do (like
> >> make use of padding) to avoid it. Is that feasible?
> >
> > You can't really rely on the contents of padding, in general. So I don't think
> > this is really something that needs to be called out.
>
> Sure, probably not a problem, but if that’s the sole qualifier for making binary changes, I think it’s worth saying,
asopposed to “we don’t make any”. Something like “Only changes to padding, which you never used anyway, right?” :-) 

Wonder if what might be more useful is some sort of distribution list
for extensions authors to be notified of specific compatibility
changes.  Could be powered by parsing commit messages for conventions
about backwards-incompatible changes; padding usage could be one,
maybe there are others we could code for.  (For padding, imagine a
tool that is able to look at struct offsets between git revisions and
note any offset differences/changes here that extensions authors could
be aware of; even diffs of `pahole` output between revisions could be
helpful.)

ABI guarantees for extensions are hard because all of core *is*
potentially the ABI (or at least if you are well-behaved, public
functions and structs), so some sort of "these interfaces won't break
in minor releases but use other internals and you're on your own"
might be useful distinction. (We discussed trying to classify APIs
with varying levels of support, but that also comes with its own set
of issues/overhead/maintenance for code authors/committers.)

Hi,

On 2024-06-03 15:21:04 -0400, David E. Wheeler wrote:
> > Extensions in general can do lots of stuff, guaranteeing that bug fixes don't
> > cause any problems is just not feasible.
> >
> > It'd be interesting to see a few examples of actual minor-version-upgrade
> > extension breakages, so we can judge what caused them.
>
> In the community Slack[4], Matthias van de Meent writes[5]:
>
> > Citus’ pg_version_compat.h[7] where it re-implements a low-level function that was newly introduced in PG14.7. If
youbuild against PG14.7+ headers, you may get random crashes when running on 14.6.
 

I don't see how this would trigger random crashes.

Unfortunately [4] doesn't seem to take me to a relevant message (pruned chat
history?), so I can't infer more from that context.


> I suppose it would work fine on 14.7 if compiled on 14.6 though. I suspect
> there aren’t many examples, though, mostly just a lot of anxiety, and some
> have decided that extensions must be recompiled for every minor release in
> order to avoid the issue. StackGres[7] is one example, but I suspect Omni
> (Yurii’s company) may follow.

Regardless of ABI issues, it's probably a good idea to continually run tests
against in-development minor versions, just to prevent something breaking from
creeping in. IIRC there were a handful of cases where we accidentally broke
some extension, because they relied on some implementation details.


> >> Unless, that is, we can provide a complete list of things not to do (like
> >> make use of padding) to avoid it. Is that feasible?
> >
> > You can't really rely on the contents of padding, in general. So I don't think
> > this is really something that needs to be called out.
>
> Sure, probably not a problem, but if that’s the sole qualifier for making
> binary changes, I think it’s worth saying, as opposed to “we don’t make
> any”. Something like “Only changes to padding, which you never used anyway,
> right?” :-)

IDK, to me something like this seems to promise more than we actually can.


Greetings,

Andres Freund

On Jun 3, 2024, at 5:56 PM, Andres Freund <andres@anarazel.de> wrote:

> I don't see how this would trigger random crashes.
>
> Unfortunately [4] doesn't seem to take me to a relevant message (pruned chat
> history?), so I can't infer more from that context.

You can use [4] to join the Slack (if you haven’t already) and [5] for the relevant post.

> Regardless of ABI issues, it's probably a good idea to continually run tests
> against in-development minor versions, just to prevent something breaking from
> creeping in. IIRC there were a handful of cases where we accidentally broke
> some extension, because they relied on some implementation details.

Oh yeah, I run regular tests against the latest minor release of all supported Postgres version for my extensions,
usingpgxn-tools[6], which looks like this[7]. Which I consider absolutely essential. But it doesn’t mean that something
compiledagainst .4 will work with .3 and vice versa. That’s what we could use the guidance/guarantees on. 

>> Sure, probably not a problem, but if that’s the sole qualifier for making
>> binary changes, I think it’s worth saying, as opposed to “we don’t make
>> any”. Something like “Only changes to padding, which you never used anyway,
>> right?” :-)
>
> IDK, to me something like this seems to promise more than we actually can.

What I’d like to do is figure out exactly what we *can* promise and perhaps some guidelines, and start with that.

Best,

David

[4]: https://pgtreats.info/slack-invite
[5]: https://postgresteam.slack.com/archives/C056ZA93H1A/p1716502630690559?thread_ts=1716500801.036709&cid=C056ZA93H1A
[6]: https://github.com/pgxn/docker-pgxn-tools
[7]: https://github.com/pgxn/docker-pgxn-tools/actions/runs/9351752462

On Mon, 2024-06-03 at 15:38 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2024-06-03 14:43:17 -0400, David E. Wheeler wrote:
> > > * The ABI is guaranteed to change only in backward compatible ways in minor
> > > releases. If for some reason it doesn’t it’s a bug that will need to be
> > > fixed.
>
> > Thus I am not really on board with this statement as-is.
>
> Me either.  There are degrees of ABI compatibility, and we'll choose
> the least invasive way, but it's seldom the case that no conceivable
> extension will be broken.

oracle_fdw has been broken by minor releases several times in the past.
This may well be because of weird things that I am doing; still, my
experience is that minor releases are not always binary compatible.

> > It'd be interesting to see a few examples of actual minor-version-upgrade
> > extension breakages, so we can judge what caused them.
>
> Yes, that could be a fruitful discussion.

Digging through my commits brought up 6214e2b2280462cbc3aa1986e350e167651b3905,
for one.

Yours,
Laurenz Albe

On 04.06.24 02:11, Laurenz Albe wrote:
> On Mon, 2024-06-03 at 15:38 -0400, Tom Lane wrote:
>> Andres Freund <andres@anarazel.de> writes:
>>> On 2024-06-03 14:43:17 -0400, David E. Wheeler wrote:
>>>> * The ABI is guaranteed to change only in backward compatible ways in minor
>>>> releases. If for some reason it doesn’t it’s a bug that will need to be
>>>> fixed.
>>
>>> Thus I am not really on board with this statement as-is.
>>
>> Me either.  There are degrees of ABI compatibility, and we'll choose
>> the least invasive way, but it's seldom the case that no conceivable
>> extension will be broken.
> 
> oracle_fdw has been broken by minor releases several times in the past.
> This may well be because of weird things that I am doing; still, my
> experience is that minor releases are not always binary compatible.
> 
>>> It'd be interesting to see a few examples of actual minor-version-upgrade
>>> extension breakages, so we can judge what caused them.
>>
>> Yes, that could be a fruitful discussion.
> 
> Digging through my commits brought up 6214e2b2280462cbc3aa1986e350e167651b3905,
> for one.

I'm not sure I can see how that would have broken oracle_fdw, but in any 
case it's an interesting example.  This patch did not change any structs 
incompatibly, but it changed the semantics of a function without 
changing the name:

  extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
                               Relation resultRelationDesc,
                               Index resultRelationIndex,
-                             Relation partition_root,
+                             ResultRelInfo *partition_root_rri,
                               int instrument_options);

If an extension calls this function, something would possibly crash if 
it's on the wrong side of the update fence.

This could possibly be avoided by renaming the symbol in backbranches. 
Maybe something like

#define InitResultRelInfo InitResultRelInfo2

Then you'd get a specific error message when loading the module, rather 
than a crash.

This might be something to consider:

no ABI break is better than an explicit ABI break is better than a 
silent ABI break

(Although this is actually an API break, isn't it?)

On Jun 4, 2024, at 03:18, Peter Eisentraut <peter@eisentraut.org> wrote:

> This could possibly be avoided by renaming the symbol in backbranches. Maybe something like
>
> #define InitResultRelInfo InitResultRelInfo2
>
> Then you'd get a specific error message when loading the module, rather than a crash.

That sounds more useful, yes. Is that a practice the project would consider adopting?

There’s also oracle_fdw@d137d15[1], which says:

> An API break in PostgreSQL 10.4 and 9.6.9 makes it impossible
> to use these versions: the "extract_actual_join_clauses" function
> gained an additional parameter.

The 10.4 commit is 68fab04, and it does indeed add a new function:

``` patch
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -36,6 +36,7 @@ extern List *get_actual_clauses(List *restrictinfo_list);
 extern List *extract_actual_clauses(List *restrictinfo_list,
                       bool pseudoconstant);
 extern void extract_actual_join_clauses(List *restrictinfo_list,
+                           Relids joinrelids,
                            List **joinquals,
                            List **otherquals);
 extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
```

I wonder if that sort of change could be avoided in backpatches, maybe by adding and using a
`extract_actual_join_clauses_compat`function and using that internally instead? 

Or, to David C’s point, perhaps it would be better to say there are some categories of APIs that are not subject to any
guaranteesin minor releases? 

Best,

David

[1]: https://github.com/laurenz/oracle_fdw/commit/d137d15edca8c67df1e5cccca01f417f4833b028
[2]:
https://github.com/postgres/postgres/commit/68fab04f7c2a07c5308e3d2957198ccd7a80ebc5#diff-bb6fa74cb115e19684092f0938131cd5d99b26fa2d49480f7ea7f28e937a7fb4

Hi,

On 2024-06-10 15:05:32 -0400, David E. Wheeler wrote:
> > An API break in PostgreSQL 10.4 and 9.6.9 makes it impossible
> > to use these versions: the "extract_actual_join_clauses" function
> > gained an additional parameter.
> 
> The 10.4 commit is 68fab04, and it does indeed add a new function:

That's 6 years ago, not sure we can really learn that much from that.

And it's not like it's actually impossible, #ifdefs aren't great, but they are
better than nothing.


> ``` patch
> --- a/src/include/optimizer/restrictinfo.h
> +++ b/src/include/optimizer/restrictinfo.h
> @@ -36,6 +36,7 @@ extern List *get_actual_clauses(List *restrictinfo_list);
>  extern List *extract_actual_clauses(List *restrictinfo_list,
>                        bool pseudoconstant);
>  extern void extract_actual_join_clauses(List *restrictinfo_list,
> +                           Relids joinrelids,
>                             List **joinquals,
>                             List **otherquals);
>  extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
> ```
> 
> I wonder if that sort of change could be avoided in backpatches, maybe by adding and using a
`extract_actual_join_clauses_compat`function and using that internally instead?
 
> 
> Or, to David C’s point, perhaps it would be better to say there are some categories of APIs that are not subject to
anyguarantees in minor releases?
 

I'm honestly very dubious that this is a good point to introduce a bunch of
formalism. It's a already a lot of work to maintain them, if we make it even
harder we'll end up more fixes not being backported, because it's not worth
the pain.

To be blunt, the number of examples raised here doesn't seem to indicate that
this is an area where we need to invest additional resources. We are already
severely constrained as a project by committer bandwidth, there are plenty
other things that seem more important to focus on.

Greetings,

Andres Freund

On Jun 10, 2024, at 15:39, Andres Freund <andres@anarazel.de> wrote:

> That's 6 years ago, not sure we can really learn that much from that.
>
> And it's not like it's actually impossible, #ifdefs aren't great, but they are
> better than nothing.

Right, it’s just that extension authors could use some notification that such a change is coming so they can update
theircode, if necessary. 

>> Or, to David C’s point, perhaps it would be better to say there are some categories of APIs that are not subject to
anyguarantees in minor releases? 
>
> I'm honestly very dubious that this is a good point to introduce a bunch of
> formalism. It's a already a lot of work to maintain them, if we make it even
> harder we'll end up more fixes not being backported, because it's not worth
> the pain.

Well it’s a matter of distributing the work. I don’t want to increase anyone’s workload unnecessarily, but as it is
stufflike this can be surprising to extension maintainers with some expectation of minor release stability who had no
warningof the change. That kind of thing can dissuade some people from deciding to write or maintain extensions, and
leadothers to recompile and distribute binaries for every single minor release. 

> To be blunt, the number of examples raised here doesn't seem to indicate that
> this is an area where we need to invest additional resources. We are already
> severely constrained as a project by committer bandwidth, there are plenty
> other things that seem more important to focus on.

So my question is, what’s the least onerous thing for committers to commit to doing that we can write down to properly
setexpectations? That’s where I want to start: can we publish a policy that reflects what committers already adhere to?
Andis there some way to let people know that an incompatible change is being released? Even if it just starts out in
therelease notes? 

Based on this thread, I’ve drafted the sort of policy I have in mind. Please don’t assume I’m advocating for exactly
thewording here! Let’s workshop this until it’s something the committers and core team can agree to. (At that point
I’llturn it into a doc patch) Have a look and let me know what you think. 

``` md

ABI Policy
==========

The PostgreSQL core team maintains two application binary interface (ABI) guarantees: one for major releases and one
forminor releases. 

Major Releases
--------------

Applications that use the PostgreSQL APIs must be compiled for each major release supported by the application. The
inclusionof `PG_MODULE_MAGIC` ensures that code compiled for one major version will rejected by other major versions. 

Furthermore, new releases may make API changes that require code changes. Use the `PG_VERSION_NUM` constant to adjust
codein a backwards compatible way: 

``` c
#if PG_VERSION_NUM >= 160000
#include "varatt.h"
#endif
```

PostgreSQL avoids unnecessary API changes in major releases, but usually ships a few necessary API changes, including
deprecation,renaming, and argument variation. In such cases the incompatible changes will be listed in the Release
Notes.

Minor Releases
--------------

PostgreSQL makes every effort to avoid both API and ABI breaks in minor releases. In general, an application compiled
againstany minor release will work with any other minor release, past or future. 

When a change *is* required, PostgreSQL will choose the least invasive way possible, for example by squeezing a new
fieldinto padding space or appending it to the end of a struct. This sort of change should not impact dependent
applicationsunless they use `sizeof(the struct)` or create their own instances of such structs --- patterns best
avoided.

In rare cases, however, even such non-invasive changes may be impractical or impossible. In such an event, the change
willbe documented in the Release Notes, and details on the issue will also be posted to [TBD; mail list? Blog post?
Newsitem?]. 

The project strongly recommends that developers adopt continuous integration testing at least for the latest minor
releaseall major versions of Postgres they support. 
```

Best,

David

On 11.06.24 16:55, David E. Wheeler wrote:
> On Jun 10, 2024, at 15:39, Andres Freund <andres@anarazel.de> wrote:
> 
>> That's 6 years ago, not sure we can really learn that much from that.
>>
>> And it's not like it's actually impossible, #ifdefs aren't great, but they are
>> better than nothing.
> 
> Right, it’s just that extension authors could use some notification that such a change is coming so they can update
theircode, if necessary.
 

I think since around 6 years ago we have been much more vigilant about 
avoiding ABI breaks.  So if there aren't any more recent examples of 
breakage, then maybe that was ultimately successful, and the upshot is, 
continue to be vigilant at about the same level?

On Wed, 12 Jun 2024 at 14:44, Peter Eisentraut <peter@eisentraut.org> wrote:
> I think since around 6 years ago we have been much more vigilant about
> avoiding ABI breaks.  So if there aren't any more recent examples of
> breakage, then maybe that was ultimately successful, and the upshot is,
> continue to be vigilant at about the same level?

While not strictly an ABI break I guess, the backport of 32d5a4974c81
broke building Citus against 13.10 and 14.7[1].

[1]: https://github.com/citusdata/citus/pull/6711

On Jun 12, 2024, at 8:43 AM, Peter Eisentraut <peter@eisentraut.org> wrote:

>> Right, it’s just that extension authors could use some notification that such a change is coming so they can update
theircode, if necessary. 
>
> I think since around 6 years ago we have been much more vigilant about avoiding ABI breaks.  So if there aren't any
morerecent examples of breakage, then maybe that was ultimately successful, and the upshot is, continue to be vigilant
atabout the same level? 

That sounds great to me. I’d like to get it documented, though, so that extension and other third party developers are
awareof it, and not just making wild guesses and scaring each other over (perhaps) misconceptions. 

D

On Jun 12, 2024, at 8:58 AM, Jelte Fennema-Nio <postgres@jeltef.nl> wrote:

> While not strictly an ABI break I guess, the backport of 32d5a4974c81
> broke building Citus against 13.10 and 14.7[1].
>
> [1]: https://github.com/citusdata/citus/pull/6711

Interesting one. We might want to advise projects to use deferent names if they copy code from the core, use an
extension-specificprefix perhaps. That way if it gets backported by the core, as in this example, it won’t break
anything,and the extension can choose to switch. 

D

On Mon, Jun 3, 2024 at 3:39 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Me either.  There are degrees of ABI compatibility

Exactly this!

What I think would be useful to document is our usual practices e.g.
adding new struct members at the end of structs, trying to avoid
changing public function signatures. If we document promises to
extension authors, I don't know how much difference that will make:
we'll probably end up needing to violate them at some point for one
reason or another. But if we document what committers should do, then
we might do better than we're now, because committers will be more
likely to do it right, and extension authors can also read those
instructions to understand what our practices are.

--
Robert Haas
EDB: http://www.enterprisedb.com

On Jun 12, 2024, at 10:47, Robert Haas <robertmhaas@gmail.com> wrote:

> What I think would be useful to document is our usual practices e.g.
> adding new struct members at the end of structs, trying to avoid
> changing public function signatures. If we document promises to
> extension authors, I don't know how much difference that will make:
> we'll probably end up needing to violate them at some point for one
> reason or another.

I think that’s fine if there is some sort of notification process. The policy I drafted upthread starts with making
surethe such a break is mentioned in the release notes. 

> But if we document what committers should do, then
> we might do better than we're now, because committers will be more
> likely to do it right, and extension authors can also read those
> instructions to understand what our practices are.

Yes, this, thank you!

D

On 2024-06-11 10:55:38 -0400, David E. Wheeler wrote:
> ABI Policy
> ==========
> 
> The PostgreSQL core team maintains two application binary interface (ABI) guarantees: one for major releases and one
forminor releases.
 

I.e. for major versions it's "there is none"?

> Major Releases
> --------------
> 
> Applications that use the PostgreSQL APIs must be compiled for each major release supported by the application. The
inclusionof `PG_MODULE_MAGIC` ensures that code compiled for one major version will rejected by other major versions.
 
> 
> Furthermore, new releases may make API changes that require code changes. Use the `PG_VERSION_NUM` constant to adjust
codein a backwards compatible way:
 
> 
> ``` c
> #if PG_VERSION_NUM >= 160000
> #include "varatt.h"
> #endif
> ```
> 
> PostgreSQL avoids unnecessary API changes in major releases, but usually
> ships a few necessary API changes, including deprecation, renaming, and
> argument variation.


> In such cases the incompatible changes will be listed in the Release Notes.

I don't think we actually exhaustively list all of them.


> Minor Releases
> --------------
> 
> PostgreSQL makes every effort to avoid both API and ABI breaks in minor releases. In general, an application compiled
againstany minor release will work with any other minor release, past or future.
 

s/every/a reasonable/ or just s/every/an/


> When a change *is* required, PostgreSQL will choose the least invasive way
> possible, for example by squeezing a new field into padding space or
> appending it to the end of a struct. This sort of change should not impact
> dependent applications unless they use `sizeof(the struct)` or create their
> own instances of such structs --- patterns best avoided.

The padding case doesn't affect sizeof() fwiw.

I think there's too often not an alternative to using sizeof(), potentially
indirectly (via makeNode() or such. So this sounds a bit too general.

Greetings,

Andres Freund

Hi,

On 2024-06-12 14:58:04 +0200, Jelte Fennema-Nio wrote:
> On Wed, 12 Jun 2024 at 14:44, Peter Eisentraut <peter@eisentraut.org> wrote:
> > I think since around 6 years ago we have been much more vigilant about
> > avoiding ABI breaks.  So if there aren't any more recent examples of
> > breakage, then maybe that was ultimately successful, and the upshot is,
> > continue to be vigilant at about the same level?
> 
> While not strictly an ABI break I guess, the backport of 32d5a4974c81
> broke building Citus against 13.10 and 14.7[1].

I think that kind of thing is not something we (PG devs) really can do
anything about. It's also
a) fairly easy thing to fix
b) fails during compilation
c) doesn't break ABI afaict

Greetings,

Andres Freund

On Mon, Jun 3, 2024 at 3:38 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Thus I am not really on board with this statement as-is.
>
> Me either.  There are degrees of ABI compatibility, and we'll choose
> the least invasive way, but it's seldom the case that no conceivable
> extension will be broken.  For example, if we can't squeeze a new
> field into padding space, we'll typically put it at the end of the
> struct in existing branches.  That's okay unless some extension has
> a dependency on sizeof(the struct), for instance because it's
> allocating such structs itself.

Right. While there certainly are code bases (mostly C libraries
without a huge surface area) where taking the hardest possible line on
ABI breakage makes sense, and where ABI breakage can be detected by a
mechanical process, that isn't us. In fact I'd say that Postgres is
just about the further possible thing from that.

I'm a little surprised that we don't seem to have all that many
problems with ABI breakage, though. Although we theoretically have a
huge number of APIs that extension authors might choose to use, that
isn't really true in practical terms. The universe of theoretically
possible problems is vastly larger than the areas where we see
problems in practice. You have to be pragmatic about it.

--
Peter Geoghegan

On Tue, Jun 11, 2024 at 10:55 AM David E. Wheeler <david@justatheory.com> wrote:
> Right, it’s just that extension authors could use some notification that such a change is coming so they can update
theircode, if necessary. 

In general our strategy around ABI breaks is to avoid them whenever
possible. We also make the most conservative assumptions about what a
true ABI break is -- strictly speaking we can never be fully sure
about the impact of a theoretical/mechanical ABI break (we can only
make well educated guesses). My sense is that we're approaching having
the fewest possible real ABI breaks already -- we're already doing the
best we can. That doesn't seem like a useful area to focus on.

As an example, my bugfix commit 714780dc was apparently discussed by
Yurii Rashkovskii during his pgConf.dev talk. I was (and still am)
approaching 100% certainty that that wasn't a true ABI break.
Documenting this somewhere seems rather unappealing. Strictly speaking
I'm not 100% certain that this is a non-issue, but who benefits from
hearing my hand-wavy characterisation of why I believe it's a
non-issue? You might as well just look at an ABI change report
yourself.

Approximately 0% of all extensions actually use the struct in
question, and so obviously aren't affected. If anybody is using the
struct then it's merely very very likely that they aren't affected.
But why trust me here? After all, I can't even imagine why anybody
would want to use the struct in question. My hand-wavy speculation
about what it would look like if I was wrong about that is inherently
suspect, and probably just useless. Is it not?

That having been said, it would be useful if there was a community web
resource for this -- something akin to coverage.postgresql.org, but
with differential ABI breakage reports. You can see an example report
here:

https://postgr.es/m/CAH2-Wzm-W6hSn71sUkz0Rem=qDEU7TnFmc7_jG2DjrLFef_WKQ@mail.gmail.com

Theoretically anybody can do this themselves. In practice they don't.
So something as simple as providing automated reports about ABI
changes might well move the needle here.

--
Peter Geoghegan

On 12.06.24 16:47, Robert Haas wrote:
> On Mon, Jun 3, 2024 at 3:39 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Me either.  There are degrees of ABI compatibility
> 
> Exactly this!
> 
> What I think would be useful to document is our usual practices e.g.
> adding new struct members at the end of structs, trying to avoid
> changing public function signatures. If we document promises to
> extension authors, I don't know how much difference that will make:
> we'll probably end up needing to violate them at some point for one
> reason or another. But if we document what committers should do, then
> we might do better than we're now, because committers will be more
> likely to do it right, and extension authors can also read those
> instructions to understand what our practices are.

Fun fact:  At the end of src/tools/RELEASE_CHANGES, there is some 
guidance on how to maintain ABI compatibility in *libpq*.  That used to 
be a problem.  We have come far since then.

But yes, a bit of documentation like that (maybe not in that file 
though) would make sense.

On Jun 12, 2024, at 11:20, Andres Freund <andres@anarazel.de> wrote:

>> The PostgreSQL core team maintains two application binary interface (ABI) guarantees: one for major releases and one
forminor releases. 
>
> I.e. for major versions it's "there is none"?

Is it? ISTM that there is the intention not to break things that don’t need to be broken, though that doesn’t rule out
interfaceimprovements. 

>> In such cases the incompatible changes will be listed in the Release Notes.
>
> I don't think we actually exhaustively list all of them.

Should they be? I can maybe see the argument not to for major releases. But I’ve also has the experience of a new
failureon a major release and having to go find the reason for it and the fix, often requiring the attention of someone
ona mailing list who might rather tap the “compatibility changes” sign in the latest change log. :-) 

> s/every/a reasonable/ or just s/every/an/

✅

> The padding case doesn't affect sizeof() fwiw.

✅

> I think there's too often not an alternative to using sizeof(), potentially
> indirectly (via makeNode() or such. So this sounds a bit too general.

Is there some other way to avoid the issue? Or would constructor APIs need to be added to core?


Updated with your suggestions:


``` md

ABI Policy
==========

The PostgreSQL core team maintains two application binary interface (ABI) guarantees: one for major releases and one
forminor releases. 

Major Releases
--------------

Applications that use the PostgreSQL APIs must be compiled for each major release supported by the application. The
inclusionof `PG_MODULE_MAGIC` ensures that code compiled for one major version will rejected by other major versions. 

Furthermore, new releases may make API changes that require code changes. Use the `PG_VERSION_NUM` constant to adjust
codein a backwards compatible way: 

``` c
#if PG_VERSION_NUM >= 160000
#include "varatt.h"
#endif
```

PostgreSQL avoids unnecessary API changes in major releases, but usually ships a few necessary API changes, including
deprecation,renaming, and argument variation. In such cases the incompatible changes will be listed in the Release
Notes.

Minor Releases
--------------

PostgreSQL makes an effort to avoid both API and ABI breaks in minor releases. In general, an application compiled
againstany minor release will work with any other minor release, past or future. 

When a change *is* required, PostgreSQL will choose the least invasive way possible, for example by squeezing a new
fieldinto padding space or appending it to the end of a struct. This sort of change should not impact dependent
applicationsunless, they use `sizeof(the struct)` on a struct with an appended field, or create their own instances of
suchstructs --- patterns best avoided. 

In rare cases, however, even such non-invasive changes may be impractical or impossible. In such an event, the change
willbe documented in the Release Notes, and details on the issue will also be posted to [TBD; mail list? Blog post?
Newsitem?]. 

The project strongly recommends that developers adopt continuous integration testing at least for the latest minor
releaseall major versions of Postgres they support. 
```

Best,

David

On Jun 12, 2024, at 11:30, Peter Geoghegan <pg@bowt.ie> wrote:

> I'm a little surprised that we don't seem to have all that many
> problems with ABI breakage, though. Although we theoretically have a
> huge number of APIs that extension authors might choose to use, that
> isn't really true in practical terms. The universe of theoretically
> possible problems is vastly larger than the areas where we see
> problems in practice. You have to be pragmatic about it.

Things go wrong far less often than one might fear! Given this relative stability, I think it’s reasonable to document
whatheretofore assumed the policy is so that the fears can largely be put to rest by clear expectations. 

Best,

David

On Jun 12, 2024, at 11:57, Peter Geoghegan <pg@bowt.ie> wrote:

> That having been said, it would be useful if there was a community web
> resource for this -- something akin to coverage.postgresql.org, but
> with differential ABI breakage reports. You can see an example report
> here:
>
> https://postgr.es/m/CAH2-Wzm-W6hSn71sUkz0Rem=qDEU7TnFmc7_jG2DjrLFef_WKQ@mail.gmail.com
>
> Theoretically anybody can do this themselves. In practice they don't.
> So something as simple as providing automated reports about ABI
> changes might well move the needle here.

What would be required to make such a thing? Maybe it’d make a good Summer of Code project.

Best,

David

On 03/06/2024 21:21, David E. Wheeler wrote:
> On Jun 3, 2024, at 14:58, Andres Freund <andres@anarazel.de> wrote:
>
>> Hi,
> Hello Andres.
>
>> Are there notes for the session?
> Yes, but not posted yet. Here’s what Andreas 'ads' Scherbaum sent me for that bit of the conversation:
>
> *   Core is focused on core ABI stability
> *   David: No "statement of stability" in Core
>      *   David/Jeremy/Tom: coding guidelines, style guidelines
>      *    useful to have docs in core about what's stable and what's not, what you should compile against or not, and
ABIguarantees
 
> *   Abigale: there are hooks, but no overall concept for extensions
> *   Tom: Peter Eisentraut is working on tests for extensions stability
> *   Jeremy: nothing is preventing people from installing incompatible versions

The full "discussion" is here:

https://wiki.postgresql.org/wiki/PGConf.dev_2024_Developer_Unconference#Improving_extensions_in_core

And the ABI discussion here:
https://wiki.postgresql.org/wiki/PGConf.dev_2024_Extension_Summit#ABI.2FAPI_discussion

-- 
                Andreas 'ads' Scherbaum
German PostgreSQL User Group
European PostgreSQL User Group - Board of Directors
Volunteer Regional Contact, Germany - PostgreSQL Project

On 18.06.24 00:37, David E. Wheeler wrote:
> ABI Policy
> ==========
> 
> The PostgreSQL core team maintains two application binary interface (ABI) guarantees: one for major releases and one
forminor releases.
 
> 
> Major Releases
> --------------
> 
> Applications that use the PostgreSQL APIs

This is probably a bit confusing.  This might as well mean client 
application code against libpq.  Better something like "server plugin 
code that uses the PostgreSQL server APIs".

> must be compiled for each major release supported by the application. The inclusion of `PG_MODULE_MAGIC` ensures that
codecompiled for one major version will rejected by other major versions.
 

ok so far

> Furthermore, new releases may make API changes that require code changes. Use the `PG_VERSION_NUM` constant to adjust
codein a backwards compatible way:
 
> 
> ``` c
> #if PG_VERSION_NUM >= 160000
> #include "varatt.h"
> #endif
> ```

But now we're talking about API.  That might be subject of another 
document or another section in this one, but it seems confusing to mix 
this with the ABI discussion.

> PostgreSQL avoids unnecessary API changes in major releases, but usually ships a few necessary API changes, including
deprecation,renaming, and argument variation.
 

Obviously, as a practical matter, there won't be random pointless 
changes.  But I wouldn't go as far as writing anything down about how 
these APIs are developed.

> In such cases the incompatible changes will be listed in the Release Notes.

I don't think anyone is signing up to do that.


> Minor Releases
> --------------
> 
> PostgreSQL makes an effort to avoid both API and ABI breaks in minor releases. In general, an application compiled
againstany minor release will work with any other minor release, past or future.
 
> 
> When a change *is* required, PostgreSQL will choose the least invasive way possible, for example by squeezing a new
fieldinto padding space or appending it to the end of a struct. This sort of change should not impact dependent
applicationsunless, they use `sizeof(the struct)` on a struct with an appended field, or create their own instances of
suchstructs --- patterns best avoided.
 
> 
> In rare cases, however, even such non-invasive changes may be impractical or impossible. In such an event, the change
willbe documented in the Release Notes, and details on the issue will also be posted to [TBD; mail list? Blog post?
Newsitem?].
 

I think one major problem besides actively avoiding or managing such 
minor-version ABI breaks is some automated detection.  Otherwise, this 
just means "we try, but who knows".

On 18.06.24 00:40, David E. Wheeler wrote:
> On Jun 12, 2024, at 11:57, Peter Geoghegan <pg@bowt.ie> wrote:
> 
>> That having been said, it would be useful if there was a community web
>> resource for this -- something akin to coverage.postgresql.org, but
>> with differential ABI breakage reports. You can see an example report
>> here:
>>
>> https://postgr.es/m/CAH2-Wzm-W6hSn71sUkz0Rem=qDEU7TnFmc7_jG2DjrLFef_WKQ@mail.gmail.com
>>
>> Theoretically anybody can do this themselves. In practice they don't.
>> So something as simple as providing automated reports about ABI
>> changes might well move the needle here.
> 
> What would be required to make such a thing? Maybe it’d make a good Summer of Code project.

The above thread contains a lengthy discussion about what one could do.

On Mon, Jun 17, 2024 at 6:38 PM David E. Wheeler <david@justatheory.com> wrote:
> Is it? ISTM that there is the intention not to break things that don’t need to be broken, though that doesn’t rule
outinterface improvements. 

I suppose that it's true that we try to avoid gratuitous breakage, but
I feel like it would be weird to document that.

Sometimes I go to a store and I see a sign that says "shoplifters will
be prosecuted." But I have yet to see a store with a sign that says
"people who appear to be doing absolutely nothing wrong will not be
prosecuted." If I did see such a sign, I would frankly be a little
concerned.

--
Robert Haas
EDB: http://www.enterprisedb.com

On Jun 19, 2024, at 05:42, Peter Eisentraut <peter@eisentraut.org> wrote:

>>> https://postgr.es/m/CAH2-Wzm-W6hSn71sUkz0Rem=qDEU7TnFmc7_jG2DjrLFef_WKQ@mail.gmail.com
>>>
>>> Theoretically anybody can do this themselves. In practice they don't.
>>> So something as simple as providing automated reports about ABI
>>> changes might well move the needle here.
>> What would be required to make such a thing? Maybe it’d make a good Summer of Code project.
>
> The above thread contains a lengthy discussion about what one could do.

I somehow missed that GSoC 2024 is already going with contributors. Making a mental note to add an item for 2025.

D

On Jun 24, 2024, at 14:51, Robert Haas <robertmhaas@gmail.com> wrote:

> I suppose that it's true that we try to avoid gratuitous breakage, but
> I feel like it would be weird to document that.

I see how that can seem weird to a committer deeply familiar with the development process and how things happen. But
peopleoutside the -hackers bubble have very little idea. It’s fair to say it needn’t be a long statement for major
versions:a single sentence such as “we try to avoid gratuitous breakage” is a perfectly reasonable framing. But I’d
say,in the interest of completeness, it would be useful to document the policy for major release *as well as* minor
releases.

Best,

David

On Jun 19, 2024, at 05:41, Peter Eisentraut <peter@eisentraut.org> wrote:

> This is probably a bit confusing.  This might as well mean client application code against libpq.  Better something
like"server plugin code that uses the PostgreSQL server APIs". 

That works.

> But now we're talking about API.  That might be subject of another document or another section in this one, but it
seemsconfusing to mix this with the ABI discussion. 

Hrm. They’re super closely-related in my mind, as an extension developer. I need to know both! I guess I’m taking of
thispolicy as what I can expect may be changed (and how to adapt to it) and what won’t. 

That said, I’m fine to remove the API stuff if there’s consensus objecting to it, to be defined in a separate policy
(perhapson the same doc page). 

>> PostgreSQL avoids unnecessary API changes in major releases, but usually ships a few necessary API changes,
includingdeprecation, renaming, and argument variation. 
>
> Obviously, as a practical matter, there won't be random pointless changes.  But I wouldn't go as far as writing
anythingdown about how these APIs are developed. 

Fair enough, was trying to give some idea of the sorts of changes. Don’t have to include them.

>> In such cases the incompatible changes will be listed in the Release Notes.
>
> I don't think anyone is signing up to do that.

It needn’t be comprehensive. Just mention that an ABI or API changed in the release note item. Unless they almost *all*
makesuch changes. 

>> Minor Releases
>> --------------

> I think one major problem besides actively avoiding or managing such minor-version ABI breaks is some automated
detection. Otherwise, this just means "we try, but who knows”. 

I think you *do* try, and the fact that there are so few issues means you succeed at that. I’m not advocating for an
ABIguarantee here, just a description of the policy committees already follow. 

Here’s an update based on all the feedback, framing things more from the perspective of “do I need to recompile this or
changemy code”. Many thanks! 

``` md
ABI Policy
==========

Changes to the the PostgreSQL server APIs may require recompilation of server plugin code that uses them. This policy
describesthe core team's approach to such changes, and what server API users can expect. 

Major Releases
--------------

Applications that use server APIs must be compiled for each major release supported by the application. The inclusion
of`PG_MODULE_MAGIC` ensures that code compiled for one major version will rejected by other major versions. Developers
needingto support multiple versions of PostgreSQL with incompatible APIs should use the `PG_VERSION_NUM` constant to
adjustcode as appropriate. For example: 

``` c
#if PG_VERSION_NUM >= 160000
#include "varatt.h"
#endif
```

The core team avoids unnecessary breakage, but users of the server APIs should expect and be prepared to make
adjustmentsand recompile for every major release. 

Minor Releases
--------------

PostgreSQL makes an effort to avoid server API and ABI breaks in minor releases. In general, an application compiled
againstany minor release will work with any other minor release, past or future. In the absence of automated detection
ofsuch changes, this is not a guarantee, but history such breaking changes have been extremely rare. 

When a change *is* required, PostgreSQL will choose the least invasive change possible, for example by squeezing a new
fieldinto padding space or appending it to the end of a struct. This sort of change should not impact dependent
applicationsunless they use `sizeof(the struct)` on a struct with an appended field, or create their own instances of
suchstructs --- patterns best avoided. 

In rare cases, however, even such non-invasive changes may be impractical or impossible. In such an event, the change
willbe documented in the Release Notes, and details on the issue will also be posted to [TBD; mail list? Blog post?
Newsitem?]. 

To minimize issues and catch changes early, the project strongly recommends that developers adopt continuous
integrationtesting at least for the latest minor release all major versions of Postgres they support. 
```


Best,

David

On 24.06.24 22:26, David E. Wheeler wrote:
>> But now we're talking about API.  That might be subject of another document or another section in this one, but it
seemsconfusing to mix this with the ABI discussion.
 
> Hrm. They’re super closely-related in my mind, as an extension developer. I need to know both! I guess I’m taking of
thispolicy as what I can expect may be changed (and how to adapt to it) and what won’t.
 
> 
> That said, I’m fine to remove the API stuff if there’s consensus objecting to it, to be defined in a separate policy
(perhapson the same doc page).
 

I took at a stab at this, using some of your text, but discussing API 
and ABI separately.


# Server API and ABI guidance

This section contains guidance to authors of extensions and other
server plugins about ABI and API stability in the PostgreSQL server.

## General

The PostgreSQL server contains several well-delimited APIs for server
plugins, such as the function manager (fmgr), SPI, and various hooks
specifically designed for extensions.  These interfaces are carefully
managed for long-term stability and compatibility.  However, the
entire set of global functions and variables in the server effectively
constitutes the publicly usable API, but most parts of that were not
designed with extensibility and long-term stability in mind.  That
means, while taking advantage of these interfaces is valid, the
further one strays from the well-trodden path, the likelier it will be
that one might encounter ABI or API compatibility issues at some
point.  Extension authors are also encouraged to provide feedback
about their requirements, so that over time, as new use patterns
arise, certain interfaces can be consider more stabilized or new
better-designed interfaces for new uses can be added.

## API compatibility

(API = application programming interface, meaning the interface used
at compile time)

### Major versions

There is _no_ promise of API compatibility between PostgreSQL major
versions.  That means, extension code might require source code
changes to work with multiple major versions.  These can usually be
managed with preprocessor conditions like `#if PG_VERSION_NUM >=
160000`.  Sophisticated extensions that use interfaces beyond the
well-delimited ones usually require a few such changes for each major
server version.

### Minor versions

PostgreSQL makes an effort to avoid server API breaks in minor
releases.  In general, extension code that compiles and works with
some minor release should also compile and work with any other minor
release, past or future.

When a change *is* required, this will be carefully managed, taking
the requirements of extensions into account.  Such changes will be
communicated in the release notes.

## ABI compatibility

(ABI = application binary interface, meaning the interface used at run
time)

### Major versions

Servers of different major versions have intentionally incompatible
ABIs.  That means, extensions that use server APIs must be re-compiled
for each major release.  The inclusion of `PG_MODULE_MAGIC` ensures
that code compiled for one major version will be rejected by other
major versions.

### Minor versions

PostgreSQL makes an effort to avoid server ABI breaks in minor
releases.  In general, an extension compiled against any minor release
should work with any other minor release, past or future.

When a change *is* required, PostgreSQL will choose the least invasive
change possible, for example by squeezing a new field into padding
space or appending it to the end of a struct.  These sorts of changes
should not impact extensions unless they use very unusual code
patterns.

In rare cases, however, even such non-invasive changes may be
impractical or impossible.  In such an event, the change will be
carefully managed, taking the requirements of extensions into account.
Such changes will also be documented in the release notes.

Note, however, again that many parts of the server are not designed or
maintained as publicly-consumable APIs (and that, in most cases, the
actual boundary is also not well-defined).  If urgent needs arise,
changes in those parts will naturally be done with less consideration
for extension code than changes in well-defined and widely used
interfaces.

Also, in the absence of automated detection of such changes, this is
not a guarantee, but historically such breaking changes have been
extremely rare.

Peter Eisentraut <peter@eisentraut.org> writes:

> On 24.06.24 22:26, David E. Wheeler wrote:
>>> But now we're talking about API.  That might be subject of another
>> document or another section in this one, but it seems confusing to mix
>> this with the ABI discussion.
>> Hrm. They’re super closely-related in my mind, as an extension
>> developer. I need to know both! I guess I’m taking of this policy as
>> what I can expect may be changed (and how to adapt to it) and what
>> won’t.
>> That said, I’m fine to remove the API stuff if there’s consensus
>> objecting to it, to be defined in a separate policy (perhaps on the
>> same doc page).
>
> I took at a stab at this, using some of your text, but discussing API
> and ABI separately.

This looks good to me, just one minor nitpick:

> ### Minor versions
>
> PostgreSQL makes an effort to avoid server API breaks in minor
> releases.  In general, extension code that compiles and works with
> some minor release should also compile and work with any other minor
> release, past or future.

I think this should explicitly say "any other minor release within [or
"from" or "of"?] the same major version" (and ditto in the ABI section).

- ilmari

On Jun 25, 2024, at 7:33 AM, Peter Eisentraut <peter@eisentraut.org> wrote:

> I took at a stab at this, using some of your text, but discussing API and ABI separately.

Oh man this is fantastic, thank you! I’d be more than happy to just turn this into a patch. But where should it go?
UpthreadI assumed xfunc.sgml, and still think that’s a likely candidate. Perhaps I’ll just start there --- unless
someonethinks it should go somewhere other than the docs. 

Best,

David

On Tue, 2024-06-25 at 13:55 -0400, David E. Wheeler wrote:
> On Jun 25, 2024, at 7:33 AM, Peter Eisentraut <peter@eisentraut.org> wrote:
>
> > I took at a stab at this, using some of your text, but discussing API and ABI separately.
>
> Oh man this is fantastic, thank you! I’d be more than happy to just turn this into a patch.
> But where should it go? Upthread I assumed xfunc.sgml, and still think that’s a likely
> candidate. Perhaps I’ll just start there --- unless someone thinks it should go somewhere
> other than the docs.

Perhaps such information should go somewhere here:
https://www.postgresql.org/support/versioning/

Yours,
Laurenz Albe

On Jun 26, 2024, at 04:48, Laurenz Albe <laurenz.albe@cybertec.at> wrote:

> Perhaps such information should go somewhere here:
> https://www.postgresql.org/support/versioning/

This seems deeper and more detailed than what’s there now, but I can certainly imagine wanting to include this policy
onthe web site. That said, it didn’t occur to me to look under support when trying to find a place to put this; I was
lookingunder Developers, on the principle that extension developers would look there. 

Best,

David

On Jun 25, 2024, at 13:55, David E. Wheeler <david@justatheory.com> wrote:

> Oh man this is fantastic, thank you! I’d be more than happy to just turn this into a patch. But where should it go?
UpthreadI assumed xfunc.sgml, and still think that’s a likely candidate. Perhaps I’ll just start there --- unless
someonethinks it should go somewhere other than the docs. 

Okay here’s a patch that adds the proposed API and ABI guidance to the C Language docs. The content is the same as
Peterproposed, with some light copy-editing. 

Best,

David

On Jun 26, 2024, at 15:14, David E. Wheeler <david@justatheory.com> wrote:

> Okay here’s a patch that adds the proposed API and ABI guidance to the C Language docs. The content is the same as
Peterproposed, with some light copy-editing. 

CF: https://commitfest.postgresql.org/48/5080/
PR: https://github.com/theory/postgres/pull/6

D

On Jun 26, 2024, at 15:20, David E. Wheeler <david@justatheory.com> wrote:

> CF: https://commitfest.postgresql.org/48/5080/
> PR: https://github.com/theory/postgres/pull/6

Aaaand v2 without the unnecessary formatting of unrelated documentation 🤦🏻‍♂️.

Best,

David

On 6/26/24 12:23 PM, David E. Wheeler wrote:
> On Jun 26, 2024, at 15:20, David E. Wheeler <david@justatheory.com> wrote:
> 
>> CF: https://commitfest.postgresql.org/48/5080/
>> PR: https://github.com/theory/postgres/pull/6
> 
> Aaaand v2 without the unnecessary formatting of unrelated documentation 🤦🏻‍♂️.

Minor nit - misspelled "considerd"

-Jeremy


-- 
http://about.me/jeremy_schneider

On Jun 27, 2024, at 17:48, Jeremy Schneider <schneider@ardentperf.com> wrote:

> Minor nit - misspelled “considerd"

Thank you, Jeremy. V3 attached.

Best,

David