Обсуждение: Question and suggestion about application binary compatibility policy

Hello,

I'd like to ask you about the policy of application binary compatibility.  And have a suggestion as well.

QUESTION
==================================================

My customer asked me if the following usage is correct.

- Build an embedded SQL C application with PostgreSQL 9.2.
- Distribute the app without ecpg nor libpq libraries.  Require users to install PostgreSQL client which includes ecpg
andlibpq libraries.
 
- Use the app with newer PostgreSQL major versions without rebuilding the app.  That is, the users just replaces the
PostgreSQLclient.
 

I expect this is legal, because the so_major versions of ecpg and libpq are 6 and 5 respectively for all PostgreSQL 9.x
versionsso far.  However, I could not find any description of this binary compatibility policy in the manual, so I
haven'tbeen able to answer the customer.
 

What's the official policy of application binary compatibility?  I found the same discussion in the below thread, but
I'mafraid any clear answer wasn't given:
 

https://www.postgresql.org/message-id/522F0B6B.1040006@4js.com


SUGGESTION
==================================================

How about adding an article about application binary compatibility in the following section, as well as chapters for
libpq,ECPG, etc?
 

17.6. Upgrading a PostgreSQL Clust
https://www.postgresql.org/docs/devel/static/upgrading.html

There are three kinds of application assets that users are concerned about when upgrading.  Are there anything else to
mention?

* libpq app
* ECPG app
* C-language user defined function (and other stuff dependent on it, such as extensions, UDTs, etc.)

Regards
Takayuki Tsunakawa

Hi,

> - Build an embedded SQL C application with PostgreSQL 9.2.
> - Distribute the app without ecpg nor libpq libraries.  Require users
> to install PostgreSQL client which includes ecpg and libpq libraries.
> - Use the app with newer PostgreSQL major versions without rebuilding
> the app.  That is, the users just replaces the PostgreSQL client.
> 
> I expect this is legal, because the so_major versions of ecpg and
> libpq are 6 and 5 respectively for all PostgreSQL 9.x versions so
> far.  However, I could not find any description of this binary
> compatibility policy in the manual, so I haven't been able to answer
> the customer.

Sorry I fail to understand what you mean with "legal"? Are you
wondering about license restrictions? There are none.

As for compatibility, that's what major version numbers are for. This
is not a PostgreSQL topic, but a general system one as for instance the
same holds for libc.

Michael
-- 
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Meskes at (Debian|Postgresql) dot Org
Jabber: michael at xmpp dot meskes dot org
VfL Borussia! Força Barça! SF 49ers! Use Debian GNU/Linux, PostgreSQL

> From: pgsql-hackers-owner@postgresql.org
> Sorry I fail to understand what you mean with "legal"? Are you wondering
> about license restrictions? There are none.

Sorry, I just meant "correct" or "valid".


> As for compatibility, that's what major version numbers are for. This is
> not a PostgreSQL topic, but a general system one as for instance the same
> holds for libc.

Yes, but Windows users probably don't understand or know it.  So, I suggested explicitly describing the application
binarycompatibility policy in the PostgreSQL manual.  What do you think about it?
 


Regards
Takayuki Tsunakawa

> > Sorry I fail to understand what you mean with "legal"? Are you
> > wondering
> > about license restrictions? There are none.
> 
> Sorry, I just meant "correct" or "valid".

Ah ok, thanks for clarifying.

> > As for compatibility, that's what major version numbers are for.
> > This is
> > not a PostgreSQL topic, but a general system one as for instance
> > the same
> > holds for libc.
> 
> Yes, but Windows users probably don't understand or know it.  So, I
> suggested explicitly describing the application binary compatibility
> policy in the PostgreSQL manual.  What do you think about it?

Couldn't you point your customer to the system documentation?

Personally I don't think standard system behavior should be documented
for each application relying on it but ymmv.

Michael
-- 
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Meskes at (Debian|Postgresql) dot Org
Jabber: michael at xmpp dot meskes dot org
VfL Borussia! Força Barça! SF 49ers! Use Debian GNU/Linux, PostgreSQL

> From: Michael Meskes [mailto:meskes@postgresql.org]
> > Yes, but Windows users probably don't understand or know it.  So, I
> > suggested explicitly describing the application binary compatibility
> > policy in the PostgreSQL manual.  What do you think about it?
> 
> Couldn't you point your customer to the system documentation?
> 
> Personally I don't think standard system behavior should be documented for
> each application relying on it but ymmv.

I couldn't find appropriate system documentation.  Regarding Linux, I remember I saw some HOWTO on tldp.org website
whichexplains the concept of shared library soname, but it's not very friendly for users who just want to know the
applicationbinary compatibility policy of PostgreSQL.  And I don't think there's suitable documentation on Windows.
Evenif there is any, users will not be sure whether PostgreSQL follows those platform-specific conventions.  They may
havedoubts about it, because even the product version "PostgreSQL x.y.z" causes misconception that x is the major
versionand y is the minor one.
 

So, I suggested documenting the compatibility policy for clarification and user friendliness as in the Oracle Database
documentationbelow.
 

http://docs.oracle.com/database/121/UPGRD/app.htm#UPGRD12547


BTW, although this may be a separate topic, it may be better that we add the major version in the library names like
libpq5.dlland libecpg6.dll, so that the application can fail to run with the incompatible versions of libpq and
libecpg. FYI:
 

https://en.wikipedia.org/wiki/Side-by-side_assembly

[Excerpt]
Microsoft Visual C++ 2005 and 2008 employ SxS with all C runtime libraries. However, runtime libraries in Visual C++
2010no longer use this technology; instead, they include the version number of a DLL in its file name, which means that
differentversions of one DLL will technically be completely different DLLs now.
 


Any comments on these?  If there's no strong objection, I think I'll submit a documentation patch in the future.

Regards
Takayuki Tsunakawa

On 31/05/2016 08:10, Tsunakawa, Takayuki wrote:
>> From: Michael Meskes [mailto:meskes@postgresql.org]
>>> Yes, but Windows users probably don't understand or know it.  So, I
>>> suggested explicitly describing the application binary compatibility
>>> policy in the PostgreSQL manual.  What do you think about it?
>>
>> Couldn't you point your customer to the system documentation?
>>
>> Personally I don't think standard system behavior should be documented for
>> each application relying on it but ymmv.
>
> I couldn't find appropriate system documentation.  Regarding Linux, I remember I saw some HOWTO on tldp.org website
whichexplains the concept of shared library soname, but it's not very friendly for users who just want to know the
applicationbinary compatibility policy of PostgreSQL.  And I don't think there's suitable documentation on Windows.
Evenif there is any, users will not be sure whether PostgreSQL follows those platform-specific conventions.  They may
havedoubts about it, because even the product version "PostgreSQL x.y.z" causes misconception that x is the major
versionand y is the minor one. 
>
> So, I suggested documenting the compatibility policy for clarification and user friendliness as in the Oracle
Databasedocumentation below. 
>
> http://docs.oracle.com/database/121/UPGRD/app.htm#UPGRD12547
>
>
> BTW, although this may be a separate topic, it may be better that we add the major version in the library names like
libpq5.dlland libecpg6.dll, so that the application can fail to run with the incompatible versions of libpq and
libecpg. FYI: 
>
> https://en.wikipedia.org/wiki/Side-by-side_assembly
>
> [Excerpt]
> Microsoft Visual C++ 2005 and 2008 employ SxS with all C runtime libraries. However, runtime libraries in Visual C++
2010no longer use this technology; instead, they include the version number of a DLL in its file name, which means that
differentversions of one DLL will technically be completely different DLLs now. 
>
>
> Any comments on these?  If there's no strong objection, I think I'll submit a documentation patch in the future.
>
> Regards
> Takayuki Tsunakawa
>

Hi,
on cygwin the postgresql binary package already include
the library versions:

   /usr/bin/cygecpg-6.dll
   /usr/bin/cygecpg_compat-3.dll
   /usr/bin/cygpgtypes-3.dll
   /usr/bin/cygpq-5.dll

attached the patch used for the build.

Regards
Marco

> I couldn't find appropriate system documentation.  Regarding Linux, I
> remember I saw some HOWTO on tldp.org website which explains the
> concept of shared library soname, but it's not very friendly for
> users who just want to know the application binary compatibility
> policy of PostgreSQL.  And I don't think there's suitable 

I would expect Unix sysadmins to understand that howto, but there are
others, e.g. a random hit from google: https://www.bottomupcs.com/libra
ries_and_the_linker.xhtml

There even is a wikipedia page about it: https://en.wikipedia.org/wiki/
Soname

> documentation on Windows.  Even if there is any, users will not be
> sure whether PostgreSQL follows those platform-specific
> conventions.  They may have doubts about it, because even the product
> version "PostgreSQL x.y.z" causes misconception that x is the major
> version and y is the minor one.

I don't know anything about the Windows setup in PostgreSQL, but I
would find it fair to assume that PostgreSQL follows conventions.

> BTW, although this may be a separate topic, it may be better that we
> add the major version in the library names like libpq5.dll and
> libecpg6.dll, so that the application can fail to run with the
> incompatible versions of libpq and libecpg.  FYI:

Does this mean you cannot have to versions of libpq installed on the
same Windows system at the same time?

Michael
-- 
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Meskes at (Debian|Postgresql) dot Org
Jabber: michael at xmpp dot meskes dot org
VfL Borussia! Força Barça! SF 49ers! Use Debian GNU/Linux, PostgreSQL

> From: pgsql-hackers-owner@postgresql.org
> [mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Marco Atzeri
> on cygwin the postgresql binary package already include the library
> versions:
> 
>    /usr/bin/cygecpg-6.dll
>    /usr/bin/cygecpg_compat-3.dll
>    /usr/bin/cygpgtypes-3.dll
>    /usr/bin/cygpq-5.dll
> 
> attached the patch used for the build.

Thanks for the information.   I didn't know there's a PostgreSQL binary package for Cygwin.

Regards
Takayuki Tsunakawa

> From: Michael Meskes [mailto:meskes@postgresql.org]
> e.g. a random hit from google:=C2=A0https://www.bottomupcs.com/libra
> ries_and_the_linker.xhtml
> 
> There even is a wikipedia page about
> it:=C2=A0https://en.wikipedia.org/wiki/
> Soname

Thank you for good pointers.  The former is particularly nice.

> > BTW, although this may be a separate topic, it may be better that we
> > add the major version in the library names like libpq5.dll and
> > libecpg6.dll, so that the application can fail to run with the
> > incompatible versions of libpq and libecpg.=C2=A0=C2=A0FYI:
> 
> Does this mean you cannot have to versions of libpq installed on the same
> Windows system at the same time?

No, you can have different versions in separate folders, as in:

C:\Program Files\PostgreSQL\9.2
C:\Program Files\PostgreSQL\9.5

and change the PATH environment variable to point to a newer version when you want to use the existing application
withoutrebuilding it.
 

However, the problem I pointed out is that when the new library is incompatible with the older one, say the major
versionof libpq.dll changes from 5 to 6, the application user and/or developer cannot notice the incompatibility
immediatelyand easily.  On Unix/Linux, the application fails to start because the older library is not found.  On the
otherhand, the application will start on Windows and probably cause difficult trouble due to the incompatibility.
 

Regards
Takayuki Tsunakawa

> However, the problem I pointed out is that when the new library is
> incompatible with the older one, say the major version of libpq.dll
> changes from 5 to 6, the application user and/or developer cannot
> notice the incompatibility immediately and easily.  On Unix/Linux,
> the application fails to start because the older library is not
> found.  On the other hand, the application will start on Windows and
> probably cause difficult trouble due to the incompatibility.

I don't think this is a very good situation, but I have no idea if this
can be solved. However, I'd prefer a technical solution over a
documentation one.

Michael
-- 
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Meskes at (Debian|Postgresql) dot Org
Jabber: michael at xmpp dot meskes dot org
VfL Borussia! Força Barça! SF 49ers! Use Debian GNU/Linux, PostgreSQL

On Mon, May 30, 2016 at 03:04:24AM +0000, Tsunakawa, Takayuki wrote:
> Hello,
> 
> I'd like to ask you about the policy of application binary compatibility.  And have a suggestion as well.
> 
> QUESTION
> ==================================================
> 
> My customer asked me if the following usage is correct.
> 
> - Build an embedded SQL C application with PostgreSQL 9.2.
> - Distribute the app without ecpg nor libpq libraries.  Require users to install PostgreSQL client which includes
ecpgand libpq libraries.
 
> - Use the app with newer PostgreSQL major versions without rebuilding the app.  That is, the users just replaces the
PostgreSQLclient.
 
> 
> I expect this is legal, because the so_major versions of ecpg and libpq are 6 and 5 respectively for all PostgreSQL
9.xversions so far.  However, I could not find any description of this binary compatibility policy in the manual, so I
haven'tbeen able to answer the customer.
 
> 
> What's the official policy of application binary compatibility?  I found the same discussion in the below thread, but
I'mafraid any clear answer wasn't given:
 
> 
> https://www.postgresql.org/message-id/522F0B6B.1040006@4js.com
> 
> 
> SUGGESTION
> ==================================================
> 
> How about adding an article about application binary compatibility in the following section, as well as chapters for
libpq,ECPG, etc?
 
> 
> 17.6. Upgrading a PostgreSQL Clust
> https://www.postgresql.org/docs/devel/static/upgrading.html
> 
> There are three kinds of application assets that users are concerned about when upgrading.  Are there anything else
tomention?
 
> 
> * libpq app
> * ECPG app
> * C-language user defined function (and other stuff dependent on it, such as extensions, UDTs, etc.)

I know this thread is old but it bounced around a lot of ideas.  I think
there are some open questions:

*  Will a new application link against an older minor-version libpq?
*  Will an old application link against a newer minor-version libpq?

I think we are all in agreement that a binary cannot run using a libpq
with a different major version number.

We have this text in src/tools/RELEASE_CHANGES:
Major Version=============The major version number should be updated whenever the source of thelibrary changes to make
itbinary incompatible. Such changes include,but are not limited to:1. Removing a public function or structure (or
typedef,enum, ...)2. Modifying a public functions arguments.3. Removing a field from a public structure.4. Adding a
fieldto a public structure, unless steps have beenpreviously taken to shield users from such a change, for example
bysuchstructures only ever being allocated/instantiated by a libraryfunction which would give the new field a suitable
defaultvalue.Adding a new function should NOT force an increase in the major versionnumber. (Packagers will see the
standardminor number update and installthe new library.)  When the major version is increased all applicationswhich
linkto the library MUST be recompiled - this is not desirable. Whenthe major version is updated the minor version gets
reset.MinorVersion=============The minor version number should be updated whenever the functionality ofthe library has
changed,typically a change in source code between releaseswould mean an increase in the minor version number so long as
itdoes notrequire a major version increase.Given that we make at least minor changes to our libraries in every
majorPostgreSQLversion, we always bump all minor library version numbers at thestart of each development cycle as a
matterof policy.
 

This is saying running against a mismatched minor version should be fine
for a binary.

ecpg is a little tricker because it has knowledge of SQL data types and
might not support certain features if the ecpg library isn't _run_
against the same major server version.  My guess is that older ecpg
libraries will run fine against newer servers, but the opposite might
not be true.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+                     Ancient Roman grave inscription +

> From: Bruce Momjian [mailto:bruce@momjian.us]
> We have this text in src/tools/RELEASE_CHANGES:
> ...
> This is saying running against a mismatched minor version should be fine
> for a binary.

Thanks for a good rationale.


> I know this thread is old but it bounced around a lot of ideas.  I think
> there are some open questions:
> 
> *  Will a new application link against an older minor-version libpq?
> *  Will an old application link against a newer minor-version libpq?

The former does not always hold true, if the application uses a new libpq function which is not in an old
miner-version. But I think the backward-compatibility is enough.
 


Regards
Takayuki Tsunakawa

On Thu, Jun 23, 2016 at 06:42:57AM +0000, Tsunakawa, Takayuki wrote:
> > From: Bruce Momjian [mailto:bruce@momjian.us]
> > We have this text in src/tools/RELEASE_CHANGES:
> > ...
> > This is saying running against a mismatched minor version should be fine
> > for a binary.
> 
> Thanks for a good rationale.
> 
> 
> > I know this thread is old but it bounced around a lot of ideas.  I think
> > there are some open questions:
> > 
> > *  Will a new application link against an older minor-version libpq?
> > *  Will an old application link against a newer minor-version libpq?
> 
> The former does not always hold true, if the application uses a new libpq function which is not in an old
miner-version. But I think the backward-compatibility is enough.
 

Yes, I think that is correct, and I think that is covered in the file
posted:
Adding a new function should NOT force an increase in the major versionnumber.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+                     Ancient Roman grave inscription +