Re: RFC: Extension Packaging & Lookup

On Oct 29, 2024, at 13:16, Paul Ramsey <pramsey@cleverelephant.ca> wrote:

> An apposite choice, since it not only demonstrates depending on a common system library, it also demonstrates the way
thesethings loop on each other, as curl then depends on libssl, which postgres also depends on. 

Ooh, yeah, vicious!

> Relative rpaths as I have seen them are relative to the executable or library in which they are defined (as far as I
know,I’m not a dylib expert by any stretch). The implication is that extension.so could have an rpath=. and dependent
dylibssitting next to it. This is how, for example, cmake out-of-tree builds can run tests against the newly built
librarybefore it’s installed,.. the test execs have an rpath of ../lib on them. 

Oh that’s super interesting. Will be worth trying. I was hoping to avoid having to set rpath in every extension,
though,but maybe that’d be the way forward. 

The issue, though, is a tree of dependencies. Would you really want to include both libcurl and OpenSSL in a pgsql-http
binarydistribution package --- especially since Postgres itself will be using its own OpenSSL package? To Christophe’s
point,I think we might want to delegate the provisioning of dependency DSOs to the system package manager. 

>> Yeah, I think the issue will be to figure out how to manage OS package-provided system dependencies in immutable
environmentslike a Docker container. I suspect some combination of -rpath compiled into Postgres and mounting
individualDSO files not included in the base image will be the way to go. 
>
> Do you see immutable images as the main deployment path going forward? I’m not container-pilled yet, so it still
feelslike a niche concern. Meanwhile being able to “add an extension that my cloud provider doesn’t support yet” feels
quitevital. 

I think it will become increasingly common. At first I thought it was just Docker/Kubernetes --- and in fairness, a lot
oforgs are running Postgres in such environments these days. Some Postgres as a Service companies exclusively use
Kubernetes(my employer, Tembo, is one). 

But then Tobias Bussmann pointed out[1] that Postgres.app for macOS has the same issue: it has to be immutable in order
tovalidate the app via Apple’s provisioning certificate. If you change the contents of the Postgres.app bundle, you can
nolonger run it! It would not surprise me if there were other examples, and that the pattern becomes increasingly
common.

Best,

David

[1]: https://justatheory.com/2024/03/mini-summit-two/