Re: Where to load modules from?

On Wed, Sep 18, 2013 at 9:26 AM, Dimitri Fontaine
<dimitri@2ndquadrant.fr> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>>>   - consider on-disk extension as templates and move their module files
>>>     somewhere private in $PGDATA and load the code from there
>>
>> I think this will be horrid mess of security vulnerabilities and upgrade woes.
>
> I think it's a solution to that horrid mess. Care to expand on your thoughs?

I think that would largely be rehashing previous discussions, in which
it's already been established that we don't see eye to eye on this
issue.  But briefly, I think that replacing shared libraries ought to
be the job of the OS packaging system.  System administrators are
already familiar with the notion that they need to run 'yum update' or
whatever the comparable command is from time to time, or they're going
to be vulnerable.  But such packaging systems aren't going to update
copies of shared libraries we've placed inside $PGDATA.

And, I can't see what advantage it offers.  If you want to be able to
have more than one version of the extension installed, you can solve
that at the OS level by choosing distinct names for their shared
libraries.  If you want to be able to download and install C code into
your PostgreSQL database via a libpq connection, someone can write an
extension to do that today, without any core changes at all.  If it's
nifty enough, it might even be worth shipping in contrib as an example
of how you can make deployment really, really easy at some price in
security.  If your concern is that someone might modify the
OS-installed versions of those extensions in a way that renders them
incompatible with your running PostgreSQL binaries, well, they
shouldn't be doing that *anyway*.  If your OS distributor is in the
habit of shipping new versions of shared libraries that break backward
compatibility in minor OS releases, you need a new OS distributor.

If you upgrade your PostgreSQL distribution, and it breaks ABI
compatibility, then using the shared libraries baked into the old
cluster is exactly the wrong thing to do.  You need to use the NEW
shared libraries compiled against the new ABI, or your code may do
anything from "not load" to "crash he server".  You had better hope
that both the old and new libraries have the same user-facing API, or
there's going to be trouble, but we've taken care to do that with the
things we ship in contrib and there's no reason to assume other module
authors can't do likewise.

>> Here's another idea.  At initdb time, create an empty directory called
>> called pg_you_can_load_stuff_from_here (pick a better name) inside
>> $PGDATA.  Allow it to be replaced with a symlink.  This would be
>> similar to what we do today with pg_xlog.  In fact, you can imagine an
>> equivalent of initdb -X that does something precisely analogous.  This
>> feels a bit more natural to me than a GUC.
>
> I like that too.
>
> Now the thing I've heard the most about our extension infrastructure is
> how to make it support loading different versions of the same module in
> different databases in the same cluster. The second one on that topic
> might be why are extensions managed per-database except for their binary
> module parts?

The basic problem here is that extensions are allowed to do anything,
including cluster-wide stuff like "allocate and use chunks of shared
memory".  If you try to load and use two different and incompatible
versions of pg_stat_statements in two different databases, there will
not be a happy ending.  Also, if you load a version of some library in
shared_preload_libraries, you won't subsequently be able to decide to
load some other version in a backend bound to some particular
database.  Maybe these problems are solvable, but as things stand
today I think that trying to use identically-named .so files in
different databases at the same time should be regarded as dangerously
unsupported.  In any event, any kind of infrastructure intended to
support such use cases  is clearly a different project from simply
allowing modules to be loaded from a different location.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company