Re: Extension Templates S03E11

* Jeff Davis (pgsql@j-davis.com) wrote:
> On Sun, 2013-12-01 at 22:34 -0500, Stephen Frost wrote:
> > Perhaps I'm missing something- but we already *have* a catalog
> > representation for every extension that's ever installed into a given
> > database.  A representation that's a heck of a lot better than a big
> > text blob.
>
> I meant "extension template" represented in the catalog.

Ok.

> > > But bringing more of an extension into the catalog can be done, and I
> > > think we'll see big benefits from that.
> >
> > I'm not following here- what's 'missing'?

The above was in reference to an extension, not an extension template.

> It seems that you are making the assumption that installing an extension
> template or creating an extension are major operations, and anyone
> encountering an error is a superuser with admin access to the server and
> can easily correct it.

I'm not following this train of thought at all..  Unless you're
referring to the case where it involves a .so, in which case then I
*would* expect a superuser with admin access to the server to be
involved.  For the non-.so-including case, I'm hopeful we can provide
a mechanism which doesn't require a superuser or admin at all.

> If the admin messes up and the extension template isn't there (e.g.
> after a failover), the person to encounter the error at CREATE EXTENSION
> time might not have admin access or there might be a process required to
> deploy the new files. But if the extension templates were carried along
> with replication and backup naturally, then they'd be there.

The *extension* is already brought along with replication and binary
backup naturally, for non-.so-based extensions.  For logical backups, we
depend on the filesystem to provide the extension definition- I don't
particularly like that and it *isn't necessary* for non-.so-based
extensions.  In many ways, we do it the way we do specifically due to
how we distribute -contrib, as Dimitri has pointed out.  What I'd like
to see is support for dumping the extension's objects during a logical
dump, perhaps only for specific types of extensions (ones installed that
way), and maybe with some additional caveats on it (should we have
multiple extension namespaces?  or come up with some other name for
these objects?).

For .so-based extensions, yes, an admin is going to need to be involved
and I am not impressed by these notions that such a requirement is
unreasonable or overly draconian.

> And it would be nice if there was some hope for non-superusers to create
> extension templates, but that will never happen as long as they are
> files.

Agreed and as a follow-on to that, I'm pretty sure .so's will always be
files...

> > You need a running PG for the *extension* to be installed, but with the
> > filesystem-based extension approach we have today, the "template" (which
> > are the files on the filesystem) don't need PG running
>
> I think you misread -- this is a list of issues if we move templates
> into the catalog. File-based templates obviously don't have this
> problem.

I don't see us being able to completely get rid of the existing
extension system which uses the file-based approach.  Certainly not for
a very long time, so I'm not sure that there's much discussion to be
had along these lines.

> I was just suggesting that a little more information in the catalog
> could improve dependency tracking and error handling. I'm not suggesting
> we "ship" any shared libraries anywhere -- that's still up to extension
> authors and PGXN. I'm also not suggesting that the error handling will
> be perfect or catch subtle mismatches.

Having more information about an extension in the catalog seems
reasonable to me- along the same lines of how we have dependency
information in dpkg for .deb's.  What I don't like is the notion of
upgrade scripts which are installed into the catalog as *scripts*.
Ditto for initial extension creation.  We already know how to drop and
recreate every object in the database, let's make use of that mechanism
instead.  That doesn't allow pg_dump/restore to *invisibly upgrade an
extension* (which is what happens now..), but I find that to be an
entirely good thing...
Thanks,
    Stephen