Re: Extension Templates S03E11

On 12/15/2013 10:47 PM, Jeff Davis wrote:
> The patch offers an alternative to dropping files on the filesystem
> before doing CREATE EXTENSION. Instead, if the extension has no C code,
> you can put it in the catalog using ordinary SQL access, and execute the
> same kind of CREATE EXTENSION. Aside from that, it's pretty much
> identical to existing extensions.

OK, so the idea is that for psql-only extensions (i.e. ones without
.so's) the user shouldn't be required to create a file on disk.  That
seems straightforwards and acceptable.

> Stephen doesn't like the idea that the SQL in an extension is a blob of
> text. 

I, personally, would prefer per-object line-items, but I don't think
that's a deal-breaker.  Having a single text blob does match up with
existing Extension design.

Note for Dimitri, though: I think that having line-item objects in
dependancy order would make this feature vastly more useful for
schema-template maintenance.  Give it some thought.

> There are weird cases, like if you make local modifications to
> objects held in an extension, then dump/reload will lose those local
> modifications.

What does DUMP/Reload do with regular Extensions currently in that case?

> Another issue, which I agree is dubious in many
> situations, is that the version of an extension is not preserved across
> dump/reload (this is actually a feature, which was designed with
> contrib-style extensions in mind, but can be surprising in other
> circumstances).

Well, this should work with a versioning system, in which dump/reload
can load older versions of the extension if they are present, the same
as external Extensions do now.  Is that built in?

> This isn't necessarily a dead-end, but there are a lot of unsettled
> issues, and it will take some soul-searching to answer them. Is an
> "extension" a blob of text with a version, that's maintained in some
> external repo?

Well, plus potentially binaries and library references, yes.  Although
you could describe all of Postgres as a bunch of text blobs and some
library references, when you get right down to it.

> Is it the job of postgres to ensure that dump/reload
> creates the same situation that you started with, including local
> modifications to objects that are part of an extension?

IMHO: No.  AFAIK, if a user modifies, say, information_schema views in
PostgreSQL, we don't respect that in dump/restore either.

Now, I can see adding to this mechanism a method for tracking such
modifications in a way that pgdump can support them.  But that can
easily be a version 2 feature.

> Should
> everything be an extension, or do we need to invent a new concept for
> some of the use cases? What role to external tools play in all of this?

So, the reason I was confused by this feature -- and the reason Stephen
hates it, I think -- is that I thought it was solving the "Extensions
don't follow replication, and they are complicated to install if your OS
doesn't have good packages" problem.  It's not, and it never will solve
that issue.

It's solving a completely different problem, to wit:

Some PostgreSQL shops with lots of servers have large internal libraries
of functions, views, and similar code that they've written to support
their applications, which don't comprise a complete database.  This
feature would allow them to "package" those libraries, and version,
upgrade and track them, without requiring a filesystem-based install.  I
myself have a couple clients who could benefit from this.

I think the name "Extension Templates" is horrible because it misleads
all of us on this list into thinking the proposed feature is completely
something other than what it is.  I don't have a better name offhand,
but that's got to change before it becomes a feature.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com