Обсуждение: Wanna help PostgreSQL

* Zhe-Wei Jiang (jrreinhardt@gmail.com) wrote:
> Therefore, I'd like to know if there is any entry-level implementation
> needed by PostgreSQL now?
> Any advise is appreciated.

You might take a look at the GSoC ideas page instead:

https://wiki.postgresql.org/wiki/GSoC_2014
Thanks,
    Stephen

Zhe-Wei Jiang <jrreinhardt@gmail.com> writes:
>> I read the GSoC ideas page and think the only one I'm possibly to help for
>> now is
>> "Rewrite (add) pg_dump and pg_restore utilities as libraries (.so, .dll &
>> .dylib)",
>> while I'm still not clear with the whole postgreSQL project.
>> 
>> If my understanding is correct, taking pg_dump as an example, this should
>> include refactoring the pg_dump.c and rewrite the Makefile to make it as
>> shared libraries.
>> Do I miss anything?

The reason that that project has gone untouched for upwards of ten years
is that it's not just a large coding project, but involves a lot of
complex API design with uncertain goals.  It's not very clear what
features people would want from a "pg_dump library", though one capability
that gets mentioned often is the ability to extract the SQL definition
for a single object.  So before anything else you'd need to identify a
satisfactory set of library capabilities.  The next nasty problem is that
pg_dump has a large set of odd behaviors that have evolved for good and
sufficient reason, and that we'd not want to give up, but that it's not
clear whether anyone else would want --- and they'd complicate any API
definition quite a bit.  One example is that pg_dump knows how to dump
objects in a safe order to avoid forward references.  If there turn out
to be circular references (which arise in more cases than you might think)
it even knows how to split certain kinds of objects into multiple
commands so as to break the circularity.  How would we expose all that?
The features for parallel pg_dump and pg_restore are another thing that
doesn't seem to fit all that well into a clean library API.  But how much
of this should actually be in a library, rather than in the wrapper
programs?

So while this is certainly a worthwhile task, it's not one to
underestimate the scope and difficulty of.
        regards, tom lane

On Wed, May 14, 2014 at 9:44 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> The reason that that project has gone untouched for upwards of ten years
> is that it's not just a large coding project, but involves a lot of
> complex API design with uncertain goals.  It's not very clear what
> features people would want from a "pg_dump library", though one capability
> that gets mentioned often is the ability to extract the SQL definition
> for a single object.

Personally I'd prefer the creation of definitional SQL be moved out of
pg_dump and into the database proper via something like
'pg_sql_definition(oid)' or something like that.  There are lot of
reasons applications (especially administrative ones like pgadmin and
psql but also end user applications in some cases) would want to do
that and forcing everything through pg_dump et al is awkward.  The
less magic in the external applications the better.

merlin

On 2014-05-14 10:49:05 -0500, Merlin Moncure wrote:
> On Wed, May 14, 2014 at 9:44 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > The reason that that project has gone untouched for upwards of ten years
> > is that it's not just a large coding project, but involves a lot of
> > complex API design with uncertain goals.  It's not very clear what
> > features people would want from a "pg_dump library", though one capability
> > that gets mentioned often is the ability to extract the SQL definition
> > for a single object.
> 
> Personally I'd prefer the creation of definitional SQL be moved out of
> pg_dump and into the database proper via something like
> 'pg_sql_definition(oid)' or something like that.  There are lot of
> reasons applications (especially administrative ones like pgadmin and
> psql but also end user applications in some cases) would want to do
> that and forcing everything through pg_dump et al is awkward.  The
> less magic in the external applications the better.

That'd be a separate feature from pg_dump though. pg_dump needs to be
cross-version compatible and the above prevents that...

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services

Merlin Moncure <mmoncure@gmail.com> writes:
> On Wed, May 14, 2014 at 9:44 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> The reason that that project has gone untouched for upwards of ten years
>> is that it's not just a large coding project, but involves a lot of
>> complex API design with uncertain goals.  It's not very clear what
>> features people would want from a "pg_dump library", though one capability
>> that gets mentioned often is the ability to extract the SQL definition
>> for a single object.

> Personally I'd prefer the creation of definitional SQL be moved out of
> pg_dump and into the database proper via something like
> 'pg_sql_definition(oid)' or something like that.

Well, that's just a different way of packaging a library, no?  It doesn't
make the library-API problems any less difficult.  If anything, it makes
things even harder, because now you have to consider version skew between
pg_dump and the server.  And if you get any API details wrong you have
no ability to change them till the next major release cycle.

While we might someday do it like that, I'd think it foolish to proceed
with such an approach until we had a proven library API design on the
client side.  The costs of iterating there are a lot less.
        regards, tom lane

On Wed, May 14, 2014 at 10:58 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Merlin Moncure <mmoncure@gmail.com> writes:
>> On Wed, May 14, 2014 at 9:44 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> The reason that that project has gone untouched for upwards of ten years
>>> is that it's not just a large coding project, but involves a lot of
>>> complex API design with uncertain goals.  It's not very clear what
>>> features people would want from a "pg_dump library", though one capability
>>> that gets mentioned often is the ability to extract the SQL definition
>>> for a single object.
>
>> Personally I'd prefer the creation of definitional SQL be moved out of
>> pg_dump and into the database proper via something like
>> 'pg_sql_definition(oid)' or something like that.
>
> Well, that's just a different way of packaging a library, no?  It doesn't
> make the library-API problems any less difficult.  If anything, it makes
> things even harder, because now you have to consider version skew between
> pg_dump and the server.  And if you get any API details wrong you have
> no ability to change them till the next major release cycle.
>
> While we might someday do it like that, I'd think it foolish to proceed
> with such an approach until we had a proven library API design on the
> client side.  The costs of iterating there are a lot less.

Yeah -- Andres said it even more cleanly: for forward upgrades you'd
need to communicate with both versioned backends to produce a dump.
That's not a complete deal breaker but definitely a lot more complex
than I was thinking.

merlin