Обсуждение: Maliing list request: pgsql-forks@
At pgCon we had an BoF for projects/companies that are maintaining either forks of Postgres (ie: Aurora Postgres, EDB, Greenplum, BDR), or extensions that interact heavily with internals (ie: Citus, TimescaleDB). We’d like to have a mailing list to facilitate further discussion in this area.
On 2019-Jul-29, Nasby, Jim wrote: > At pgCon we had an BoF for projects/companies that are maintaining > either forks of Postgres (ie: Aurora Postgres, EDB, Greenplum, BDR), > or extensions that interact heavily with internals (ie: Citus, > TimescaleDB). We’d like to have a mailing list to facilitate further > discussion in this area. Neat. Where do they hold such discussions currently? -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
> On Jul 29, 2019, at 5:11 PM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote: > > On 2019-Jul-29, Nasby, Jim wrote: > >> At pgCon we had an BoF for projects/companies that are maintaining >> either forks of Postgres (ie: Aurora Postgres, EDB, Greenplum, BDR), >> or extensions that interact heavily with internals (ie: Citus, >> TimescaleDB). We’d like to have a mailing list to facilitate further >> discussion in this area. > > Neat. Where do they hold such discussions currently? Any discussion is currently completely ad-hoc, via various methods. Though really I suspect the best answer is “they don’t”.
On Mon, 29 Jul 2019 at 22:57, Nasby, Jim <nasbyj@amazon.com> wrote:
At pgCon we had an BoF for projects/companies that are maintaining either forks of Postgres (ie: Aurora Postgres, EDB, Greenplum, BDR), or extensions that interact heavily with internals (ie: Citus, TimescaleDB). We’d like to have a mailing list to facilitate further discussion in this area.
BDR3 is not a fork of Postgres, but it is an extension.
> On 30 Jul 2019, at 00:25, Nasby, Jim <nasbyj@amazon.com> wrote: > >> On Jul 29, 2019, at 5:11 PM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote: >> >> On 2019-Jul-29, Nasby, Jim wrote: >> >>> At pgCon we had an BoF for projects/companies that are maintaining >>> either forks of Postgres (ie: Aurora Postgres, EDB, Greenplum, BDR), >>> or extensions that interact heavily with internals (ie: Citus, >>> TimescaleDB). We’d like to have a mailing list to facilitate further >>> discussion in this area. >> >> Neat. Where do they hold such discussions currently? > > Any discussion is currently completely ad-hoc, via various methods. Though really I suspect the best answer is “they don’t”. Being one of the stakeholders (and BoF participants), I can only agree with this, they largely don’t (or are limited to hallway tracks). The idea is to try and improve collaboration by providing a place for discussion. cheers ./daniel
On 7/30/19 8:16 AM, Simon Riggs wrote: > On Mon, 29 Jul 2019 at 22:57, Nasby, Jim <nasbyj@amazon.com > <mailto:nasbyj@amazon.com>> wrote: > > At pgCon we had an BoF for projects/companies that are maintaining > either forks of Postgres (ie: Aurora Postgres, EDB, Greenplum, BDR), > or extensions that interact heavily with internals (ie: Citus, > TimescaleDB). We’d like to have a mailing list to facilitate further > discussion in this area.____ > > > BDR3 is not a fork of Postgres, but it is an extension. On that note, I'd be a +1 for "pgsql-extensions" as it would certainly help facilitate above suggested communication. It would be all encompassing of extensions, forks, etc. Jonathan
Вложения
> On 30 Jul 2019, at 15:35, Jonathan S. Katz <jkatz@postgresql.org> wrote: > > On 7/30/19 8:16 AM, Simon Riggs wrote: >> On Mon, 29 Jul 2019 at 22:57, Nasby, Jim <nasbyj@amazon.com >> <mailto:nasbyj@amazon.com>> wrote: >> >> At pgCon we had an BoF for projects/companies that are maintaining >> either forks of Postgres (ie: Aurora Postgres, EDB, Greenplum, BDR), >> or extensions that interact heavily with internals (ie: Citus, >> TimescaleDB). We’d like to have a mailing list to facilitate further >> discussion in this area.____ >> >> >> BDR3 is not a fork of Postgres, but it is an extension. > > On that note, I'd be a +1 for "pgsql-extensions" as it would certainly > help facilitate above suggested communication. It would be all > encompassing of extensions, forks, etc. Or perhaps pgsql-distributions@, to make it even clearer that it’s a forum for all postgres derived distributions regardless of technical implementation. cheers ./daniel
On Tue, 30 Jul 2019 at 14:36, Jonathan S. Katz <jkatz@postgresql.org> wrote:
On 7/30/19 8:16 AM, Simon Riggs wrote:
> On Mon, 29 Jul 2019 at 22:57, Nasby, Jim <nasbyj@amazon.com
> <mailto:nasbyj@amazon.com>> wrote:
>
> At pgCon we had an BoF for projects/companies that are maintaining
> either forks of Postgres (ie: Aurora Postgres, EDB, Greenplum, BDR),
> or extensions that interact heavily with internals (ie: Citus,
> TimescaleDB). We’d like to have a mailing list to facilitate further
> discussion in this area.____
>
>
> BDR3 is not a fork of Postgres, but it is an extension.
On that note, I'd be a +1 for "pgsql-extensions" as it would certainly
help facilitate above suggested communication. It would be all
encompassing of extensions, forks, etc.
Sounds like a better name and focus.
On Tue, Jul 30, 2019 at 3:39 PM Simon Riggs <simon@2ndquadrant.com> wrote:
On Tue, 30 Jul 2019 at 14:36, Jonathan S. Katz <jkatz@postgresql.org> wrote:On 7/30/19 8:16 AM, Simon Riggs wrote:
> On Mon, 29 Jul 2019 at 22:57, Nasby, Jim <nasbyj@amazon.com
> <mailto:nasbyj@amazon.com>> wrote:
>
> At pgCon we had an BoF for projects/companies that are maintaining
> either forks of Postgres (ie: Aurora Postgres, EDB, Greenplum, BDR),
> or extensions that interact heavily with internals (ie: Citus,
> TimescaleDB). We’d like to have a mailing list to facilitate further
> discussion in this area.____
>
>
> BDR3 is not a fork of Postgres, but it is an extension.
On that note, I'd be a +1 for "pgsql-extensions" as it would certainly
help facilitate above suggested communication. It would be all
encompassing of extensions, forks, etc.Sounds like a better name and focus.
+1.
I think the other part is, what would be the "level" of focus. Is this for people building the extensions, or people using them? If it's for people working on them, perhaps it should be pgsql-extensions-hackers or similar, to make that clear?
On 7/30/19 8:38 AM, Daniel Gustafsson wrote: >> On 30 Jul 2019, at 15:35, Jonathan S. Katz <jkatz@postgresql.org> wrote: >> >> On 7/30/19 8:16 AM, Simon Riggs wrote: >>> On Mon, 29 Jul 2019 at 22:57, Nasby, Jim <nasbyj@amazon.com >>> <mailto:nasbyj@amazon.com>> wrote: >>> >>> At pgCon we had an BoF for projects/companies that are maintaining >>> either forks of Postgres (ie: Aurora Postgres, EDB, Greenplum, BDR), >>> or extensions that interact heavily with internals (ie: Citus, >>> TimescaleDB). We’d like to have a mailing list to facilitate further >>> discussion in this area.____ >>> >>> >>> BDR3 is not a fork of Postgres, but it is an extension. >> >> On that note, I'd be a +1 for "pgsql-extensions" as it would certainly >> help facilitate above suggested communication. It would be all >> encompassing of extensions, forks, etc. > > Or perhaps pgsql-distributions@, to make it even clearer that it’s a forum for > all postgres derived distributions regardless of technical implementation. (and while I do like blue for the color of the bike ;) I think that "distributions" is too close to "packaging" and could be confusing. Why I suggested "-extensions" is that it appears to be inclusive of things that augment PostgreSQL abilities, be they actual PG extensions or forks. Jonathan
Вложения
> On Jul 30, 2019, at 06:42, Jonathan S. Katz <jkatz@postgresql.org> wrote: > > (and while I do like blue for the color of the bike ;) I think that > "distributions" is too close to "packaging" and could be confusing. > > Why I suggested "-extensions" is that it appears to be inclusive of > things that augment PostgreSQL abilities, be they actual PG extensions > or forks. +1 for "-extensions" over "-distributions". An extension generally doesn't imply an entire distribution. When I first saw"-distributions", my thought was, "Don't we have that list already?" -- -- Christophe Pettus xof@thebuild.com
> On 30 Jul 2019, at 16:35, Christophe Pettus <xof@thebuild.com> wrote: > >> On Jul 30, 2019, at 06:42, Jonathan S. Katz <jkatz@postgresql.org> wrote: >> >> (and while I do like blue for the color of the bike ;) I think that >> "distributions" is too close to "packaging" and could be confusing. >> >> Why I suggested "-extensions" is that it appears to be inclusive of >> things that augment PostgreSQL abilities, be they actual PG extensions >> or forks. > > +1 for "-extensions" over "-distributions". An extension generally doesn't imply an entire distribution. When I firstsaw "-distributions", my thought was, "Don't we have that list already?” Fair enough. As long as we paint it a color, regardless of which, I’m happy to park there. cheers ./daniel
> On Jul 30, 2019, at 10:14 AM, Daniel Gustafsson <daniel@yesql.se> wrote: > >> On 30 Jul 2019, at 16:35, Christophe Pettus <xof@thebuild.com> wrote: >> >>> On Jul 30, 2019, at 06:42, Jonathan S. Katz <jkatz@postgresql.org> wrote: >>> >>> (and while I do like blue for the color of the bike ;) I think that >>> "distributions" is too close to "packaging" and could be confusing. >>> >>> Why I suggested "-extensions" is that it appears to be inclusive of >>> things that augment PostgreSQL abilities, be they actual PG extensions >>> or forks. >> >> +1 for "-extensions" over "-distributions". An extension generally doesn't imply an entire distribution. When I firstsaw "-distributions", my thought was, "Don't we have that list already?” > > Fair enough. As long as we paint it a color, regardless of which, I’m happy to > park there. My concern about calling this -extensions is that it’s really meant for people that are at least at the level of C extensions(and probably only those making use of hooks). That’s a relatively small number of extensions, even if they aresome of the most important extensions. Maybe paint it “-binary”? -internals might work too. In case it adds some better perspective, here’s the list of pain points that were captured in the BoF: Pain for forks due to high velocity of upstream commits Forks can break extension compatibility PG DLL import Backend static functions / structs Not enough hooks Ordering of hook registration More pluggable modules/components Extension upgrading is difficult Not enough test coverage for hooks Hard to extend grammar (add more WITH options?) Improvements to typmodin() Problems with extensions calling other extensions (both SQL and C; this item is actually broader than the others) Still too much is_superuser() (possibly also broader)
> At pgCon we had an BoF for projects/companies that are maintaining either forks of Postgres (ie: Aurora Postgres, EDB,Greenplum, BDR), or extensions that interact heavily with internals (ie: Citus, TimescaleDB). We’d like to have a mailinglist to facilitate further discussion in this area. Pgpool-II does not fall into "forks" mentioned above but it actually forks part of PostgreSQL code (raw parser, memory manager and elog stuffs). It has experienced part of the pains as other forks Jim already mentioned: Pain for forks due to high velocity of upstream commits Backend static functions / structs So +1 to create new ML. -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp
On Wed, 31 Jul 2019 at 08:43, Nasby, Jim <nasbyj@amazon.com> wrote:
In case it adds some better perspective, here’s the list of pain points that were captured in the BoF:
For what it's worth, though I entirely agree with the below I've also been truly amazed at how much can be achieved with PostgreSQL extensions.
Even in some Petr has been able to do things with BDR has been able to do things I just didn't think would be possible, let alone practical, and often done them quite cleanly.
Backend static functions / structs
Yeah. I've had to copy postgres code too many times because the functionality is in core exactly where I need it, but I can't access it.
Sometimes I understand the deliberate hiding of access, but often it's pretty arbitrary.
A bigger symbol table is not really much concern for C code. It's not like we're working with C++ where the symbol table is many many megabytes of template mangled gibberish that must be trimmed at all costs to avoid giving the dynamic linker a heart attack.
We don't have any formal API or any well defined approach to deciding what is/isn't "public" so it's mostly pretty arbitrary what is visible and what is not.
Not enough hooks
Never enough hooks. ALL. THE. HOOKS.
We should really get together, make a list of what we want, and start working together to add them. They're cheap and there are a variety of tricks available to make them cheaper still if we need them in hot paths.
I looked around a while ago for a well supported, portable runtime patching library to support NOOP jump replacement based hooking btw. Unfortunately I didn't find anything that looks remotely viable, let alone usable by something that needs a high level of of maturity and portability like PostgreSQL.
Ordering of hook registration
Haven't really had to face this one. I'm interested?
More pluggable modules/components
Like? Not arguing, but anything like that can have a big cost in maintenance on core and the interface design is often hard.
Extension upgrading is difficult
Couldn't agree more. Extension upgrades are ok-ish when linear, but as soon as you start doing stable/dev versions and branching it's hell.
Hard to extend grammar (add more WITH options?)
It'd be nice to have a few limited extension points like that, but the discussions around more general parser extension have tended to flounder for a variety of good reasons. Our parser tooling is extremely poorly suited to pluggable extensions, and that aside it's really hard to do things like maintain good clear error messages while making the grammar extensible in general terms.
Problems with extensions calling other extensions (both SQL and C; this item is actually broader than the others)
Really?
We do this tons in 2ndQuadrant, to the point where BDR3 is an extension on top of pglogical3. It works very well and we've had hardly any issues.
The most crucial thing is using rendezvous variables - see find_rendezvous_variable in fmgr.h. These are woefully under-documented and at some point I'd love to change that because I concocted a number of schemes ranging from awful to merely ugly to help extensions find each others' API exports before Petr spotted rendezvous variables.
The PostgreSQL extension docs could definitely use some additions to cover bgworkers, discussion of DirectFunctionCallN, caching fmgrinfo, etc too. One day...
Still too much is_superuser() (possibly also broader)
I almost wonder if we should be able to hook is_superuser, but I doubt it'd be much use without having a way to determine the calling context. Better to just fix the is_superuser() stuff to use proper privileges.
On 7/30/19 7:43 PM, Nasby, Jim wrote: > >> On Jul 30, 2019, at 10:14 AM, Daniel Gustafsson <daniel@yesql.se> wrote: >> >>> On 30 Jul 2019, at 16:35, Christophe Pettus <xof@thebuild.com> wrote: >>> >>>> On Jul 30, 2019, at 06:42, Jonathan S. Katz <jkatz@postgresql.org> wrote: >>>> >>>> (and while I do like blue for the color of the bike ;) I think that >>>> "distributions" is too close to "packaging" and could be confusing. >>>> >>>> Why I suggested "-extensions" is that it appears to be inclusive of >>>> things that augment PostgreSQL abilities, be they actual PG extensions >>>> or forks. >>> >>> +1 for "-extensions" over "-distributions". An extension generally doesn't imply an entire distribution. When I firstsaw "-distributions", my thought was, "Don't we have that list already?” >> >> Fair enough. As long as we paint it a color, regardless of which, I’m happy to >> park there. > > My concern about calling this -extensions is that it’s really meant for people that are at least at the level of C extensions(and probably only those making use of hooks). That’s a relatively small number of extensions, even if they aresome of the most important extensions. Well, that may become self-evident based on the discussions on the -extensions mailing list ;) Jonathan
Вложения
"Jonathan S. Katz" <jkatz@postgresql.org> writes:
> On 7/30/19 7:43 PM, Nasby, Jim wrote:
>> My concern about calling this -extensions is that it’s really meant for people that are at least at the level of C
extensions(and probably only those making use of hooks). That’s a relatively small number of extensions, even if they
aresome of the most important extensions.
> Well, that may become self-evident based on the discussions on the
> -extensions mailing list ;)
Yeah, but "-forks" doesn't seem to capture the proposed discussion
space very well either. I also dislike that name because it seems to
be encouraging people to use community resources to discuss projects
that are, um, not exactly well connected to the community.
"-extensions" seems better from that standpoint.
Somebody previously suggested "pgsql-extension-hackers", which is a
bit long but might help to push the technical content towards the
level Jim is hoping for.
regards, tom lane
On 7/31/19 11:31 AM, Tom Lane wrote: > "Jonathan S. Katz" <jkatz@postgresql.org> writes: >> On 7/30/19 7:43 PM, Nasby, Jim wrote: >>> My concern about calling this -extensions is that it’s really meant for people that are at least at the level of C extensions(and probably only those making use of hooks). That’s a relatively small number of extensions, even if they aresome of the most important extensions. > >> Well, that may become self-evident based on the discussions on the >> -extensions mailing list ;) > > Yeah, but "-forks" doesn't seem to capture the proposed discussion > space very well either. I also dislike that name because it seems to > be encouraging people to use community resources to discuss projects > that are, um, not exactly well connected to the community. > "-extensions" seems better from that standpoint. > > Somebody previously suggested "pgsql-extension-hackers", which is a > bit long but might help to push the technical content towards the > level Jim is hoping for. +1 for -extension-hackers as an alternative. Jonathan
Вложения
Hi, On 2019-07-31 00:43:11 +0000, Nasby, Jim wrote: > My concern about calling this -extensions is that it’s really meant for people that are at least at the level of C extensions(and probably only those making use of hooks). That’s a relatively small number of extensions, even if they aresome of the most important extensions. My concern with this proposed list is that I'm not clear what is gained by having it separate. If it's effectively mainly about how to best survive being a fork, I don't see why pg.o should host it. If it's mainly about making postgres more extensible, I don't think it's good to have this discussion separate from -hackers, that'll just lead to proposing changes that won't fly (or all more senior hackers need to subscribe). Greetings, Andres Freund
> On Jul 31, 2019, at 11:53 AM, Andres Freund <andres@anarazel.de> wrote: > > Hi, > > On 2019-07-31 00:43:11 +0000, Nasby, Jim wrote: >> My concern about calling this -extensions is that it’s really meant for people that are at least at the level of C extensions(and probably only those making use of hooks). That’s a relatively small number of extensions, even if they aresome of the most important extensions. > > My concern with this proposed list is that I'm not clear what is gained > by having it separate. If it's effectively mainly about how to best > survive being a fork, I don't see why pg.o should host it. If it's > mainly about making postgres more extensible, I don't think it's good to > have this discussion separate from -hackers, that'll just lead to > proposing changes that won't fly (or all more senior hackers need to > subscribe). Fair points, and I have pondered the idea of just doing a google group or something similar. However, I’d still prefer tosee this on pg.o because I’d rather see us expanding the definition of community rather than contracting it. But if folkswould rather pg.o not host this, that’s OK. In either case, any concrete proposals, etc would certainly get moved over to -hackers. I don’t think any of us are tryingto hide anything here, just reduce the (already enormous) volume of -hackers. FWIW I’m happy with the color -extension-hackers.
On 2019-08-01 02:53, Andres Freund wrote: <snip> > If it's effectively mainly about how to best > survive being a fork, I don't see why pg.o should host it. Well, they'd clearly be PG related and encouraging people to experiment and try things out is part of how our Community grows. Some of those forks may never contribute to the PG Community, but there's going to be overlap in people/staff and it's pretty likely being friendly will work out better. :) + Justin
> On Jul 31, 2019, at 13:32, Justin Clift <justin@postgresql.org> wrote: > > Some of those forks may never contribute to the PG Community, > but there's going to be overlap in people/staff and it's pretty > likely being friendly will work out better. :) Anything we can do to help fork teams integrate better with the community ecosystem, the better. -- -- Christophe Pettus xof@thebuild.com
Greetings, * Nasby, Jim (nasbyj@amazon.com) wrote: > My concern about calling this -extensions is that it’s really meant for people that are at least at the level of C extensions(and probably only those making use of hooks). That’s a relatively small number of extensions, even if they aresome of the most important extensions. There seems to me, anyway, to be an even smaller set of extensions that *aren't* at the C level, so I don't really buy off on this argument. -extensions seems like a good name to me. > In case it adds some better perspective, here’s the list of pain points that were captured in the BoF: > > Pain for forks due to high velocity of upstream commits If you manage to succeed in addressing this, I think we would all be the worse off for it. I certainly don't think it's the purview of some relatively minor list to try and influence that... > Forks can break extension compatibility That's kind of a "don't do that" on the part of the forks, isn't it..? Or, at least accept it when you do. ;) > Backend static functions / structs I'd love to see more capability available through the common library for inspecting things with both core tools (i.e. pg_controldata, pg_waldump) and for external tools to leverage. One of the big challenges here is that we don't currently have any idea of multi-version support, even though a lot of external tools deal with multiple versions and therefore end up writing their own. Having that done in the common library would allow those external tools to leverage it, along with tools like pg_controldata. This should really be discussed on -hackers though. > Not enough hooks > Ordering of hook registration > More pluggable modules/components > Extension upgrading is difficult > Not enough test coverage for hooks > Hard to extend grammar (add more WITH options?) > Improvements to typmodin() > Problems with extensions calling other extensions (both SQL and C; this item is actually broader than the others) Should all be discussed on -hackers... > Still too much is_superuser() (possibly also broader) This should be discussed on -hackers (and feel free to CC me, given my personal vendetta against is_superuser()...). Thanks, Stephen
Вложения
Greetings, * Nasby, Jim (nasbyj@amazon.com) wrote: > > On Jul 31, 2019, at 11:53 AM, Andres Freund <andres@anarazel.de> wrote: > > On 2019-07-31 00:43:11 +0000, Nasby, Jim wrote: > >> My concern about calling this -extensions is that it’s really meant for people that are at least at the level of C extensions(and probably only those making use of hooks). That’s a relatively small number of extensions, even if they aresome of the most important extensions. > > > > My concern with this proposed list is that I'm not clear what is gained > > by having it separate. If it's effectively mainly about how to best > > survive being a fork, I don't see why pg.o should host it. If it's > > mainly about making postgres more extensible, I don't think it's good to > > have this discussion separate from -hackers, that'll just lead to > > proposing changes that won't fly (or all more senior hackers need to > > subscribe). > > Fair points, and I have pondered the idea of just doing a google group or something similar. However, I’d still preferto see this on pg.o because I’d rather see us expanding the definition of community rather than contracting it. Butif folks would rather pg.o not host this, that’s OK. Uh, status quo is *not* "contracting" the community, by definition, it's status quo. > In either case, any concrete proposals, etc would certainly get moved over to -hackers. I don’t think any of us are tryingto hide anything here, just reduce the (already enormous) volume of -hackers. By having proposals discussed somewhere other than hackers, only to have a bunch of discussion that the core hackers have to read through at some later point, or be on that list for? Doesn't sound particularly ideal to me. If we're talking about changes to core, they should be discussed on -hackers, not on some other list.. Thanks, Stephen
Вложения
On 2019-07-30 15:41, Magnus Hagander wrote: > On that note, I'd be a +1 for "pgsql-extensions" as it would > certainly > help facilitate above suggested communication. It would be all > encompassing of extensions, forks, etc. > > > Sounds like a better name and focus. > > > +1. > > I think the other part is, what would be the "level" of focus. Is this > for people building the extensions, or people using them? If it's for > people working on them, perhaps it should be pgsql-extensions-hackers or > similar, to make that clear? My interpretation of those names involving "extension" would be that that is a mailing list where the authors of plr, orafce, and ip4r hang out and discuss battling the idiosyncrasies of pgxs. Which might be useful (probably not), but it's totally not what this mailing list proposal is meant for. I'm personally not opposed to pgsql-forks, but I see that this name could be contentious. How about something like pgsql-derived-hackers? -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
I'm on-board with that color of paint for the shed. (FWIW, I'm not actually opposed to -extensions either; worst-case
wesplit that list later if needed.)
On 8/7/19, 5:08 AM, "Peter Eisentraut" <peter.eisentraut@2ndquadrant.com> wrote:
I'm personally not opposed to pgsql-forks, but I see that this name
could be contentious. How about something like pgsql-derived-hackers?
On Thu, 8 Aug 2019 at 04:03, Nasby, Jim <nasbyj@amazon.com> wrote:
I'm on-board with that color of paint for the shed. (FWIW, I'm not actually opposed to -extensions either; worst-case we split that list later if needed.)
Right. Does it matter, really? People working on these things will be well enough connected to -hackers and the community.
Personally the only advantage I see to splitting it off -hackers is that it makes posts related to addins/forks/extensions/spoons/magic unicorn fairies/whatever more visible to that interest group.
Greetings, * Craig Ringer (craig@2ndquadrant.com) wrote: > On Thu, 8 Aug 2019 at 04:03, Nasby, Jim <nasbyj@amazon.com> wrote: > > I'm on-board with that color of paint for the shed. (FWIW, I'm not > > actually opposed to -extensions either; worst-case we split that list later > > if needed.) > > Right. Does it matter, really? People working on these things will be well > enough connected to -hackers and the community. > > Personally the only advantage I see to splitting it off -hackers is that it > makes posts related to addins/forks/extensions/spoons/magic unicorn > fairies/whatever more visible to that interest group. I agree that makes sense- if what they're talking about are forks/extensions/unicorns, but the list of topics provided previously looked like things that the fork folks want changed in core, and anything along those lines belongs on hackers, not on some other list. This really seems like it's going to lead in a direction where the various forks discuss on some other list things they want to see in core (such as reducing the velocity of commits to core...), and then they're going to spend a bunch of time on it and eventually propose something on -hackers that ends up getting shot down, and I don't really think that's going to be very satisfying for anyone. Let's keep the discussion about changes to core on -hackers. If there's folks who want a list to get together and discuss how to address certain limitations and how to work around them in the extensions system or the set of hooks that are provided, that's great, but it should be clear that it's that and not intended to be some kind of alternative list for discussing core changes. Thanks, Stephen
Вложения
> On Aug 7, 2019, at 9:07 PM, Stephen Frost <sfrost@snowman.net> wrote: > > Greetings, > > * Craig Ringer (craig@2ndquadrant.com) wrote: >> Personally the only advantage I see to splitting it off -hackers is that it >> makes posts related to addins/forks/extensions/spoons/magic unicorn >> fairies/whatever more visible to that interest group. Yes, that’s definitely one of the advantages. > This really seems like it's going to lead in a direction where the > various forks discuss on some other list things they want to see in core > (such as reducing the velocity of commits to core...), and then they're > going to spend a bunch of time on it and eventually propose something on > -hackers that ends up getting shot down, and I don't really think that's > going to be very satisfying for anyone. No one is suggesting reducing commit velocity. > Let's keep the discussion about changes to core on -hackers. If there's > folks who want a list to get together and discuss how to address certain > limitations and how to work around them in the extensions system or the > set of hooks that are provided, that's great, but it should be clear > that it's that and not intended to be some kind of alternative list for > discussing core changes. The majority of people in the BoF at pgCon that spurred this are more than experienced enough to know that nothing will moveforward without full discussion on -hackers. This list is not meant to be any sort of replacement for that process.
On 8/7/19 03:08, Peter Eisentraut wrote:
I'm personally not opposed to pgsql-forks, but I see that this name could be contentious. How about something like pgsql-derived-hackers?
I don't like the word "forks" either. I like Peter's idea "pgsql-derivatives" though I wonder if there's still a better word we haven't thought of yet. There's a significant difference between the idea of people forking PostgreSQL code and the idea of downstream builds that are supplementing PostgreSQL capabilities but regularly merging from upstream (even if they aren't presently able to package as extensions).
-Jeremy
-- Jeremy Schneider Database Engineer Amazon Web Services
> On Aug 8, 2019, at 10:55, Jeremy Schneider <schnjere@amazon.com> wrote: > There's a significant difference between the idea of people forking PostgreSQL code and the idea of downstream builds thatare supplementing PostgreSQL capabilities but regularly merging from upstream (even if they aren't presently able topackage as extensions). I think that's a distinction without a difference. Most "forks" these days (unless we're talking about ones that are veryhistorical) merge regularly from upstream. -- -- Christophe Pettus xof@thebuild.com
On 31/07/2019 18:37, Jonathan S. Katz wrote: > On 7/31/19 11:31 AM, Tom Lane wrote: >> "Jonathan S. Katz" <jkatz@postgresql.org> writes: >>> On 7/30/19 7:43 PM, Nasby, Jim wrote: >>>> My concern about calling this -extensions is that it’s really meant for people that are at least at the level of C extensions(and probably only those making use of hooks). That’s a relatively small number of extensions, even if they aresome of the most important extensions. >> >>> Well, that may become self-evident based on the discussions on the >>> -extensions mailing list ;) >> >> Yeah, but "-forks" doesn't seem to capture the proposed discussion >> space very well either. I also dislike that name because it seems to >> be encouraging people to use community resources to discuss projects >> that are, um, not exactly well connected to the community. >> "-extensions" seems better from that standpoint. >> >> Somebody previously suggested "pgsql-extension-hackers", which is a >> bit long but might help to push the technical content towards the >> level Jim is hoping for. > > +1 for -extension-hackers as an alternative. > +1 For this too. -- Petr Jelinek 2ndQuadrant - PostgreSQL Solutions for the Enterprise https://www.2ndQuadrant.com/
Hi, On 08/08/2019 04:07, Stephen Frost wrote: > Greetings, > > * Craig Ringer (craig@2ndquadrant.com) wrote: >> On Thu, 8 Aug 2019 at 04:03, Nasby, Jim <nasbyj@amazon.com> wrote: >>> I'm on-board with that color of paint for the shed. (FWIW, I'm not >>> actually opposed to -extensions either; worst-case we split that list later >>> if needed.) >> >> Right. Does it matter, really? People working on these things will be well >> enough connected to -hackers and the community. >> >> Personally the only advantage I see to splitting it off -hackers is that it >> makes posts related to addins/forks/extensions/spoons/magic unicorn >> fairies/whatever more visible to that interest group. > > I agree that makes sense- if what they're talking about are > forks/extensions/unicorns, but the list of topics provided previously > looked like things that the fork folks want changed in core, and > anything along those lines belongs on hackers, not on some other list. > > This really seems like it's going to lead in a direction where the > various forks discuss on some other list things they want to see in core > (such as reducing the velocity of commits to core...), and then they're > going to spend a bunch of time on it and eventually propose something on > -hackers that ends up getting shot down, and I don't really think that's > going to be very satisfying for anyone. > > Let's keep the discussion about changes to core on -hackers. If there's > folks who want a list to get together and discuss how to address certain > limitations and how to work around them in the extensions system or the > set of hooks that are provided, that's great, but it should be clear > that it's that and not intended to be some kind of alternative list for > discussing core changes. > So when somebody wants to do some core changes in your company, they never discuss it with their colleagues before posting to -hackers? I see this list as two main purposes (at least for me, and I wasn't at BoF): - help each other using hooks/apis when there does not seem to be straightforward way of doing something (this does not belong to -hackers IMHO and there is no other C level hacker mailing list) - do initial discussion on ideas for extensibility enhancements of PostgreSQL - it does help to know what you are proposing is useful for others or that it can be ironed out to be more broadly useful -- Petr Jelinek 2ndQuadrant - PostgreSQL Solutions for the Enterprise https://www.2ndQuadrant.com/
So when somebody wants to do some core changes in your company, they
never discuss it with their colleagues before posting to -hackers?
Exactly. Or even build the whole feature for a customer and deliver it before trying to upstream the functionality in the next community release, for that matter.
That happens whether it's a solo developer or a team in a company. Letting people with common interests find out about work earlier and potentially make suggestions to help it meet their needs too and/or make it more likely to be acceptable upstream seems unlikely to be a bad thing.
It won't stop anyone going off and writing something that meets their immediate needs and is utterly unsuitable for upstream. But they can already do that now.
In fact I'd argue that given how Pg development works, people are often almost required to write it first anyway. Everyone's so busy that it can be very hard to get attention from key stakeholders on early design discussions or prototype work, so often one must develop a believable implementation of a feature and add it to a CF before it gets much attention. At that point as often or not the original developer may be told "that's an awful way to do it, you can't do that because of X, go do it $otherway instead," so they go scrap their work and start again. There aren't any easy ways to prevent that really as it's a pretty natural evolution of the development process, communication style and decision making approach used in the community. We don't really have "subsystem owners" or anything and most key devs can effectively veto something, but we have too much being developed to let everyone who might object examine the initial design instead of a much quicker and easier to study concrete testable implementation.
I don't actually much care if this list is created or not, though if it's not I'd like to see more active discussion of addins/forks/extensions/etc encouraged on hackers proper. I very much do care about improving collaboration between the companies and teams working on extensions to reduce wasted work and to make the core of postgres more extensible and pluggable.
- help each other using hooks/apis when there does not seem to be
straightforward way of doing something (this does not belong to -hackers
IMHO and there is no other C level hacker mailing list)
Absolutely.
I scratched out a wiki page to start on this by the way. I will link it from the main TODO wiki page soon. The goal is to let people get together and identify common sets of needs for hooks/callbacks/tracepoints/plugin APIs/etc so that they can be added in an organised manner.
I've copied my existing notes on the tracepoints and accounting I want to add for logical decoding and the walsender as a starting point.
- do initial discussion on ideas for extensibility enhancements of
PostgreSQL - it does help to know what you are proposing is useful for
others or that it can be ironed out to be more broadly useful
Yep, or if someone else already solved it in a way that doesn't require core changes in the first place. There is so much of PostgreSQL that you just have to know about because we don't have much in the way of overview documentation on extending PostgreSQL at a scale beyond writing C functions.
I started some notes on that too with the intention of helping out people who're getting on board with extension development or joining teams that work on complex extensions. At some point I'd like to tidy these notes and add them to the core docs in the extending the server section but I'll be unlikely to find the time to organise, xref and DocBook-ify them any time soon so for now they're in a wiki too:
I need to dig up Andres's old notes on how logical decoding works internally and a few other things to try to merge into that and/or the main docs too, but that'll have to wait until a lull in my main development priorities.