Re: CF3+4 (was Re: Parallel query execution)

On 2013-01-23 11:44:29 -0500, Robert Haas wrote:
> On Tue, Jan 22, 2013 at 1:15 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Yeah, and a lot more fairly-new developers who don't understand all the
> > connections in the existing system.  Let me just push back a bit here:
> > based on the amount of time I've had to spend fixing bugs over the past
> > five months, 9.2 was our worst release ever.  I don't like that trend,
> > and I don't want to see it continued because we get laxer about
> > accepting patches.  IMO we are probably too lax already.
> 
> Really?  Hmm, that's not good.  I seem to recall 8.4.x being pretty
> bad, and some of the recent bugs we fixed were actually 9.1.x problems
> that slipped through the cracks.

FWIW I concur with Tom's assessment.

> On Tue, Jan 22, 2013 at 1:15 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > For a very long time we've tried to encourage people to submit rough
> > ideas to pgsql-hackers for discussion *before* they start coding.
> > The point of that is to weed out, or fix if possible, (some of) the bad
> > ideas before a lot of effort has gotten expended on them.  It seems
> > though that less and less of that is actually happening, so I wonder
> > whether the commitfest process is encouraging inefficiency by making
> > people think they should start a discussion with a mostly-complete
> > patch.  Or maybe CF review is just crowding out any serious discussion
> > of rough ideas.  There was some discussion at the last dev meeting of
> > creating a "design review" process within commitfests, but nothing got
> > done about it ...

Are you thinking of specific patches? From what I remember all all the
bigger patches arround the 9.3 cycle were quite heavily discussed during
multiple stages of their development.

I agree that its not necessarily the case for smaller patches but at
least for me in those cases its often hard to judge how complex
something is before doing an initial prototype.

One aspect of this might be that its sometimes easier to convince
-hackers of some idea if you can prove its doable.
Another that the amount of bikeshedding seems to be far lower if a patch
already shapes a feature in some way.

> I think there's been something of a professionalization of PostgreSQL
> development over the last few years.   More and more people are able
> to get paid to work on PostgreSQL as part or in a few cases all of
> their job.  This trend includes me, but also a lot of other people.

Yes.

> There are certainly good things about this, but I think it increases
> the pressure to get patches committed.  If you are developing a patch
> on your own time and it doesn't go in, it may be annoying but that's
> about it.  If you're getting paid to get that patch committed and it
> doesn't go in, either your boss cares or your customer cares, or both,
> so now you have a bigger problem.

And it also might shape the likelihood of getting paid for future work,
be that a specific patch or time for hacking/maintenance.

> For all of that, I'm not sure that people failing to seek consensus
> before coding is really so much of a problem as you seem to think.  A
> lot of the major efforts underway for this release have been discussed
> extensively on multiple threads.  The fact that some of those ideas
> may be less than half-baked at this point may in some cases be the
> submitter's fault, but there also cases where it's just that they
> haven't got enough looking-at from the people who know enough to
> evaluate them in detail, and perhaps some cases that are really
> nobody's fault: nothing in life is going to be perfect all the time no
> matter how hard everyone tries.

I agree. Take logical replication/decoding for example. While we
developed a prototype first, we/I tried to take in as much feedback from
the community as we could. Just about no code, and only few concepts,
from the initial prototype remain, and thats absolutely good.
I don't think a more "talk about it first" approach would have helped
here. Do others disagree?

But as soon as the rough, rough design was laid out the amount of
specific feedback shrank. Part of that is that some people involved in
the discussions had changes in their work situation that influenced the
amount of time they could spend on it, but I think one other problem is
that at some point it gets hard to give feedback on a complex, evolving
patch without it eating up big amount of time.

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services