Re: commitfest.postgresql.org is no longer fit for purpose

On Sun, May 19, 2024 at 3:18 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Dmitry Dolgov <9erthalion6@gmail.com> writes:
> > How do we make sure this time it will be different?
>
> Things *have* changed, if you take the long view.

That's true, but I think Dmitry's point is well-taken all the same: we
haven't really made a significant process improvement in many years,
and in some ways, I think things have been slowly degrading. I don't
believe it's necessary or possible to solve all of the accumulated
problems overnight, but we need to get serious about admitting that
there is a problem which, in my opinion, is an existential threat to
the project.

> IMV one really fundamental problem is the same as it's been for a
> couple of decades: too many patch submissions, too few committers.
> We can't fix that by just appointing a ton more committers, at least
> not if we want to keep the project's quality up.  We have to grow
> qualified committers.  IIRC, one of the main reasons for instituting
> the commitfests at all was the hope that if we got more people to
> spend time reading the code and reviewing patches, some of them would
> learn enough to become good committers.  I think that's worked, again
> taking a long view.  I just did some quick statistics on the commit
> history, and I see that we were hovering at somewhere around ten
> active committers from 1999 to 2012, but since then it's slowly crept
> up to about two dozen today.  (I'm counting "active" as "at least 10
> commits per year", which is an arbitrary cutoff --- feel free to slice
> the data for yourself.)  Meanwhile the number of submissions has also
> grown, so I'm not sure how much better the load ratio is.

That's an interesting statistic. I had not realized that the numbers
had actually grown significantly. However, I think that the
new-patch-submitter experience has not improved; if anything, I think
it may have gotten worse. It's hard to compare the subjective
experience between 2008 when I first got involved and now, especially
since at that time I was a rank newcomer experiencing things as a
newcomer, and now I'm a long-time committer trying to judge the
newcomer experience. But it seems to me that when I started, there was
more of a "middle tier" of people who were not committers but could do
meaningful review of patches and help you push them in the direction
of being something that a committer might not loathe. Now, I feel like
there are a lot of non-committer reviews that aren't actually adding a
lot of value: people come along and say the patch doesn't apply, or a
word is spelled wrong, and we don't get meaningful review of whether
the design makes sense, or if we do, it's wrong. Perhaps this is just
viewing the past with rose-colored glasses: I wasn't in as good a
position to judge the value of reviews that I gave and received at
that point as I am now. But what I do think is happening today is that
a lot of committer energy gets focused on the promising non-committers
who someone thinks might be able to become committers, and other
non-committers struggle to make any headway.

On the plus side, I think we make more of an effort not to be a jerk
to newcomers than we used to. I also have a strong hunch that it may
not be as good as it needs to be.

> * Patches that sit in the queue a long time tend to be ones that lack
> consensus, either about the goal or the details of how to achieve it.
> Sometimes "lacks consensus" really means "nobody but the author thinks
> this is a good idea, but we haven't mustered the will to say no".
> But I think it's more usually the case that there are plausible
> competing opinions about what the patch should do or how it should
> do it.  How can we resolve such differences and get something done?

This is a great question. We need to do better with that.

Also, it would be helpful to have better ways of handling it when the
author has gotten to a certain point with it but doesn't necessarily
have the time/skills/whatever to drive it forward. Such patches are
quite often a good idea, but it's not clear what we can do with them
procedurally other than hit the reject button, which doesn't feel
great.

> * Another reason for things sitting a long time is that they're too
> big to review without an unreasonable amount of effort.  We should
> encourage authors to break large patches into smaller stepwise
> refinements.  It may seem like that will result in taking forever
> to reach the end goal, but dropping a huge patchset on the community
> isn't going to give speedy results either.

Especially if it has a high rate of subtle defects, which is common.

> * Before starting this thread, Robert did a lot of very valuable
> review of some individual patches.  I think what prompted him to
> start the thread was the realization that he'd only made a small
> dent in the problem.  Maybe we could divide and conquer: get a
> dozen-or-so senior contributors to split up the list of pending
> patches and then look at each one with an eye to what needs to
> happen to move it along (*not* to commit it right away, although
> in some cases maybe that's the thing to do).  It'd be great if
> that could happen just before each commitfest, but that's probably
> not practical with the current patch volume.  What I'm thinking
> for the moment is to try to make that happen once a year or so.

I like this idea.

> Yeah, all this stuff ultimately gets done "for the good of the
> project", which isn't the most reliable motivation perhaps.
> I don't see a better one...

This is the really hard part.

--
Robert Haas
EDB: http://www.enterprisedb.com