Обсуждение: Commitfest statistics

Hi, hackers!

I want to share some stats and thoughts about CF.

***
The first is a graph with the numbers of committed, moved, returned, and rejected CF patches over time - [cf_items_status.png]. Credits to Dmitry Dolgov for sharing his scripts to gather this stat.

***
Besides, I noticed that we have a lot of long-living discussions. And I was curious what is the chance to get something committed after several CFs. The graph is in [num_commitfests.png]. So, most entries make it to release after just one or two commitfests.

I think that the issue here is that the commitfest application now serves two different purposes:

Firstly, we use it to track patches that we want to see in the nearest releases and concentrate our efforts on. And current CFM guideline [1] reflects this idea. It suggests, that after the commitfest closure date we relentlessly throw to RWF patches that got at least some feedback. To be honest, I was reluctant to return semi-ready patches, because it means that they will get lost somewhere in mailing lists. And it seems like other CFMs did the same.
On the other hand, we use Commitfest to track important entries that we want to remember at least once in a while. You can find many examples in the 'Bug Fixes' group of patches. They are too serious to move them to TODO list, yet too complex and/or rare to move on. And such entries simply move from one CF to another.

I wonder if we can improve the workflow somehow? Todo list was recently cleaned up, so maybe we can use it? Or we could add a special 'Backlog' section to the commitfest application.

What do you think?


***
I am also planning to update the CommitFest Checklist:
- remove references to pgsql-rrreviewers;
- add info about cfbot;
- remove the 'Sudden Death Overtime' chapter as it no longer reflects reality.

Thoughts?

[1] https://wiki.postgresql.org/wiki/CommitFest_Checklist#Sudden_Death_Overtime

-- 
Anastasia Lubennikova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Anastasia Lubennikova <a.lubennikova@postgrespro.ru> writes:
> I want to share some stats and thoughts about CF.

First, thanks again for managing this CF!

> The first is a graph with the numbers of committed, moved, returned, and 
> rejected CF patches over time - [cf_items_status.png]. Credits to Dmitry 
> Dolgov for sharing his scripts to gather this stat.

Yeah, that is a very interesting graph.  It shows that our actual
throughput of resolving patches has held more or less steady, which
isn't very surprising because the available person-power has not
changed much in the last couple of years.  But it looks like the
number of cans getting kicked down the road is progressively
increasing.  That's not something we can sustain indefinitely.

> Besides, I noticed that we have a lot of long-living discussions. And I 
> was curious what is the chance to get something committed after several 
> CFs. The graph is in [num_commitfests.png]. So, most entries make it to 
> release after just one or two commitfests.

It's hard to see anything in this graph about what happens after the
first couple of CFs.  Maybe if you re-did it with a log Y axis, the
tail would be more readable?

> I think that the issue here is that the commitfest application now 
> serves two different purposes:

> Firstly, we use it to track patches that we want to see in the nearest 
> releases and concentrate our efforts on. And current CFM guideline [1] 
> reflects this idea. It suggests, that after the commitfest closure date 
> we relentlessly throw to RWF patches that got at least some feedback. To 
> be honest, I was reluctant to return semi-ready patches, because it 
> means that they will get lost somewhere in mailing lists. And it seems 
> like other CFMs did the same.
> On the other hand, we use Commitfest to track important entries that we 
> want to remember at least once in a while. You can find many examples in 
> the 'Bug Fixes' group of patches. They are too serious to move them to 
> TODO list, yet too complex and/or rare to move on. And such entries 
> simply move from one CF to another.

Yeah, the aggressive policy suggested in "Sudden Death Overtime" is
certainly not what's been followed lately.  I agree that that's
probably too draconic.  On the other hand, if a patch sits in the
queue for several CFs without getting committed, that suggests that
maybe we ought to reject it on the grounds of "apparently nobody but
the author cares about this".  That argument is easier to make for
features than bug fixes of course, so maybe the policy needs to
distinguish what kind of change is being considered.

            regards, tom lane

On 2020-Dec-07, Tom Lane wrote:

> Anastasia Lubennikova <a.lubennikova@postgrespro.ru> writes:

> > Firstly, we use it to track patches that we want to see in the nearest 
> > releases and concentrate our efforts on. And current CFM guideline [1] 
> > reflects this idea. It suggests, that after the commitfest closure date 
> > we relentlessly throw to RWF patches that got at least some feedback. To 
> > be honest, I was reluctant to return semi-ready patches, because it 
> > means that they will get lost somewhere in mailing lists. And it seems 
> > like other CFMs did the same.
> 
> Yeah, the aggressive policy suggested in "Sudden Death Overtime" is
> certainly not what's been followed lately.  I agree that that's
> probably too draconic.  On the other hand, if a patch sits in the
> queue for several CFs without getting committed, that suggests that
> maybe we ought to reject it on the grounds of "apparently nobody but
> the author cares about this".  That argument is easier to make for
> features than bug fixes of course, so maybe the policy needs to
> distinguish what kind of change is being considered.

Note that this checklist was written in 2013 and has never been updated
since then.  I think there is nothing in that policy that we do use.
I'm thinking that rather than try to fine-tune that document, we ought
to rewrite one from scratch.

For one thing, "a beer or three" only at end of CF is surely not
sufficient.