Re: Formatting Curmudgeons WAS: MMAP Buffers

On Thu, Apr 21, 2011 at 2:43 AM, Peter Eisentraut <peter_e@gmx.net> wrote:
> On Wed, 2011-04-20 at 21:09 -0400, Robert Haas wrote:
>> But
>> even then I think we'd have this problem of people being unwilling to
>> give up on jamming stuff into a release, regardless of the scheduling
>> impact of doing so.  I actually think the problem of getting releases
>> out on time is a *much* bigger problem for us than how long or short
>> CommitFests are.
>
> I think to really address that problem, you need to think about shorter
> release cycles overall, like every 6 months.  Otherwise, the current 12
> to 14 month horizon is just too long psychologically.

I agree.  I am in favor of a shorter release cycle.  But I think that
a shorter release cycle won't work well if there is still four month
long integration period at the end of each series of CommitFests.  The
problem is a bit circular here: because release cycles are long,
people really, really want to slip as much as possible in at the end.
But being under time pressure to get things committed results in a
higher bug count, which means more things that have to be fixed after
feature freeze, which translates into a long release cycle.

I think that it's not too bad if the process of a release getting out
the door results in effectively missing one CommitFest.  For example,
if we imagine one-month CommitFests starting every two months, and we
had a CommitFest starting on January 15th, it wouldn't be too painful
if we skipped a hypothetical March 15th CommitFest to get the release
done, and then started up the process again on May 15th.  However, in
practice, what happens is we miss *two* CommitFests: the expectation
is that the next CommitFest will be on the order of July 15th, which
is just too long.  Similarly, if we did shorter CommitFests and
shorter releases - say, five one-week-a-month CommitFests in July,
August, September, October, and November, I'd want to kick a release
out in December and reopen for development in January, not get stuck
with the same six-month feature freeze we have now, or even a
four-month feature freeze.  But that isn't going to work if people do
the same sort of throwing everything into the kitchen sink at the last
minute that we have been doing for at least the last couple of
releases.

In fact, I don't believe that the current CF cycle really forces a
huge amount of waiting-for-feedback.  It's true that if you submit a
patch at a randomly chosen time, you will have to wait up to two
months for a CommitFest to start, and then you might not get a review
until late in the CommitFest, so it could take you up to three months
to get a review.  In practice, patches are not submitted at random
times - in fact, probably 50% of the patches come in during the last
week before the CF starts, and typically perhaps 50% of the patches
get a review in the first week, and maybe 80% within the first two
weeks.   Some patches also get an initial review between CommitFests,
which further improves the average.  Overall, I bet the average time
between patch submission and first review is <3 weeks.  You can
typically get 2 or 3 followup reviews during the same cycle with only
a few days latency for each.  Even though it would be nice to do
better, for an all-volunteer project, I think it's respectable.   I
can't say the same thing about our process from getting from feature
freeze to release.  It's really long, and it's nearly all fixing bugs
in code that was committed in the last CF, and the last CF produces
exponentially more bugs than the earlier ones, and it's often the case
that people don't fix their own bugs and someone else has to jump in
to pick up the slack.  Meanwhile, the regular flow of reviewing and
committing patches is completely disrupted; and once in a while
someone gets flamed for so much as bringing up a new feature that
they're interested in working on for the next release (which I think
is totally unwarranted; now is the PERFECT time to begin roughing out
plans for 9.2 work... but I digress).

So while I'm mildly interested in the idea of shifting the CF cycle
around to provide more timely review, I can't really get that excited
about it, especially if there's any risk that we are just shifting
more of the work from the CommitFest cycle to the
end-of-release-interminable-integration-period.  However, if there's
some way of avoiding the phenomenon where all hell breaks loose
because people jam four major new features into the tree in as many
weeks, sign me up.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company