Re: [CORE] postpone next week's release

On Sat, May 30, 2015 at 10:47:27PM +0200, Andres Freund wrote:
> > The bottom line is that we just can't keep going on like this.  The fact
> > we put out a release two weeks ago, then need to put out a fix release
> > for that, but we have more multi-xact bugs to fix and can't decide if we
> > should do one or two minor releases, and are pushing out an alpha of 9.5
> > because we know we aren't ready for a beta, just confirms my analysis.
> 
> I don't think that alone confirms very much.

Huh?  In what world is that release timeline ever reasonable?  It points
to a serious problem.

> > I hate to be the bearer of bad news, but I think bad news is what we
> > must face.
> 
> Well, the question is what we do with that observation. Personally I
> think it's not a new one. This point has been made repeatedly, including
> at most if not all developer meetings I attended. I definitely had
> conversations around it both in person, on IM and on list.

Well, I think we stop what we are doing, focus on restructuring,
testing, and reviewing areas that historically have had problems, and
when we are done, we can look to go to 9.5 beta.  What we don't want to
do is to push out more code and get back into a
wack-a-bug-as-they-are-found mode, which obviously did not serve us well
for multi-xact, and which is what releasing a beta will do, and of
course, more commit-fests, and more features.  

If we have to totally stop feature development until we are all happy
with the code we have, so be it.  If people feel they have to get into
cleanup mode or they will never get to add a feature to Postgres again,
so be it.  If people say, heh, I am not going to do anything and just
come back when cleanup is done (by someone else), then we will end up
with a smaller but more dedicated development team, and I am fine with
that too.  I am suggesting that until everyone is happy with the code we
have, we should not move forward.  Forget 9.5 feature testing --- we
don't even have 9.3 and 9.4 working to my satisfaction yet, and I bet
others share my opinion.  We do not want to look back on this period and
say _this_ is when Postgres lost its reputation for reliability, and
when other databases took that reputation from us.

> I don't think it's primarily a problem of lack of review; although that
> is a large problem.  I think the biggest systematic problem is that the
> compound complexity of postgres has increased dramatically over the
> years.  Features have added complexity little by little, each not
> incrementally not looking that bad.  But very little has been done to
> manage complexity. Since 8.0 the codesize has roughly doubled, but
> little has been done to manage the increased complexity. Few new
> abstractions have been introduced and the structure of the code is
> largely the same.
> 
> As a somewhat extreme example, let's look at StartupXLOG(). In 8.0 it
> was ~500 LOC, in master it's ~1500.  The interactions in 8.0 were
> complex, they have gotten much more complex since.  It fullfills lots of
> different roles, all in one function:

Yep, great please to start our work.

> So, I think we have built up a lot of technical debt. And very little
> effort has been made to fix that; and in the cases where people have the
> reception has often been cool, because refactoring things obviously will
> destabilize in the short term, even if it fixes problems in the long
> term.  I don't think that's sustainable.

Agreed.

> We can't improve the situation by just delaying the 9.5 release or
> something like that. We need to actively work on making the codebase
> easier to understand and better tested. But that is actual development
> work, and shouldn't happen at the tail end of a release.

It should start right now, and then, once we are happy with our code, we
can take periodic breaks to revisit the exact issues you describe.  What
I am saying is that we shouldn't wait until after 9.5 beta or after 9.5
final, or after the next commitfest or whatever.  We have already waited
too long to do this.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +