Re: Learning curves and such (was Re: pgFoundry)

On Wed, May 18, 2005 at 12:07:14AM -0400, Tom Lane wrote:

> Steve Atkins  writes:
> > The useful bug tracking systems I've used have also included QA.  Any
> > bug submitted doesn't get accepted without a standalone test case.
> 
> Side note: while test cases are certainly Good Things that make life
> easier for developers, so we should encourage people to provide 'em,
> I can't say that I like the idea of a tracking system designed around
> the concept that a bug for which you don't have a test case isn't real.
> It's not all that easy to make a test case for bugs involving concurrent
> behavior.  I'd go so far as to say that most of the seriously
> interesting bugs that I've dealt with in this project were ones that the
> original reporter didn't have a reproducible test case for.

Depends on the context. For a code base like PGs (with, as you say,
many possibilities for weird concurrency issues or data driven bugs),
or for a development style like PGs (i.e. mostly volunteer),
_requiring_ a test case before a bug is worked on makes no sense.

Some environments I've worked in, though, have had a stage between "bug
submitted" and "bug passed to developer" where someone in QA makes an
attempt to create a test case where one was not submitted with the bug.
That was more the idea I was suggesting as a possibility - if someone has
a QA itch to scratch then trolling postgresql-bugs for bugs without test
cases and creating recreatable test cases to attach to those bugs might
be a useful thing to do.

Cheers, Steve