Robert Haas <robertmhaas@gmail.com> writes:
> Well, I wasn't suggesting adding a lot more testing of things that
> we're already testing. I was assuming that we would craft the
> additional tests to hit areas that we are not now covering well. My
> point here is only to what Peter said upthread: we want to be able to
> get positive results rather than waiting for "enough" negative results
> (whatever that means). To get positive results, you must have a test
> suite. While letting beta testers test whatever they want has some
> value, testing things we think might be likely hiding places for bugs
> (such as WAL recovery) has merit, too. Making those tests
> well-organized and easily repeatable is, IMHO, a Good Thing.
The problem here is the "easily repeatable" bit. Almost by definition,
easily repeatable tests don't find hard-to-reproduce problems. I don't
mean to suggest that they're without value, but they are no substitute
for beta testers doing unpredictable things.
regards, tom lane