On 2016-05-06 14:39:57 -0700, Joshua D. Drake wrote:
> > > What are we looking for exactly?
> >
> > Data corruption, efficiency problems.
> >
>
> I am really not trying to be difficult here but Data Corruption is an easy
> one... what is the metric we accept as an efficiency problem?
That's indeed not easy to define. In this case I'd say vacuums taking
longer, index only scans being slower, more WAL being generated would
count?
> > I think tests without reading the code are quite sensible and
> > important. And it perfectly makes sense to ask for information about
> > what to test. But fundamentally testing is a lot of work, as is writing
> > and reviewing code; unless you're really really good at destructive
> > testing, you won't find much in a 15 minute break.
> >
>
> Yes, this is true but with a proper testing framework, I don't need a 15
> minute break. I need 1 hour to configure, the rest just "happens" and
> reports back.
That only works if somebody writes such tests. And in that case the
tester having run will often suffice (until related changes are being
made). I'm not arguing against introducing more tests into the codebase
- I rather fervently for that. But that really isn't what's going to
avoid issues like this feature (or multixact) causing problems, because
those tests will just test what the author thought of.
> > > You want me (or people like me) to test more? Give us an easy way to
> > > do it.
> >
> > Useful additional testing and easy just don't go well together. By the
> > time I have made it easy I've done the testing that's needed.
>
> I don't know that I can agree with this. A proper harness allows you to
> execute: go.sh and boom... 2, 4, even 8 hours later you get a report. I will
> not argue that it isn't easy to implement but I know it can be done.
The problem is that the contents of go.sh are the much more relevant
part than the 8 hours.
Greetings,
Andres Freund