Bruce Momjian <maillist@candle.pha.pa.us> writes:
> The author did test with the regression tests. In fact, the
> regression tests are not up-to-date, so there are meny diffs even when
> the code works, and we can't expect someone to keep the regression
> tests spotless at all times.
Actually, I sympathize with David on this: I got burnt the same way
just a couple weeks ago. (I blithely assumed that the regression tests
would test copy in/out ... they don't ...)
Perhaps the real lesson to be learned is that a little more effort
should be expended on the regression tests. I have a couple of
suggestions:
1. As far as I've seen there is no documentation on how to create
regression tests. This should be documented and made as easy as
possible, to encourage people to create tests for missing cases.
2. System variations (roundoff error differences, etc) create spurious
test complaints that make it hard to interpret the results properly.
Can anything be done to clean this up?
3. The TODO list should maintain a section on missing regression tests;
any failure that gets by the regression tests should cause an entry
to get made here. This list would have a side benefit of warning
developers about areas that are not getting tested, so that they know
they have to do some hand testing if they change relevant code.
We can start the new TODO section with:
* Check destroydb. (Currently, running regression a second time checks
this, but a single run in a clean tree won't.)
* Check copy from stdin/to stdout.
* Check large-object interface.
What else have people been burnt by lately?
regards, tom lane