On 6/27/13 10:57 AM, Tom Lane wrote:
> Peter Eisentraut <peter_e@gmx.net> writes:
>> On 6/26/13 12:17 PM, Tom Lane wrote:
>>> (I like to
>>> point at mysql's regression tests, which take well over an hour even on
>>> fast machines. If lots of tests are so helpful, why is their bug rate
>>> no better than ours?)
>
>> Tests are not (primarily) there to prevent bugs.
>
> Really? Pray tell, what do you think they're for? Particularly code
> coverage tests, which is what these are?
Tests document the existing functionality of the code, to facilitate
refactoring. (paraphrased, Uncle Bob)
Case in point, the existing ALTER DDL code could arguably use even more
refactoring. But no one wants to do it because it's unclear what will
break. With the proposed set of tests (which I have not read to
completion), this could become quite a bit easier, both for the coder
and the reviewer. But these tests probably won't detect, say, locking
bugs in such new code. That can only be prevented by careful code
review and a clean architecture. Perhaps, MySQL has neither. ;-)
Code coverage is not an end itself, it's a tool.
In this sense, tests prevent existing functionality being broken, which
might be classified as a bug. But it's wrong to infer that because
system X has a lot of bugs and a lot of tests, having a lot of tests
must be useless.