On Wed, Aug 12, 2015 at 3:10 AM, Noah Misch <noah@leadboat.com> wrote:
> Committers press authors to delete tests more often than we press them to
> resubmit with more tests. No wonder so many patches have insufficient tests;
> we treat those patches more favorably, on average. I have no objective
> principles for determining whether a test is pointlessly redundant, but I
> think the principles should become roughly 10x more permissive than the
> (unspecified) ones we've been using.
I would suggest the metric should be "if this test fails is it more
likely to be noise due to an intentional change in behaviour or more
likely to be a bug?"
The only time I've seen pushback against tests is when the test author
made valiant efforts to test every codepath and the expected output
embeds the precise behaviour of the current code as "correct". Even
when patches have extensive tests I don't recall seeing much pushback
(though I've been having trouble keeping up with the list in recent
months) if the tests are written in a way that they will only fail if
there's a bug, even if behaviour changes in unrelated ways.
--
greg