On 02.02.2011 16:27, Kevin Grittner wrote:
> Heikki Linnakangas wrote:
>> It produces over two hundred permutations, so it's going to be a
>> bit tedious to check the output for that, but I think it's still
>> acceptable so that we don't need more complicated rules for what is
>> accepted. IOW, we can just print the output of all permutations and
>> "diff", we won't need "if step X ran before step Y, commit should
>> succeed" rules that the python version had.
>
> During initial development, I was very glad to have the one-line-
> per-permutation summary; however, lately I've been wishing for more
> detail, such as is available with this approach. At least for the
> moment, what this provides is exactly what is most useful. If there
> is ever a major refactoring (versus incremental enhancements), it
> might be worth developing a way to filter the detailed input into the
> sort of summary we were getting from dtester, but we can cross that
> bridge when (and if) we come to it.
Ok, great. Let's just add comments to the test specs to explain which
permutations should succeed and which should fail, then. And which ones
fail because they're false positives.
-- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com