On Mon, May 30, 2022 at 12:10:04PM -0700, Andres Freund wrote:
> I think it's ok if we have a heuristic test for this kind of thing. It
> sometimes can even be good, because it means we'll get different schedulings
> over time, hitting "unknown" bugs.
Yeah, the tricky part is how to parametize that to be cheap, still
useful. Having a larger number of attributes makes the particular
problem of this thread easier to hit because it enlarges the
validation window in the concurrent build, and generating such dummy
data with a simple schema should be quick enough, but I would not
accept in the tree a test that takes ~5s to run without a rather good
hit rate. For this particular issue, I would say that something able
to fail up to 20%~25% of the time for a short run-time would be quite
decent, actually, even if that sounds low in a single run because the
odds of detecting one failure increase a "lot" after a few repeated
runs.
On my own laptop, the original test of the reporter takes up to 3~4s
to reproduce the issue, for example, all the time, so that's good :)
--
Michael