On 5/31/20 11:04 PM, Peter Geoghegan wrote:
> We generally like to produce tests for SSI, ON CONFLICT DO UPDATE, and
> anything else involving concurrent behavior using something called isolation
> tester: https://github.com/postgres/postgres/tree/master/src/test/isolation We
> may end up writing an isolation test for the issue you reported as part of an
> eventual fix. You might find it helpful to review some of the existing tests.
Ah, wonderful! I don't exactly know how to plug Elle's history analysis into
this, but I think it... should be possible to write down some special cases
based on the histories I've seen.
> Could you test Postgres 9.5? It would be nice to determine if this is
> a new issue, or a regression.
I'll look into that tomorrow morning! :)
I, uh, backed off to running these tests at read committed (which, uh... should
be SI, right?) and found what appear to be scads of SI violations, including
read skew and even *internal* consistency anomalies. Read-only transactions
can... apparently... see changing values of a record? Here's a single
transaction which read key 21, got [1], then read key 21 again, and saw [1 2 3]:
[[:r 21 [1]] [:r 20 [1 2]] [:r 20 [1 2]] [:r 21 [1 2 3]]]
See
http://jepsen.io.s3.amazonaws.com/analyses/postgresql-12.3/20200531T223558.000-0400.zip
-- jepsen.log from 22:36:09,907 to 22:36:09,909:
2020-05-31 22:36:09,907{GMT} INFO [jepsen worker 6] jepsen.util: 6
:invoke :txn [[:r 21 nil] [:r 20 nil] [:r 20 nil] [:r 21 nil]]
...
2020-05-31 22:36:09,909{GMT} INFO [jepsen worker 6] jepsen.util: 6
:ok :txn [[:r 21 [1]] [:r 20 [1 2]] [:r 20 [1 2]] [:r 21 [1 2 3]]]
You can fire up wireshark and point it at the pcap file in n1/ to
double-check--try `tcp.stream eq 4`. The BEGIN statement for this transaction is
at 22:36:09.908115. There are a bunch more anomalies called out in analysis.edn,
if it's helpful.
This looks so weird that I assume I've *got* to be doing it wrong, but trawling
through the source code and pcap trace, I can't see where the mistake is. Maybe
I'll have fresher eyes in the morning. :)
Sincerely,
--Kyle