Re: Potential G2-item cycles under serializable isolation

OK! So I've designed a variant of this test which doesn't use ON CONFLICT. 
Instead, we do a homebrew sort of upsert: we try to update the row in place by 
primary key; if we see zero records updated, we insert a new row, and if *that* 
fails due to the primary key conflict, we try the update again, under the theory 
that since we now know a copy of the row exists, we should be able to update it.


https://github.com/jepsen-io/jepsen/blob/f47eb25ab32529a7b66f1dfdd3b5ef2fc84ed778/stolon/src/jepsen/stolon/append.clj#L31-L108

This isn't bulletproof; even under SERIALIZABLE, Postgres will allow a 
transaction to fail to update any rows, then fail to insert due to a primary key 
conflict--but because the primary key conflict forces a transaction abort (or 
rolling back to a savepoint, which negates the conflict), I think it's still 
serializable. The isolation docs explain this, I think.

Unfortunately, I'm still seeing tons of G2-item cycles. Whatever this is, it's 
not related to ON CONFLICT.

As a side note, I can confirm that Postgres repeatable read is definitely weaker 
than repeatable read--at least as formalized by Adya et al. This makes sense if 
you understand repeatable read to mean SI (after all, the SI paper says they're 
incomparable!), but the Postgres docs seem to imply Postgres is strictly 
*stronger* than the ANSI SQL spec, and I'm not convinced that's entirely true:

https://www.postgresql.org/docs/12/transaction-iso.html

 > The table also shows that PostgreSQL's Repeatable Read implementation does 
not allow phantom reads. Stricter behavior is permitted by the SQL standard...

 > This is a stronger guarantee than is required by the SQL standard for this 
isolation level...

I get the sense that the Postgres docs have already diverged from the ANSI SQL 
standard a bit, since SQL 92 only defines three anomalies (P1, P2, P3), and 
Postgres defines a fourth: "serialization anomaly".

This results in a sort of weird situation: on the one hand, it's true: we don't 
(I think) observe P1 or P2 under Postgres Repeatable Read. On the other hand, 
SQL 92 says that the difference between repeatable read and serializable is 
*exactly* the prohibition of P3 ("phantom"). Even though all our operations are 
performed by primary key, we can observe a distinct difference between Postgres 
repeatable read and Postgres serializable.

I can see two ways to reconcile this--one being that Postgres chose the anomaly 
interpretation of the SQL spec, and the result is... maybe internally 
inconsistent? Or perhaps one of the operations in this workload actually *is* a 
predicate operation--maybe by dint of relying on a uniqueness constraint?

I'm surprised the transaction isolation docs don't say something like "Postgres 
repeatable read level means snapshot isolation, which is incomparable with SQL 
repeatable read." Then it'd be obvious that Postgres repeatable read exhibits 
G2-item! Weirdly, the term "snapshot isolation" doesn't show up on the page at all!

--Kyle