Re: Bit by "commands ignored until end of transaction block" again

Glenn Maynard wrote:
> Postgres kills a transaction when an error happens.  This is a pain;
> it assumes that all SQL errors are unexpected and fatal to the
> transaction.
> 
> There's a very simple case where it's not: UNIQUE checks. 

Ah, it's usually "syntax errors" wrt interactive sessions.
> I'm
> generating a cache, with a simple flow:
>  - Search for the cache key; if it exists, return its value.
>  - If it didn't exist, create the data based on the key, insert it
> into the table, and return it.
> 
> This has an obvious race: another thread looks up the same key and
> creates it between the search and the insert.  Both threads will
> create the cached data, thread A will insert it into the table, and
> thread B will get an integrity error when it tries to insert it, since
> it duplicates the unique key.

Yep.

> Here, by far the simplest fix is simply to ignore the integrity error.
>  Both threads generated the same data; the failed insert is expected
> and harmless.  Postgres is turning this into a fatal error.

Well, all errors are considered fatal. But same difference.

> There's so much that could make this trivially easy:
> 
>  - SQLite has the handy ON CONFLICT IGNORE, but Postgres has nothing
> like that.  (ON CONFLICT REPLACE is great, too.)

True. Been proposed. Fiddly to implement for all use-cases if I remember 
correctly.

>  - Let me use SAVEPOINT outside of a transaction,

You are never outside a transaction. All queries are executed within a 
transaction.
> with the effect of
> starting a transaction with the savepoint and ending it when it's
> committed.  Then, I could use savepoints without needing to know
> whether I'm already in a transaction or not; one would simply be
> started and committed for me if necessary.  (That's by far my biggest
> issue with savepoints: they force me to either specify "a transaction
> must be open when this function is called", or need to be able to
> query whether one is running to decide whether to start a transaction
> or a savepoint.  My function's use of transactions should be invisible
> to the caller.)

I think this is the root of your problem - all queries are within a 
transaction so either:
1. You have a transaction that wraps a single statement. If you get an 
error then only that statement was affected.
2. You have an explicit BEGIN...COMMIT transaction which could use a 
savepoint.

Nothing to stop you setting savepoints in #1 (although they're not much 
use).

Typically, if you're in a plpgsql function you would just catch "unique" 
exception codes from your insert. Or, update, see if any rows were 
affected, if not try an insert and if that gives a duplicate go back and 
try the update. You might want the second approach if 99% of the time 
the cache is already populated.

>  - Let me disable this error.  I don't want it.  (We're grownups; we
> can decide for ourselves which errors are fatal.)

You could always try submitting a patch. However, that's really what 
savepoints do - let you decide whether an error can be worked around.

> The first two are cleaner, since ignoring the error means I might
> ignore some other integrity error from the same statement, but I can
> live with that.
> 
> Lacking anything better, I'll probably end up dropping out of the ORM
> and using some uglier SQL to work around this, but this is so trivial
> that it's silly to have to do that.  I can't do it within the ORM; it
> doesn't have the vocabulary.

The ORM can't control transactions, can't call functions or can't set 
savepoints?

--   Richard Huxton  Archonet Ltd