Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Tue, Sep 26, 2017 at 5:06 AM, Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> Based on the review comment from Robert, I'm planning to do the big
> change to the architecture of this patch so that a backend process
> work together with a dedicated background worker that is responsible
> for resolving the foreign transactions. For the usage of this feature,
> it will be almost the same as what this patch has been doing except
> for adding a new GUC paramter that controls the number of resovler
> process launch. That is, we can have multiple resolver process to keep
> latency down.

Multiple resolver processes is useful but gets a bit complicated.  For
example, if process 1 has a connection open to foreign server A and
process 2 does not, and a request arrives that needs to be handled on
foreign server A, what happens?  If process 1 is already busy doing
something else, probably we want process 2 to try to open a new
connection to foreign server A and handle the request.  But if process
1 and 2 are both idle, ideally we'd like 1 to get that request rather
than 2.  That seems a bit difficult to get working though.  Maybe we
should just ignore such considerations in the first version.

> * Resovler processes
> 1. Fetch PGPROC entry from the shmem queue and get its XID (say, XID-a).
> 2. Get the fdw_xact_state entry from shmem hash by XID-a.
> 3. Iterate fdw_xact entries using the index, and resolve the foreign
> transactions.
> 3-a. If even one foreign transaction failed to resolve, raise an error.
> 4. Change the waiting backend state to FDWXACT_COMPLETED and release it.

Comments:

- Note that any error we raise here won't reach the user; this is a
background process.  We don't want to get into a loop where we just
error out repeatedly forever -- at least not if there's any other
reasonable choice.

- I suggest that we ought to track the status for each XID separately
on each server rather than just track the XID status overall.  That
way, if transaction resolution fails on one server, we don't keep
trying to reconnect to the others.

- If we go to resolve a remote transaction and find that no such
remote transaction exists, what should we do?  I'm inclined to think
that we should regard that as if we had succeeded in resolving the
transaction.  Certainly, if we've retried the server repeatedly, it
might be that we previously succeeded in resolving the transaction but
then the network connection was broken before we got the success
message back from the remote server.  But even if that's not the
scenario, I think we should assume that the DBA or some other system
resolved it and therefore we don't need to do anything further.  If we
assume anything else, then we just go into an infinite error loop,
which isn't useful behavior.  We could log a message, though (for
example, LOG: unable to resolve foreign transaction ... because it
does not exist).

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers