Re: pg_dump directory archive format / parallel pg_dump

On Wed, Feb 2, 2011 at 13:32, Joachim Wieland  wrote:
> Here is a rebased version with some minor changes as well.

I read the patch works as below. Am I understanding correctly? 1. Open all connections in a parent process. 2. Start transactions for each connection in the parent. 3. Spawn child processes with fork(). 4. Each child process uses one of the inherited connections.

I think we have 2 important technical issues here:* The consistency is not perfect. Each transaction is started  with small delays in step 1, but we cannot guarantee no other  transaction between them.* Can we inherit connections to child processes with fork() ?  Moreover, we also need to pass running transactions to children.  I wonder libpq is designed for such usage.

To solve both issues, we might want a way to control visibility
in a database server instead of client programs. Don't we need
server-side support like [1] before developing parallel dump?[1] http://wiki.postgresql.org/wiki/ClusterFeatures#Export_snapshots_to_other_sessions

> I haven't
> tested it on Windows now but will do so as soon as the Unix part has
> been reviewed.

It might be better to remove Windows-specific codes from the first try.
I doubt Windows message queue is the best API in such console-based
application. I hope we could use the same implementation for all
platforms for inter-process/thread communication.

-- 
Itagaki Takahiro