Attached is the latest version of my parallel restore work.
In this version pg_dump has learned how to tag each item with the
section it belongs to (data, pre-data, post-data). That removes the
necessity for hardcoding knowledge about the boundary of the data
section into the parallel restore routine. There is some legacy
knowledge built into the archiver routines that means parallel restore
can be supported on older archives.
Hardcoded avoidance of AccessExclusive lock contention is still there,
although it might be made redundant if Simon's proposal for reducing
certain lock strengths is accepted.
Dependencies are finally treated correctly, I believe (after many nasty
debugging sessions).
This version seems to have the same performance characteristics as
previously reported. It appears that the sweet spot is around the number
of available CPUs. I did several runs using three times the number I had
available (24 threads on 8 CPUs), and while there were no errors,
performance did fall off. Not at all surprising.
I have not done anything yet about alternative item selection
algorithms. There doesn't seem to have been much interest - perhaps the
current algoithm will suffice for now.
cheers
andrew