On Tuesday, January 15, 2013, Gavin Flower wrote:
On 16/01/13 11:14, Bruce Momjian wrote:
I mentioned last year that I wanted to start working on parallelism:
https://wiki.postgresql.org/wiki/Parallel_Query_Execution
Years ago I added thread-safety to libpq. Recently I added two parallel
execution paths to pg_upgrade. The first parallel path allows execution
of external binaries pg_dump and psql (to restore). The second parallel
path does copy/link by calling fork/thread-safe C functions. I was able
to do each in 2-3 days.
I believe it is time to start adding parallel execution to the backend.
We already have some parallelism in the backend:
effective_io_concurrency and helper processes. I think it is time we
start to consider additional options.
Parallelism isn't going to help all queries, in fact it might be just a
small subset, but it will be the larger queries. The pg_upgrade
parallelism only helps clusters with multiple databases or tablespaces,
but the improvements are significant.
I have summarized my ideas by updating our Parallel Query Execution wiki
page:
https://wiki.postgresql.org/wiki/Parallel_Query_Execution
Please consider updating the page yourself or posting your ideas to this
thread. Thanks.
Hmm...
How about being aware of multiple spindles - so if the requested data covers multiple spindles, then data could be extracted in parallel. This may, or may not, involve multiple I/O channels?
effective_io_concurrency does this for bitmap scans. I thought there was a patch in the commitfest to extend this to ordinary index scans, but now I can't find it. But it still doesn't give you CPU parallelism. The nice thing about CPU parallelism is that it usually brings some amount of IO parallelism for free, while the reverse less likely to be so.
Cheers,
Jeff