On Wed, Jul 5, 2017 at 12:14 AM, Mark Dilger <hornschnorter@gmail.com> wrote:
> I can understand this, but wonder if I could use something like
>
> FOR I TOTALLY PROMISE TO USE ALL ROWS rec IN EXECUTE sql LOOP
> ...
> END LOOP;
Actually, what you'd need is:
FOR I TOTALLY PROMISE TO USE ALL ROWS AND IT IS OK TO BUFFER THEM ALL
IN MEMORY INSTEAD OF FETCHING THEM ONE AT A TIME FROM THE QUERY rec IN
EXECUTE sql LOOP
Similarly, RETURN QUERY could be made to work with parallelism if we
had RETURN QUERY AND IT IS OK TO BUFFER ALL THE ROWS IN MEMORY TWICE
INSTEAD OF ONCE.
I've thought a bit about trying to make parallel query support partial
execution, but it seems wicked hard. The problem is that you can't
let the main process do anything parallel-unsafe (e.g., currently,
write any data) while the there are workers in existence, or things
may blow up badly. You could think about fixing that problem by
having all of the workers exit cleanly when the query is suspended,
and then firing up new ones when the query is resumed. However, that
presents two further problems: (1) having the workers exit cleanly
when the query is suspended would cause wrong answers unless any
tuples that the worker has implicitly claimed e.g. by taking a page
from a parallel scan and returning only some of the tuples on it were
somehow accounted for and (2) starting and stopping workers over and
over would be bad for performance. The second problem could be solved
by having a persistent pool of workers that attach and detach instead
of firing up new ones all the time, but that has a host of problems
all of its own. The first one would be desirable change for a bunch
of reasons but is not easy for reasons that are a little longer than I
feel like explaining right now.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company