On Fri, Jun 22, 2018 at 6:09 PM, Robert Haas <robertmhaas@gmail.com> wrote:
> On Mon, May 28, 2018 at 4:25 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:
>> On Fri, May 25, 2018 at 6:33 AM, Asim Praveen <apraveen@pivotal.io> wrote:
>>> We are evaluating the use of shared buffers for temporary tables. The
>>> advantage being queries involving temporary tables can make use of parallel
>>> workers.
>> This is one way, but I think there are other choices as well. We can
>> identify and flush all the dirty (local) buffers for the relation
>> being accessed parallelly. Now, once the parallel operation is
>> started, we won't allow performing any write operation on them. It
>> could be expensive if we have a lot of dirty local buffers for a
>> particular relation. I think if we are worried about the cost of
>> writes, then we can try some different way to parallelize temporary
>> table scan. At the beginning of the scan, leader backend will
>> remember the dirty blocks present in local buffers, it can then share
>> the list with parallel workers which will skip scanning those blocks
>> and in the end leader ensures that all those blocks will be scanned by
>> the leader. This shouldn't incur a much additional cost as the
>> skipped blocks should be present in local buffers of backend.
>
> This sounds awkward and limiting. How about using DSA to allocate
> space for the backend's temporary buffers and a dshash for lookups?
>
That's a better idea.
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com