On Wed, May 30, 2012 at 6:10 PM, Sergey Koposov <koposov@ast.cam.ac.uk> wrote:
> On Wed, 30 May 2012, Jeff Janes wrote:
>
>> But anyway, is idt_match a fairly static table? If so, I'd partition
>> that into 16 tables, and then have each one of your tasks join against
>> a different one of those tables. That should relieve the contention
>> on the index root block, and might have some other benefits as well.
>
>
> No, idt_match is getting filled by multi-threaded copy() and then joined
> with 4 other big tables like idt_phot. The result is then split into
> partitions.
That does make things more complicated. But you could you partition
it at that level and then do the joins partition-wise?
I don't have much experience at data partitioning (well, I do, but the
experience is with partitioning in Perl with terabytes of flat files,
not in PG :) ) but I think that once you have your partitioning keys
you want to apply them the same way up and down the data set.
Cheers,
Jeff