I have been doing some testing with multi-segment relations.
(Not having sixty gigabytes laying around, like some folk I've been
talking to recently, I rebuilt with a very small RELSEG_SIZE for
testing...) I have discovered that performance goes to zero as the
number of segments gets large --- in particular, if the number of
segments exceeds the number of kernel file descriptors that fd.c
thinks it can use, you can take a coffee break while inserting a
few tuples :-(. The main problem is mdnblocks() in md.c, which
is called for each tuple inserted; it insists on touching every
segment of the relation to verify its length via lseek(). That's an
unreasonable number of kernel calls in any case, and if you add a file
open and close to each touch because of file-descriptor thrashing,
it's coffee break time.
It seems to me that mdnblocks should assume that all segments
previously verified to be of length RELSEG_SIZE are still of that
length, and only do an _mdnblocks() call on the last element of the
segment chain. That way the number of kernel interactions needed
is a constant independent of the number of segments in the table.
(This doesn't make truncation of the relation by a different backend
any more or less dangerous; we're still hosed if we don't get a
notification before we try to do something with the relation.
I think that is fixed, and if it's not this doesn't make it worse.)
Any objections?
regards, tom lane