On Fri, 2008-01-11 at 11:34 +0000, Richard Huxton wrote:
> Is the following basically the same as option #3 (multiple RelFileNodes)?
>
> 1. Make an on-disk "chunk" much smaller (e.g. 64MB). Each chunk is a
> contigous range of blocks.
> 2. Make a table-partition (implied or explicit constraints) map to
> multiple "chunks".
> That would reduce fragmentation (you'd have on average 32MB's worth of
> blocks wasted per partition) and allow for stretchy partitions at the
> cost of an extra layer of indirection.
>
> For the single-partition case you'd not need to split the file of
> course, so it would end up looking much like the current arrangement.
We need to think about the "data model" of the storage layer. Space
itself isn't the issue, its the assumptions that all of the other
subsystems currently make about what how a table is structured, indexed,
accessed and manipulated.
Currently: Table 1:M Segments
Option 1: Table 1:M Segments and *separately* Table 1:M Partitions, so
partitions are always have a maximum size. The size just changes the
impact, doesn't change the impact of holes, max sizes etc.
e.g. empty table with 10 partitions would be
a) 0 bytes in 1 file
b) 0 bytes in 1 file, plus 9GB in 9 files all full of empty blocks
e.g. table with 10 partitions each of 1.5GB would be
a) 15 GB in 15 files
b) hit max size limit of partition: ERROR
Option 2: Table 1:M Child Tables 1:M Segments
e.g. empty table with 10 partitions would be
0 bytes in each of 10 files
e.g. table with 10 partitions each of 1.5GB would be
15GB in 10 groups of 2 files
Option 3: Table 1:M Nodes 1:M Segments
e.g. empty table with 10 partitions would be
0 bytes in each of 10 files
e.g. table with 10 partitions each of 1.5GB would be
15GB in 10 groups of 2 files
So 1b) seems definitely out.
The implications of 2 and 3 are what I'm worried about, which is why the
shortcomings of 1a) seem acceptable currently.
-- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com