Re: Minmax indexes

On 9/26/13 2:46 PM, Robert Haas wrote:
> On Thu, Sep 26, 2013 at 2:58 PM, Jim Nasby <jim@nasby.net> wrote:
>> Why would we add additional code complexity when forks do the trick? That
>> seems like a step backwards, not forward.
>
> Well, they sorta do the trick, but see e.g. commit
> ece01aae479227d9836294b287d872c5a6146a11.  I doubt that's the only
> code that's poorly-optimized for multiple forks; IOW, every time
> someone adds a new fork, there's a system-wide cost to that, even if
> that fork is only used in a tiny percentage of the relations that
> exist in the system.

Yeah, we obviously kept things simpler when adding forks in order to get the feature out the door. There's improvements
thatneed to be made. But IMHO that's not reason to automatically avoid forks; we need to consider the cost of improving
themvs what we gain by using them.
 

Of course there's always some added cost so we shouldn't just blindly use them all over the place without considering
thefork cost either...
 

> It's tempting to think that we can use the fork mechanism every time
> we have multiple logical "streams" of blocks within a relation and
> don't want to figure out a way to multiplex them onto the same
> physical file.  However, the reality is that the fork mechanism isn't
> up to the job.  I certainly don't want to imply that we shouldn't have
> gone in that direction - both the fsm and the vm are huge steps
> forward, and we wouldn't have gotten them in 8.4 without that
> mechanism.  But they haven't been entirely without their own pain,
> too, and that pain is going to grow the more we push in the direction
> of relying on forks.

Agreed.

Honestly, I think we actually need more obfuscation between what happens on the filesystem and the rest of postgres...
we'restarting to look at areas where that would help. For example, the recent idea of being able to truncate individual
relationfiles and not being limited to only truncating the end of the relation. My concern in that case is that 1GB is
apretty arbitrary size that we happened to pick, so if we're going to go for more efficiency in storage we probably
shouldn'tjust blindly stick with 1G (though of course initial implementation might do that to reduce complexity, but we
betterstill consider where we're headed).
 

>> If the only complaint about forks is directory traversal why wouldn't we go
>> with the well established practice of using multiple directories instead of
>> glomming everything into one place?
>
> That's not the only complaint about forks - but I support what you're
> proposing there anyhow, because it will be helpful to users with lots
> of relations regardless of what we do or do not decide to do about
> forks.
>


-- 
Jim C. Nasby, Data Architect                       jim@nasby.net
512.569.9461 (cell)                         http://jim.nasby.net