On Wed, 2008-10-22 at 09:13 -0400, Tom Lane wrote:
> Simon Riggs <simon@2ndQuadrant.com> writes:
> > On Wed, 2008-10-22 at 08:31 -0400, Tom Lane wrote:
> >> No, that was merely one component of the problem. The APIs for
> >> tidbitmaps need revision too. You can't "stream" a bitmap yet.
>
> > Please explain further.
> > Which calls? Why do we need to stream them?
>
> Well, I guess we don't absolutely *have* to --- we could insist that a
> bitmap scan on a bitmap index proceed by first sucking the whole bitmap
> into memory, the same as is done for other index types. It seems pretty
> silly though, especially for the expected use-case of low cardinality;
> the bitmaps would get big.
OK, now I understand.
Realistically, introducing a new index type is non-trivial and getting
the index working at all will be quite some feat, considering the
requirements of HOT and Vacuum.
I agree we could do the stream bitmap as well, but I'd suggest we should
save that until the rest of the patch has been checked out. GIN wasn't
all written in one release, and I think it likely that it may be another
couple of releases before we do all the tuning on BMIs.
Please let's not set the bar too high for the first implementation,
especially before we have test results that indicate where our tuning
should be focused.
> The idea that was being kicked around was to make it possible for
> TIDBitmap to be an alias representing an indexscan in progress. So
> you'd pull an index page or so's worth of TIDs from the index, hand
> them back to nodeBitmapHeapscan.c to read those tuples, repeat till
> done. With judicious use of a few "method" function pointers,
> nodeBitmapHeapscan wouldn't need to know whether it was doing this or
> using an in-memory bitmap. ANDing and ORing of bitmaps could be made
> to work streamably too.
Yes, completely agree that would be the better way.
> As Greg said, the major complaint against the original patch was that
> they'd made it do this (interleave the heap and index access) by hack
> slash and burn instead of extending the existing APIs appropriately.
Yeh, I never liked that original approach.
-- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support