Re: Preliminary notes about hash index concurrency (long)

Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > Tom Lane wrote:
> >> I've been looking at fixing the problem reported a few days ago whereby
> >> a bucket split in a hash index messes up the state of concurrent scans
> >> of the index, possibly causing some tuples to be missed by the scans.
> 
> > Seems you are adding locking similar to what we already do in btree.
> 
> Not adding locking --- hash already has locking.  The problem is the
> locking is wrong (both inadequate and deadlock prone :-().

Yea, I meant "adding locking sophistication similar to btree".  :-)

> > I know we have two sets of hash codes -- the one used for hash indexes,
> > and another used for hash joins and now aggregates and subqueries.
> 
> There's only one set of hash computation routines now.  But you are
> right that the issues under discussion only affect hash indexes, not
> in-memory hash tables.

Oh, yes, you merged them, but the index code is somehow separate for
locking issues (per-backend hashes don't have to lock anything).

> > I know someone reported a problem with the hash indexes (data loss,
> > serious)--- was that a new 7.4 but or something that has existed for a
> > long time?
> 
> AFAICT the bug's been there since Berkeley days.

Oh.

> > When were you considering making these changes?
> 
> Now.  We have three choices: fix it, ship 7.4 with a known data-loss
> bug, or remove hash indexes.  Which do you like?

Since Berkeley, huh?  Seems like a big change.  I would think the
logical solution would be to fix it in 7.5, but that leaves us with
shipping a hash that is now known to be buggy.  While it was always
buggy, we didn't know for sure until now.  We could disable hash indexes
in 7.4, but then re-enable them in 7.5 with the fix.  That seems kind of
strange becuase the current hash indexes would be moved to btree, but
then they would be have to be moved manually back to hash in 7.5.  Is
there a way to convert them to btree, but still have them dump as HASH
in pg_dump, so when they are moved to 7.5, they move back to hashes? 
That might be just too wierd.  If all the code changes are only in the
hash indexes, and they are known to be buggy, maybe we should just give
it a shot for 7.4 knowing it probably can't get worse than it already
is (but it could).

I know I am just shooting out ideas, but I am sure one of them still
stick with the group.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073