Re: update on TOAST status'

Tom Lane wrote:
> JanWieck@t-online.de (Jan Wieck) writes:
> >     ... So do we want to  have  indices  storing  plain
> >     values allways and limit them in the index-tuple size or not?
>
> I think not: it will be seen as a robustness failure, even (or
> especially) if it doesn't happen often.  I can see the bug reports now:
> "Hey!  I tried to insert a long value in my field, and it didn't work!
> I thought you'd fixed this bug?"
>
> You make good arguments that we shouldn't be too concerned about the
> speed of access to toasted index values, and I'm willing to accept
> that point of view (at least till we have hard evidence about it).
> But when I say "it should be bulletproof" I mean it should *work*,
> without imposing arbitrary limits on the user.  Arbitrary limits are
> exactly what we are trying to eliminate.
   After debugging something I thought was a bug in the toaster,   I've found something really causing headaches.
   TOAST AS IS IS NOT CAPABLE OF HOLDING INDEXED VALUES!
   It appears that brtee indices (at least) can keep  references   to  old  toast  values  that  survive  a  VACUUM!
Seemsthese   references live in nodes actually not  referring  to  a  heap   tuple   any   more,   but   used  during
tree traversal  in   comparisions.  As if an index tuple delete from a  btree  not   necessarily  causes  the  index
value to disappear from the   btree completely.  It'll never be returned by an index  scan,   but the value is still
theresomewhere.
 
   Everything  is  OK  with this up to a VACUUM run. The toaster   uses SnapShotAny to fetch toast values. So an
external value   can  be  fetched by the toaster even if it is already deleted   and committed. If  he  has  a
reference somewhere,  he  has   allways   a  share  or  higher  lock  on  the  main  relation   preventing VACUUM to
mangleup the toast  relation  (I  moved   toast relation vacuuming into the lock time of the main table   recently).
 
   But in the above case it is already vacuumed and not  present   any  more. Now the btree traversal needs to compare
againsta   value, long gone to the bit heaven, and that cannot work with   the toast architecture.
 
   Seems  the  designs of btree and toast are colliding. As soon   as "text" is toastable, this'll hurt - be warned.
   AFAICS, we need to detoast values for index  inserts  allways   and  have  another toaster inside the index access
methodsat   some day.  In the meantime we should decide  a  safe  maximum   index tuple size and emit an explanative
errormessage on the   attempt to insert oversized index entries instead of possibly   corrupting the index.
 
   Comment!


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #