Re: [HACKERS] LONG

> 
>     I  thought  about  the  huge size variable text type a little
>     more.  And I think I could get the  following  implementation
>     to work reliable for our upcoming release.
> 
>     For   any  relation,  having  one  or  more  LONG  data  type
>     attributes,  another  relation  (named   pg_<something>)   is
>     created,  accessible  only to superusers (and internal access
>     routines).  All LONG data items are  stored  as  a  reference
>     into  that relation, split up automatically so the chunks fit
>     into the installation specific tuple limit size.   Items  are
>     added/updated/removed totally transparent.

Should we use large objects for this, and beef them up.  Seems that
would be a good way.  I have considered putting them in a hash
bucket/directory tree for faster access to lots of large objects.

There is a lot to say about storing long tuples outside the tables
because long tuples fill cache buffers and make short fields longer to
access.

> 
>     It would not be indexable (jesus no!) and using it in a WHERE
>     clause will be expensive. But who ever uses a WHERE on a  not
>     indexable  (possibly containing megabytes per item) data type
>     is a silly fool who should get what he wanted, poor  response
>     times.

Good restriction.

>     I'd  like  to  name it LONG, like Oracle's 2G max. data type.
>     Even if I intend to restrict the data size to some  megabytes
>     for  now.  All  the data must still be processable in memory,
>     and there might be multiple instances of one item  in  memory
>     at  the  same time.  So a real 2G datatype is impossible with
>     this kind of approach.  But  isn't  a  64MB  #define'd  limit
>     enough  for  now?  This  would  possibly still blow away many
>     installations due to limited memory and/or swap space. And we
>     can  adjust  that #define in 2001 (an address space odyssey),
>     when 64bit hardware and plenty of GB real memory is  the  low
>     end standard *1).
> 
>     I already thought that the 8K default BLKSIZE is a little out
>     of date for today's  hardware  standards.  Two  weeks  ago  I
>     bought a PC for my kids. It's a 433MHz Celeron, 64MB ram, 6GB
>     disk - costs about $500 (exactly DM 999,-- at  Media  Markt).
>     With  the  actual  on  disk  cache  <->  memory and cache <->
>     surface transfer rates, the 8K size seems a little archaic to
>     me.

We use 8K blocks because that is the base size for most file systems. 
When we fsync an 8k buffer, the assumption is that that buffer is
written in a single write to the disk.  Larger buffers would be spread
over the disk, making a single fsync() impossible to be atomic, I think.

Also, larger buffers take more cache space per buffer, makeing the
buffer cache more corse holding fewer buffers.

--  Bruce Momjian                        |  http://www.op.net/~candle maillist@candle.pha.pa.us            |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026