Re: Raw device I/O for large objects

<p><font size="2">Index organized tables would do this and it would be a generic capability.<br /><br /> - Luke<br
/><br/> Msg is shrt cuz m on ma treo<br /><br />  -----Original Message-----<br /> From:   Georgi Chulkov [<a
href="mailto:godji@metapenguin.org">mailto:godji@metapenguin.org</a>]<br/> Sent:   Monday, September 17, 2007 11:50 PM
EasternStandard Time<br /> To:     Tom Lane<br /> Cc:     pgsql-hackers@postgresql.org<br /> Subject:        Re:
[HACKERS]Raw device I/O for large objects<br /><br /> Hi,<br /><br /> > We've heard this idea proposed before, and
it'sbeen shot down as a poor<br /> > use of development effort every time.  Check the archives for previous<br />
>threads, but the basic argument goes like this: when Oracle et al did<br /> > that twenty years ago, it was a
goodidea because (1) operating systems<br /> > tended to have sucky filesystems, (2) performance and reliability<br
/>> properties of same were not very consistent across platforms, and (3)<br /> > being large commercial software
vendorsthey could afford to throw lots<br /> > of warm bodies at anything that seemed like a bottleneck.  None of
those<br/> > arguments holds up well for us today however.  If you think you want to<br /> > reimplement a
filesystemyou need to have some pretty concrete reasons<br /> > why you can outsmart all the smart folks who have
workedon<br /> > your-favorite-OS's filesystems for lo these many years.  There's also<br /> > the fact that on
anyreasonably modern disk hardware, "raw I/O" is<br /> > anything but.<br /><br /> Thanks, I agree with all your
arguments.<br/><br /> Here's the reason why I'm looking at raw device storage for large objects only<br /> (as opposed
toall tables): with raw device I/O I can control, to an extent,<br /> spatial locality. So, if I have an application
thatwants to store N large<br /> objects (totaling several gigabytes) and read them back in some order that is<br />
well-knownin advance, I could store my large objects in that order on the<br /> raw device.* Sequentially reading them
backwould then be very efficient.<br /> With a file system underneath, I don't have that freedom. (Such a scenario<br
/>occurs with raster databases, for example.)<br /><br /> * assuming I have a way to communicate these requirements;
that'sa whole new<br /> problem<br /><br /> Please allow me to ask then:<br /> 1. In your opinion, would the above
scenarioindeed benefit from a raw-device<br /> interface for large objects?<br /> 2. How feasible it is to decouple
generaltable storage from large object<br /> storage?<br /><br /> Thank you for your time,<br /><br /> Georgi<br /><br
/>---------------------------(end of broadcast)---------------------------<br /> TIP 1: if posting/reading through
Usenet,please send an appropriate<br />        subscribe-nomail command to majordomo@postgresql.org so that your<br />
      message can get through to the mailing list cleanly<br /></font>