Re: Raw device I/O for large objects

Поиск
Список
Период
Сортировка
От Georgi Chulkov
Тема Re: Raw device I/O for large objects
Дата
Msg-id 200709180548.25277.godji@metapenguin.org
обсуждение исходный текст
Ответ на Re: Raw device I/O for large objects  (Tom Lane <tgl@sss.pgh.pa.us>)
Ответы Re: Raw device I/O for large objects  (Markus Schiltknecht <markus@bluegap.ch>)
Re: Raw device I/O for large objects  (Tom Lane <tgl@sss.pgh.pa.us>)
Список pgsql-hackers
Hi,

> We've heard this idea proposed before, and it's been shot down as a poor
> use of development effort every time.  Check the archives for previous
> threads, but the basic argument goes like this: when Oracle et al did
> that twenty years ago, it was a good idea because (1) operating systems
> tended to have sucky filesystems, (2) performance and reliability
> properties of same were not very consistent across platforms, and (3)
> being large commercial software vendors they could afford to throw lots
> of warm bodies at anything that seemed like a bottleneck.  None of those
> arguments holds up well for us today however.  If you think you want to
> reimplement a filesystem you need to have some pretty concrete reasons
> why you can outsmart all the smart folks who have worked on
> your-favorite-OS's filesystems for lo these many years.  There's also
> the fact that on any reasonably modern disk hardware, "raw I/O" is
> anything but.

Thanks, I agree with all your arguments.

Here's the reason why I'm looking at raw device storage for large objects only 
(as opposed to all tables): with raw device I/O I can control, to an extent, 
spatial locality. So, if I have an application that wants to store N large 
objects (totaling several gigabytes) and read them back in some order that is 
well-known in advance, I could store my large objects in that order on the 
raw device.* Sequentially reading them back would then be very efficient. 
With a file system underneath, I don't have that freedom. (Such a scenario 
occurs with raster databases, for example.)

* assuming I have a way to communicate these requirements; that's a whole new 
problem

Please allow me to ask then:
1. In your opinion, would the above scenario indeed benefit from a raw-device 
interface for large objects?
2. How feasible it is to decouple general table storage from large object 
storage?

Thank you for your time,

Georgi


В списке pgsql-hackers по дате отправления:

Предыдущее
От: Tom Lane
Дата:
Сообщение: Re: Open issues for HOT patch
Следующее
От: "Luke Lonergan"
Дата:
Сообщение: Re: Raw device I/O for large objects