Обсуждение: Pre-Allocate tablespace on disk

Hello,

I was wondering if there is a way to pre-allocate tablespace on disk before adding data and indexes.
My understanding is:
PG writes data into files sequentially. If more space is needed, disk space is requested from the OS and if there is
space,the OS will give PG a file system block and PG will write data/indexes until that block is full and so on. 
Other database that I have worked with before and that I'm still working with, allow you to pre-allocate disk space so
youget huge chunks of contiguous space at one, which has major impacts on database performance. 

Can this be done with PG at all? If not, is this planned for any time in the future?

best regards,
chris
--
chris ruprecht
database grunt and bit pusher extraordinaíre 

Le 15/11/2010 20:42, Chris Ruprecht a écrit :
> Hello,
>
> I was wondering if there is a way to pre-allocate tablespace on disk before adding data and indexes.
> My understanding is:
> PG writes data into files sequentially. If more space is needed, disk space is requested from the OS and if there is
space,the OS will give PG a file system block and PG will write data/indexes until that block is full and so on. 
> Other database that I have worked with before and that I'm still working with, allow you to pre-allocate disk space
soyou get huge chunks of contiguous space at one, which has major impacts on database performance. 
>
> Can this be done with PG at all?

No. A tablespace is a directory on PostgreSQL. And you can't allocate
space for a directory.

> If not, is this planned for any time in the future?

AFAICT, nope.


--
Guillaume
 http://www.postgresql.fr
 http://dalibo.com

On 11/15/10 11:42 AM, Chris Ruprecht wrote:
> I was wondering if there is a way to pre-allocate tablespace on disk before
>  adding data and indexes.
> My understanding is:
> PG writes data into files sequentially. If more space is needed, disk space
> is requested from the OS and if there is space, the OS will give PG a file
>  system block and PG will write data/indexes until that block is full and so on.
> Other database that I have worked with before and that I'm still working with,
> allow you to pre-allocate disk space so you get huge chunks of contiguous
> space at one, which has major impacts on database performance.

This shouldn't really be a performance issue on modern Unix-type systems.  Most file systems have fairly smart
block-allocationalgorithms that tend to allocate large blocks of contiguous space to files even when lots of processes
areasking for little chunks in interleaved requests.  It doesn't just allocate a block to the next guy who happens to
ask. It's a lot smarter than that.  In most cases, you'll barely be able to tell the difference between a file that was
allocatedall at once and one that was allocated on an as-needed basis.  The only time you get bad fragmentation is when
yourdisk gets nearly full, which takes away the file system's ability to be clever. 

If you really want to preallocate a file, build a separate partition on your disk and only put one database (or even
justone table) on that file system. 

On Mon, Nov 15, 2010 at 10:28 PM, Octavio <octaviomaiden@yahoo.com> wrote:
>
> Hello
>
> I have migrate a application from a server with postgres 8.1 to another with 8.3
> in the new server there are some page that doesn't work because seems to want to compare different data type
>
> does anyone know if it is a version issue or there is something Im missing?

Starting in 8.3 many automatic conversions from one type to another
were removed because it wasn't always obvious if this was the right
thing to do.  So, you might need to either change some column types,
or cast them explicitly to the type you want to be used for
comparison.

* Chris Ruprecht:

> Other database that I have worked with before and that I'm still
> working with, allow you to pre-allocate disk space so you get huge
> chunks of contiguous space at one, which has major impacts on
> database performance.

PostgreSQL's write patterns do not trigger significant fragmentation
with most file systems, even when other database systems (Oracle
Berkeley DB comes to my mind) would create heavily fragmented files.

--
Florian Weimer                <fweimer@bfk.de>
BFK edv-consulting GmbH       http://www.bfk.de/
Kriegsstraße 100              tel: +49-721-96201-1
D-76133 Karlsruhe             fax: +49-721-96201-99

Not only that, but if your "disk" is really a bunch of disks behind a hardware RAID controller, or SAN, or iSCSI or
NFSserver - what is contiguous for the OS may not be contiguous on the physical drives (and vice versa). <br /> Which
iswhy performance can (but may not) improve if you disable linux kernel IO schedulers that attempt to optimize blocks
tobe sequential.<br /><br /><br /><br /> On 11/19/2010 5:01 AM, Florian Weimer wrote: <blockquote
cite="mid:82zkt55ux4.fsf@mid.bfk.de"type="cite"><pre wrap="">* Chris Ruprecht: 

</pre><blockquote type="cite"><pre wrap="">Other database that I have worked with before and that I'm still
working with, allow you to pre-allocate disk space so you get huge
chunks of contiguous space at one, which has major impacts on
database performance.
</pre></blockquote><pre wrap="">
PostgreSQL's write patterns do not trigger significant fragmentation
with most file systems, even when other database systems (Oracle
Berkeley DB comes to my mind) would create heavily fragmented files.

</pre></blockquote><br /><br /><div class="moz-signature">-- <br /><style type="text/css"> .sig, .sig a, .sig td {
font:12px Arial; color: DarkBlue; white-space: nowrap; vertical-align: top; text-align: left; } .sig img { border: 0; }
.siga { font: 11px Arial; } .sig .back { color: #ffffff; background-color: DarkBlue; text-decoration: none; } .sig
.right{ text-align: right; } .sig .border { border: 1px solid DarkBlue; margin:5px; border-collapse: collapse; } .sig
.bottom{ vertical-align:bottom } .sig p { clear:both; margin:2px; } </style><div class="sig"><table
class="border"><tbody><tr><tdclass="back"> <b>Steve Francis</b></td><td class="back" style="width: 30px;"><br
/></td><tdclass="back" style="text-align: right;"><b>LogicMonitor LLC</b></td></tr><tr><td><br /></td><td><br
/></td><tdclass="right"><br /></td></tr><tr><td class="bottom"><a class="moz-txt-link-abbreviated"
href="mailto:sfrancis@logicmonitor.com">sfrancis@logicmonitor.com</a><br/><a href="http://www.logicmonitor.com"
target="_blank">www.logicmonitor.com</a></td><td><br/></td><td class="bottom" style="text-align: right;"> Ph: 1 888 41
LOGICx500<br /> Ph: 1 805 698 0770</td></tr></tbody></table><p>  <br /></div></div> 

Chris Ruprecht <chris@ruprecht.org> writes:
> I was wondering if there is a way to pre-allocate tablespace on disk
> before adding data and indexes.

As others have already said, it's not clear there's a point in doing
that with PostgreSQL. Now, you can achieve it and see for yourself by
relying on OS tools, such as LVM of ZFS, or some other tricks.

For example, create a big enough file, loop mount it as a device, and
create your tablespace in there. Of course what to do exactly when the
file is full is left as an exercise to the reader. Bonus points for any
solution not involving downtime.

Baseline: it seems that PostgreSQL works differently enough from your
usual software than you may want to reconsider your habits :)

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr     PostgreSQL : Expertise, Formation et Support