Обсуждение: Quad Xeon vs. Dual Itanium
Hi, all. I need to upgrade my dual Xeon PostgreSQL engine. Assuming similar memory and disk sub-systems, I am considering a Quad Xeon system vs. a Dual Itanium for PostgreSQL. I believe that the PostgreSQL code is written for 32 bit and not optimized for the 64 bit Itanium cpu. That makes me think that the Xeon system would be a better choice. Do any of you have thoughts on: 1. Straight performance capability 2. Price/Performance I would appreciate any feedback you might have. ...john
John Gibson <gib@edgate.com> writes: > Assuming similar memory and disk sub-systems, I am considering a Quad > Xeon system vs. a Dual Itanium for PostgreSQL. I believe that the > PostgreSQL code is written for 32 bit and not optimized for the 64 bit > Itanium cpu. That makes me think that the Xeon system would be a > better choice. Postgres runs on many 64-bit systems, including UltraSPARC, MIPS, and Alpha, plus the Intel and AMD offerings. What makes you think it's 'not optimized'? -Doug
Hello list, I'm designing a plperlu function and i was wondering about scoping on use statements for external libraries. I couldn't find any information on it in the documentation or in the mail archives, so any information would be much appreciated. The function is intended to be used as part of a select statement: select foo,my_function(bar) as baz from table where baz > some_value order by baz And the function uses an external module to do much of the heavy lifting. What I'm wondering is will the function have to reload the external module for every row, or is plperlu smart enough to only load it once for the entire query? In the other extreme, I'm hoping that it does reload the external module for each query, as I expect to be dynamically rewriting one of the modules that that external module requires. -Stephen
-----Original Message----- From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of Doug McNaught Sent: Monday, February 09, 2004 10:44 AM To: John Gibson Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] Quad Xeon vs. Dual Itanium John Gibson <gib@edgate.com> writes: > Assuming similar memory and disk sub-systems, I am considering a Quad > Xeon system vs. a Dual Itanium for PostgreSQL. I believe that the > PostgreSQL code is written for 32 bit and not optimized for the 64 bit > Itanium cpu. That makes me think that the Xeon system would be a > better choice. Postgres runs on many 64-bit systems, including UltraSPARC, MIPS, and Alpha, plus the Intel and AMD offerings. What makes you think it's 'not optimized'? -Doug ------------------------- The only way I can see that its not optimized for 64 bit would be to use 32bit binaries on it, and the only way that can even happen is on the new amd chips I believe, or will itanium run 32bit apps also? Rob
John Gibson wrote: > > Assuming similar memory and disk sub-systems, I am considering a Quad > Xeon system vs. a Dual Itanium for PostgreSQL. I believe that the > PostgreSQL code is written for 32 bit and not optimized for the 64 bit > Itanium cpu. That makes me think that the Xeon system would be a better > choice. > The Itanic hasn't lived up to its marketing hype. The comparisons I've seen between it and a 32-bit CPU show performance differences primarily due to clock speeds. So far the only advantage of 64 bits is address space. And because they are new, itanics cost much more. So with 2 itanics you get a slight improvement. With 4 xeons you get about 1.7x improvement over your current setup. -- jimoe at sohnen-moe dot com
John Gibson wrote: > Hi, all. > > I need to upgrade my dual Xeon PostgreSQL engine. > > Assuming similar memory and disk sub-systems, I am considering a Quad > Xeon system vs. a Dual Itanium for PostgreSQL. I believe that the > PostgreSQL code is written for 32 bit and not optimized for the 64 bit > Itanium cpu. That makes me think that the Xeon system would be a better > choice. > ' Bang for the buck per CPU you are best off using an Opteron based solution from AMD. This will give you the 64bit address space that Xeon can't but for a heck of a lot less money than an Itanium. Sincerely, Joshua D. Drake > Do any of you have thoughts on: > > 1. Straight performance capability > 2. Price/Performance > > > I would appreciate any feedback you might have. > > > ...john > > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
Вложения
At 11:44 AM 2/9/2004 -0500, Doug McNaught wrote: >John Gibson <gib@edgate.com> writes: > > > Assuming similar memory and disk sub-systems, I am considering a Quad > > Xeon system vs. a Dual Itanium for PostgreSQL. I believe that the > > PostgreSQL code is written for 32 bit and not optimized for the 64 bit > > Itanium cpu. That makes me think that the Xeon system would be a > > better choice. > >Postgres runs on many 64-bit systems, including UltraSPARC, MIPS, and >Alpha, plus the Intel and AMD offerings. What makes you think it's >'not optimized'? Maybe compilers aren't as good at doing Itanium yet? John Gibson <gib@edgate.com> writes: "I need to upgrade my dual Xeon PostgreSQL engine." It just might be helpful if you could tell us "where it hurts". Unless you need cutting edge floating point performance I doubt you'd want an Itanium (and even if you do, you might wish to consider powerpc as well). Without any more info, I'd ask why not dual/quad Opteron? Even if you don't recompile or wait for better compilers or use 64 bit such a system would probably run faster than your dual Xeons. --- http://www.infoworld.com/article/04/01/30/05FElinux_2.html "Tests were run on three separate hardware platforms: Intel Xeon (x86), Intel Itanium (IA-64), and AMD Opteron (x86_64). The x86 tests were conducted on an IBM eServer x335 1U rack-mount server with dual 3.06GHz P4 Xeon processors and 2GB of RAM. The Itanium tests were run on an IBM eServer x450 3U rack-mount server with dual 1.5GHz Itanium2 processors and 2GB of RAM. And the Opteron tests were run on a Newisys 4300 3U rack-mount server with dual 2.2GHz Opteron 848 processors and 2GB of RAM. " Summary: Dual Itanium slower than Xeon in many tests, Opteron fastest in most tests.
On Mon, 9 Feb 2004, John Gibson wrote: > Hi, all. > > I need to upgrade my dual Xeon PostgreSQL engine. > > Assuming similar memory and disk sub-systems, I am considering a Quad > Xeon system vs. a Dual Itanium for PostgreSQL. I believe that the > PostgreSQL code is written for 32 bit and not optimized for the 64 bit > Itanium cpu. That makes me think that the Xeon system would be a better > choice. > > Do any of you have thoughts on: > > 1. Straight performance capability > 2. Price/Performance This really depends on what you'll be doing. If your data set is very large, then having a fast drive subsystem and lots of REALLY fast memory matters more than CPU most of the time. If you'll be doing lots of CPU intensive stuff, then having four CPUs is likely to be nice. It really kinda depends on what you're doing, and how fast the memory busses / caches are in the two machines. The CPUs in a well tuned database server tend to sit near idle mostly, while the drives spin and the data in memory gets pumped around. If the Itaniums have twice the memory bandwidth of the Xeons, it might well be a wash for most loads. So, what kinda profile are we looking at for your server?
Doug McNaught wrote: >John Gibson <gib@edgate.com> writes: > > > >>Assuming similar memory and disk sub-systems, I am considering a Quad >>Xeon system vs. a Dual Itanium for PostgreSQL. I believe that the >>PostgreSQL code is written for 32 bit and not optimized for the 64 bit >>Itanium cpu. That makes me think that the Xeon system would be a >>better choice. >> >> > >Postgres runs on many 64-bit systems, including UltraSPARC, MIPS, and >Alpha, plus the Intel and AMD offerings. What makes you think it's >'not optimized'? > >-Doug > > > Please help educate me. That is why I am asking. :) ...john
James Moe wrote:
> John Gibson wrote:
> >
> > Assuming similar memory and disk sub-systems, I am considering a Quad
> > Xeon system vs. a Dual Itanium for PostgreSQL. I believe that the
> > PostgreSQL code is written for 32 bit and not optimized for the 64 bit
> > Itanium cpu. That makes me think that the Xeon system would be a better
> > choice.
> >
> The Itanic hasn't lived up to its marketing hype. The comparisons
> I've seen between it and a 32-bit CPU show performance differences
> primarily due to clock speeds. So far the only advantage of 64 bits is
> address space. And because they are new, itanics cost much more.
> So with 2 itanics you get a slight improvement. With 4 xeons you get
> about 1.7x improvement over your current setup.
Here is an interesting article about the Opteron/Itanium issue:
http://www.theinquirer.net/?article=14038
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073
On Feb 10, 2004, at 2:18 AM, Lincoln Yeoh wrote: > At 11:44 AM 2/9/2004 -0500, Doug McNaught wrote: > >> John Gibson <gib@edgate.com> writes: >> >> > Assuming similar memory and disk sub-systems, I am considering a >> Quad >> > Xeon system vs. a Dual Itanium for PostgreSQL. I believe that the >> > PostgreSQL code is written for 32 bit and not optimized for the 64 >> bit >> > Itanium cpu. That makes me think that the Xeon system would be a >> > better choice. >> >> Postgres runs on many 64-bit systems, including UltraSPARC, MIPS, and >> Alpha, plus the Intel and AMD offerings. What makes you think it's >> 'not optimized'? <snip /> > Unless you need cutting edge floating point performance I doubt you'd > want an Itanium (and even if you do, you might wish to consider > powerpc as well). Speaking of PowerPC, has anyone out there run PostgreSQL on a G5 (either PowerMac or Xserve)? From looking at the specs, it seems it's got great throughput in terms of moving data around. Michael Glaesemann grzm myrealbox com
>>>>> "JG" == John Gibson <gib@edgate.com> writes: JG> Hi, all. JG> I need to upgrade my dual Xeon PostgreSQL engine. JG> Assuming similar memory and disk sub-systems, I am considering a Quad JG> Xeon system vs. a Dual Itanium for PostgreSQL. I believe that the Save the money from the dual itanium or quad xeon and buy a dual xeon but faster disks with a larger cache in the RAID controller, and perhaps a dedicated RAID controller for the disk set that holds the write-ahead log. You're sure to find that you saturate the disk system faster than you fill up even one CPU, let alone 4. I assume that the box will be running *only* the database and any other applications will run elsewhere. -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Vivek Khera, Ph.D. Khera Communications, Inc. Internet: khera@kciLink.com Rockville, MD +1-301-869-4449 x806 AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/
Clinging to sanity, gib@edgate.com (John Gibson) mumbled into her beard: > I need to upgrade my dual Xeon PostgreSQL engine. > > Assuming similar memory and disk sub-systems, I am considering a > Quad Xeon system vs. a Dual Itanium for PostgreSQL. I believe that > the PostgreSQL code is written for 32 bit and not optimized for the > 64 bit Itanium cpu. That makes me think that the Xeon system would > be a better choice. > > Do any of you have thoughts on: > > 1. Straight performance capability > 2. Price/Performance First thing: What, in particular, makes you think that PostgreSQL code was written to make it slower or otherwise more restrictive on 64 bit systems than it needs to be? Lots of people have been running it on 64 bit systems for _years_ now. The Digital Alpha architecture, for instance, was introduced in the 1992, and Sun UltraSPARC in 1995. PostgreSQL has been running well on these sorts of systems for a lot of years now. Your belief of it being "written for 32 bit" should fly away in the wake of that. Secondly, there's a significant counterargument to this, on Intel, when you look at memory availability. I have been tearing hair out with some FreeBSD testing in that I have some quad Xeon systems with 8GB of memory, which gives me the dilemma of choosing between: a) Ignoring 4GB of it, or b) Not having disk connected. The problem is that having large amounts of memory requires invoking an Intel "hack" (on FreeBSD, the option is called "PAE"), which happens to break the disk subsystem. (At least for the controller I have got.) And irrespective of any "successful hacks," you are still limited to either 2GB or 4GB of memory for the postmaster. If you jump into a 64 bit platform, those sorts of hacks evaporate as unnecessary, and the main process can get as big as you need it to. -- (format nil "~S@~S" "cbbrowne" "acm.org") http://www.ntlug.org/~cbbrowne/rdbms.html ``God decided to take the devil to court and settle their differences once and for all. When Satan heard of this, he grinned and said, "And just where do you think you're going to find a lawyer?"''
On Mon, Feb 09, 2004 at 12:46:58PM -0500, Christopher Browne wrote:
> Lots of people have been running it on 64 bit systems for _years_ now.
> The Digital Alpha architecture, for instance, was introduced in the
> 1992, and Sun UltraSPARC in 1995. PostgreSQL has been running well on
> these sorts of systems for a lot of years now.
But actually, there are problems with using postgres as a 64 bit
application on Solaris. It works, and it's reliable, but I've never
seen any evidence that it helps anything (and I've looked plenty).
A
--
Andrew Sullivan | ajs@crankycanuck.ca
In the future this spectacle of the middle classes shocking the avant-
garde will probably become the textbook definition of Postmodernism.
--Brad Holland
Hmm, do you mean 64 bit postgresql on Solaris-Sparc isn't significantly better performance-wise than 32 bit postgresql on Solaris-Sparc? Interesting. How about very large databases? At 07:17 AM 2/13/2004 -0500, Andrew Sullivan wrote: >On Mon, Feb 09, 2004 at 12:46:58PM -0500, Christopher Browne wrote: > > Lots of people have been running it on 64 bit systems for _years_ now. > > The Digital Alpha architecture, for instance, was introduced in the > > 1992, and Sun UltraSPARC in 1995. PostgreSQL has been running well on > > these sorts of systems for a lot of years now. > >But actually, there are problems with using postgres as a 64 bit >application on Solaris. It works, and it's reliable, but I've never >seen any evidence that it helps anything (and I've looked plenty). > >A > >-- >Andrew Sullivan | ajs@crankycanuck.ca >In the future this spectacle of the middle classes shocking the avant- >garde will probably become the textbook definition of Postmodernism. > --Brad Holland > >---------------------------(end of broadcast)--------------------------- >TIP 9: the planner will ignore your desire to choose an index scan if your > joining column's datatypes do not match
On Fri, Feb 13, 2004 at 10:44:55PM +0800, Lincoln Yeoh wrote:
> Hmm, do you mean 64 bit postgresql on Solaris-Sparc isn't significantly
> better performance-wise than 32 bit postgresql on Solaris-Sparc?
Not in any test I've ever run. I haven't tried again lately, mind
you. And I never had the Sun compiler, which, for all I know, does a
better job with 64 bit binaries than gcc.
A
--
Andrew Sullivan | ajs@crankycanuck.ca
This work was visionary and imaginative, and goes to show that visionary
and imaginative work need not end up well.
--Dennis Ritchie
On Fri, 13 Feb 2004, Andrew Sullivan wrote: > On Mon, Feb 09, 2004 at 12:46:58PM -0500, Christopher Browne wrote: > > Lots of people have been running it on 64 bit systems for _years_ now. > > The Digital Alpha architecture, for instance, was introduced in the > > 1992, and Sun UltraSPARC in 1995. PostgreSQL has been running well on > > these sorts of systems for a lot of years now. > > But actually, there are problems with using postgres as a 64 bit > application on Solaris. It works, and it's reliable, but I've never > seen any evidence that it helps anything (and I've looked plenty). I wonder if this would hold true when running 64 bit linux on Sparc hardware... Could well be that the reason 64 bit is no faster than 32 bit is that Solaris is just not a very fast platform for Postgresql, so any improvements running 64 bit pgsql are lost in the Solaris/Postgresql mix.
scott.marlowe wrote: > On Fri, 13 Feb 2004, Andrew Sullivan wrote: > > > On Mon, Feb 09, 2004 at 12:46:58PM -0500, Christopher Browne wrote: > > > Lots of people have been running it on 64 bit systems for _years_ now. > > > The Digital Alpha architecture, for instance, was introduced in the > > > 1992, and Sun UltraSPARC in 1995. PostgreSQL has been running well on > > > these sorts of systems for a lot of years now. > > > > But actually, there are problems with using postgres as a 64 bit > > application on Solaris. It works, and it's reliable, but I've never > > seen any evidence that it helps anything (and I've looked plenty). > > I wonder if this would hold true when running 64 bit linux on Sparc > hardware... Could well be that the reason 64 bit is no faster than 32 bit > is that Solaris is just not a very fast platform for Postgresql, so any > improvements running 64 bit pgsql are lost in the Solaris/Postgresql mix. 64-bits isn't faster than 32, and can be slower because of the longer pointer length, decreasing cache performance. The major advantage to 64-bits is accessing more the 4gb of RAM. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
On Fri, Feb 13, 2004 at 11:24:05AM -0500, Andrew Sullivan wrote: > On Fri, Feb 13, 2004 at 10:44:55PM +0800, Lincoln Yeoh wrote: > > Hmm, do you mean 64 bit postgresql on Solaris-Sparc isn't significantly > > better performance-wise than 32 bit postgresql on Solaris-Sparc? > > Not in any test I've ever run. I haven't tried again lately, mind > you. And I never had the Sun compiler, which, for all I know, does a > better job with 64 bit binaries than gcc. As a general rule 64 bit binaries tend to run slower than 32 bit binaries on any given architecture. The 32 bit binaries still get to use 64 bit data values if they feel so inclined, so that's nothing special. But all the pointers in the 64 bit build are twice the size, so the memory footprint is larger, it makes less efficient use of cache, it takes longer to read data from main memory and so on. If you need direct access to a lot of memory, and your application is structured to use it, then a 64 bit model can be a big win, but usually not otherwise. Forte C is a lot better than gcc for Solaris/SPARC (unsurprisingly) but 32 bit builds still seem a little faster than 64 bit builds. There are some benchmarks I looked at recently that show that, but I don't seem to be able to find the article right now. Cheers, Steve
On Fri, Feb 13, 2004 at 12:19:39PM -0500, Bruce Momjian wrote:
> 64-bits isn't faster than 32, and can be slower because of the longer
> pointer length, decreasing cache performance. The major advantage to
> 64-bits is accessing more the 4gb of RAM.
I note, however, that all the Sun experts say you should get your
database applications optimised for 64 bits because you can ship
around more data at a time. I have no clue what they're basing it
on, though.
A
--
Andrew Sullivan | ajs@crankycanuck.ca
The plural of anecdote is not data.
--Roger Brinner
Wouldn't you only care about 64-bit Postgres if you wanted to make shared_buffers bigger than 4G? Various other posters have commented about the sweet-spot for shared_buffers being ~ 100-200M (or thereabouts). So it seems to me that there is nothing to be gained using a 64-bit binary with the current or previous Pg releases. However, with the new cache replacement system being used in 7.5devel, the situation *may* be different (wonder if anyone has tried this out yet?). regards Mark Andrew Sullivan wrote: >On Mon, Feb 09, 2004 at 12:46:58PM -0500, Christopher Browne wrote: > > >>Lots of people have been running it on 64 bit systems for _years_ now. >>The Digital Alpha architecture, for instance, was introduced in the >>1992, and Sun UltraSPARC in 1995. PostgreSQL has been running well on >>these sorts of systems for a lot of years now. >> >> > >But actually, there are problems with using postgres as a 64 bit >application on Solaris. It works, and it's reliable, but I've never >seen any evidence that it helps anything (and I've looked plenty). > >A > > >
> -----Original Message----- > From: Mark Kirkwood [mailto:markir@paradise.net.nz] > Sent: Friday, February 13, 2004 5:30 PM > To: Andrew Sullivan > Cc: pgsql-general@postgresql.org > Subject: Re: [GENERAL] Quad Xeon vs. Dual Itanium > > > Wouldn't you only care about 64-bit Postgres if you wanted to make > shared_buffers bigger than 4G? > > Various other posters have commented about the sweet-spot for > shared_buffers being ~ 100-200M (or thereabouts). > > So it seems to me that there is nothing to be gained using a 64-bit > binary with the current or previous Pg releases. However, > with the new > cache replacement system being used in 7.5devel, the > situation *may* be > different (wonder if anyone has tried this out yet?). Where 64 bits matters (in general -- not restricted to PG database systems): Size of the database is huge (e.g. every toll paid in New Jersey in the last 5 years) Available memory is huge (e.g. you buy a machine with 24 gigs of ram) Data bus bandwidth is huge (e.g. You buy an 8-way Opteron with 40 GB/sec bandwidth) The 32 bit machines cannot compete in these arenas.
Mark Kirkwood <markir@paradise.net.nz> writes:
> So it seems to me that there is nothing to be gained using a 64-bit
> binary with the current or previous Pg releases. However, with the new
> cache replacement system being used in 7.5devel, the situation *may* be
> different (wonder if anyone has tried this out yet?).
Quite honestly, I suspect we may be wasting our time hacking the
Postgres buffer replacement algorithm at all. There are a bunch of
reasons why the PG shared buffer arena should never be more than a
small fraction of physical RAM, and under those conditions the cache
replacement algorithm that will matter is the kernel's, not ours.
I stand ready to be proven wrong, of course ...
regards, tom lane
No disagreement from me about the 64-bit *hardware* and *os*... Now suppose you want to run a Pg database for such a situation.... may as well compile 32-bit. Why ? well you *dont* want to set shared_buffers to 20G... in fact 200M works better - why ? well your 64-bit os file cache is much more efficient at using your 24G or RAM than Pg's buffer cache logic is (at the moment anyway). regards Mark Dann Corbit wrote: > >Where 64 bits matters (in general -- not restricted to PG database >systems): > >Size of the database is huge (e.g. every toll paid in New Jersey in the >last 5 years) >Available memory is huge (e.g. you buy a machine with 24 gigs of ram) >Data bus bandwidth is huge (e.g. You buy an 8-way Opteron with 40 GB/sec >bandwidth) > >The 32 bit machines cannot compete in these arenas. > > > >
Should have mentioned : assuming you are on a platform where you *have* a choice about compilation word-length! (Solaris and ?????) Mark Kirkwood wrote: > > Now suppose you want to run a Pg database for such a situation.... may > as well compile 32-bit. > >
On Fri, Feb 13, 2004 at 06:11:08PM -0800, Dann Corbit wrote:
>
> Size of the database is huge (e.g. every toll paid in New Jersey in the
> last 5 years)
> Available memory is huge (e.g. you buy a machine with 24 gigs of ram)
> Data bus bandwidth is huge (e.g. You buy an 8-way Opteron with 40 GB/sec
> bandwidth)
>
> The 32 bit machines cannot compete in these arenas.
I'm not supporting immense databases with this, but I am using 8- and
10- way UltraSPARC II boxes with 16 G of ram. I've been unable to
show a difference. There might _be_ one, mind, I just haven't shown
it.
A
--
Andrew Sullivan | ajs@crankycanuck.ca
I remember when computers were frustrating because they *did* exactly what
you told them to. That actually seems sort of quaint now.
--J.D. Baldwin
On Fri, Feb 13, 2004 at 10:46:18PM -0500, Tom Lane wrote: > Quite honestly, I suspect we may be wasting our time hacking the > Postgres buffer replacement algorithm at all. There are a bunch of > reasons why the PG shared buffer arena should never be more than a > small fraction of physical RAM, and under those conditions the cache > replacement algorithm that will matter is the kernel's, not ours. Well, unless the Postgres cache is more efficient than the OS's, no?. You could then use the nocache filesystem option, and just let Postgres handle the whole thing. Of course, that's a pretty big unless, and not one that I'm volunteering to make go away! A -- Andrew Sullivan
> -----Original Message----- > From: Andrew Sullivan [mailto:ajs@crankycanuck.ca] > Sent: Friday, February 13, 2004 9:05 PM > To: pgsql-general@postgresql.org > Subject: Re: [GENERAL] Quad Xeon vs. Dual Itanium > > > On Fri, Feb 13, 2004 at 10:46:18PM -0500, Tom Lane wrote: > > > Quite honestly, I suspect we may be wasting our time hacking the > > Postgres buffer replacement algorithm at all. There are a bunch of > > reasons why the PG shared buffer arena should never be more than a > > small fraction of physical RAM, and under those conditions > the cache > > replacement algorithm that will matter is the kernel's, not ours. > > Well, unless the Postgres cache is more efficient than the OS's, no?. > You could then use the nocache filesystem option, and just > let Postgres handle the whole thing. Of course, that's a > pretty big unless, and not one that I'm volunteering to make go away! Most database systems I have tried scale very well with increased memory. For instance, Oracle, and SQL*Server will definitely benefit greatly by adding more memory. I suspect (therefore) that there must be some way to squeeze some benefit out of it.
At 09:30 PM 2/13/2004 -0800, Dann Corbit wrote: > > Well, unless the Postgres cache is more efficient than the OS's, no?. > > You could then use the nocache filesystem option, and just > > let Postgres handle the whole thing. Of course, that's a > > pretty big unless, and not one that I'm volunteering to make go away! > >Most database systems I have tried scale very well with increased >memory. >For instance, Oracle, and SQL*Server will definitely benefit greatly by >adding more memory. I suspect (therefore) that there must be some way >to squeeze some benefit out of it. Yeah, but if the O/S cache etc also scales well with increased memory it may not make enough difference to make it worth the effort. Might be similar to the raw disk/partition thing - sure it's faster initially, but there's probably better bang for the buck elsewhere, and what happens if you change storage hardware - arrays, etc? Unlike the Oracle etc, it doesn't seem as strategic for Postgresql to compete with the O/S makers, and try to replace various parts of the O/S. It makes sense for Oracle, coz they can charge more, plus if the O/S sucks, their stuff runs better than the other competing DBs on the same O/S. However in this day and age, I'd rather pick a better O/S - and when the O/S improves, your DB seamlessly gains. So you might as well make sure the DB is really good at working with the O/S. You probably don't want cases where the entire DB is in mem, and a single select has the system 50% idle waiting for dunno what.
Martha Stewart called it a Good Thing when DCorbit@connx.com ("Dann Corbit") wrote:
>> -----Original Message-----
>> From: Andrew Sullivan [mailto:ajs@crankycanuck.ca]
>> Sent: Friday, February 13, 2004 9:05 PM
>> To: pgsql-general@postgresql.org
>> Subject: Re: [GENERAL] Quad Xeon vs. Dual Itanium
>>
>> On Fri, Feb 13, 2004 at 10:46:18PM -0500, Tom Lane wrote:
>>
>> > Quite honestly, I suspect we may be wasting our time hacking the
>> > Postgres buffer replacement algorithm at all. There are a bunch
>> > of reasons why the PG shared buffer arena should never be more
>> > than a small fraction of physical RAM, and under those conditions
>> > the cache replacement algorithm that will matter is the kernel's,
>> > not ours.
>>
>> Well, unless the Postgres cache is more efficient than the OS's,
>> no?. You could then use the nocache filesystem option, and just let
>> Postgres handle the whole thing. Of course, that's a pretty big
>> unless, and not one that I'm volunteering to make go away!
>
> Most database systems I have tried scale very well with increased
> memory.
> For instance, Oracle, and SQL*Server will definitely benefit greatly by
> adding more memory. I suspect (therefore) that there must be some way
> to squeeze some benefit out of it.
You'll certainly "squeeze _some_ benefit" out of increased memory used
for cacheing.
The troublesome question is whether or not you win more by having:
a) More cache managed by PG, or
b) More cache managed by the OS.
There are certain "use cases" where we know that PG can do better.
For instance, if you're vacuuming a database, you _know_ that PG is
walking through all of the data in the entire database, and you _know_
that this just happens once. There is no 'locality of reference'; the
vacuum will look at every page once, and not return to it.
In that case, what would be ideal is for data read in from disk to be
treated as "flushable." We're reading that data once; there's no
reason to expect it to be re-read. Might as well read it in page by
page and throw out a page every time you read one, so that there's
only one page of "vacuuming work" consuming memory.
One of Jan's cache management changes involves using that very
strategy for the PostgreSQL buffer. Pages brought in by VACUUM get
thrown into the "least-recently-used" location even though they are
"most-recently-used" because you _know_ that the data isn't
particularly interesting to keep around, certainly less so than the
data already cached.
That change doesn't touch the OS buffering, and so we'll still find
that a VACUUM will tend to evict commonly-used data from cache.
That all adds up to there being some incentive to want more control
over OS cache. Wouldn't it be nice to be able to tell the OS: "This
page isn't very important; treat it as LRU"? Unfortunately, doing
that is troublesome, at best. There aren't good "hooks" to control
that. Certainly not portable ones.
The "big name" guys often prefer to store data on raw partitions,
without OS cacheing, which means they set up Really Enormous DBMS
buffers, and manage inclusion into/eviction from cache themselves.
We might conceivably convince Linus Torvalds to include something in
Linux, but that would worsen things, in a way, because it would
probably lead to a code fork between "PostgreSQL for Linux" and
"PostgreSQL for portable platforms." (Substitute something else for
Linux and the adverse fork merely changes names...)
--
select 'cbbrowne' || '@' || 'acm.org';
http://cbbrowne.com/info/internet.html
E.V.A., pod 5, launching...
Centuries ago, Nostradamus foresaw when DCorbit@connx.com ("Dann Corbit") would write:
> Available memory is huge (e.g. you buy a machine with 24 gigs of ram)
Actually, as soon as 2GB of memory starts to feel "restrictive," 64
bit addressing starts being at least nominally worthwhile.
The only way you get Intel 32 bit systems to recognize much more than
that is to get into a mode called "PAE," which (theoretically) offers
the ability to address as much as 64GB.
Unfortunately, it's quite a hack, having considerable likelihood of
slowing your system and possibly not working at all. The slowdown to
be expected is in management of DMA devices (like disk drives).
On the "not working at all" side of things, I can't see that there's
ANY FreeBSD RAID controller that is compatible with PAE, aside from a
Compaq one where the driver author has lately released a warning of
performance problems.
That nicely characterizes the issue that you have to be _real_ careful
about what hardware you buy when going past about 2GB of memory.
--
output = ("cbbrowne" "@" "acm.org")
http://cbbrowne.com/info/spiritual.html
"If your parents never had children, chances are you won't either."
-- Dick Cavett
Martha Stewart called it a Good Thing when ajs@crankycanuck.ca (Andrew Sullivan) wrote: > On Fri, Feb 13, 2004 at 12:19:39PM -0500, Bruce Momjian wrote: >> 64-bits isn't faster than 32, and can be slower because of the longer >> pointer length, decreasing cache performance. The major advantage to >> 64-bits is accessing more the 4gb of RAM. > > I note, however, that all the Sun experts say you should get your > database applications optimised for 64 bits because you can ship > around more data at a time. I have no clue what they're basing it > on, though. Well, presumably if you are copying values in and out of registers, it is preferable to use 64 bit registers, thereby shifting around 64 bits at a time. If that seems underwhelming, well, that might explain Solaris... -- If this was helpful, <http://svcs.affero.net/rm.php?r=cbbrowne> rate me http://www.ntlug.org/~cbbrowne/emacs.html "[In 'Doctor' mode], I spent a good ten minutes telling Emacs what I thought of it. (The response was, 'Perhaps you could try to be less abusive.')" -- Matt Welsh
John Gibson wrote: > Assuming similar memory and disk sub-systems, I am considering a Quad > Xeon system vs. a Dual Itanium for PostgreSQL. I believe that the > PostgreSQL code is written for 32 bit and not optimized for the 64 bit > Itanium cpu. That makes me think that the Xeon system would be a better > choice. Unfortunately, both have issues. With Itanium, you absolutely have to use the Intel compiler. GCC is beyond unoptimized for Itanium; you should expect 1/3rd the performance with Postgres of what the Intel compiler would do. On the otherhand, I've heard a few caveats about the Intel compiler that a lot of the tricks they use are 100% designed for the Spec benchmarks and could break real world code. Depends on what your stomach is for risk taking on how many of the optimization flags you are willing to turn on. Quad Xeon has it's own problems. The shared bus makes the 2P to 4P scaling a bit problematic. The more intensive you use the memory subsystem (which probably is the case with database uses), the bigger hit you will take. As an example, SpecIntRate's 2->4 scaling for the XeonMP is 75% but the SpecFPRate (SpecFP uses much larger datasets) scaling drops to a mere 31%. Which memory usage model your DB would fall under unfortunately can only be tested by you. Also throw in the need for PAE to go over 2GB (~1GB for caching under most OS's) and you could see some performance penalties there versus a 64-bit server. But looking at the straight SpecIntRate numbers though, a 4P Xeon MP will be faster than an 2P Itanium. There's enough of a performance gap where penalties for shared memory bus and PAE won't make enough of a difference. 4P XeonMP 2.8GHz 47.4 4P Xeon 2GHz 34.7 2P Itanium2 1.5GHz 25.4 As for pricing, you'll have to look that up yourself. :) Personally, I'm very fond of Opteron servers due to the combination of 64-bit + ondie memory controller + point-to-point inter-cpu connect. As a point of comparison, 2P-4P Opteron Spec scaling numbers are 87% ad 76%.
Tom Lane wrote:
> Mark Kirkwood <markir@paradise.net.nz> writes:
>> So it seems to me that there is nothing to be gained using a 64-bit
>> binary with the current or previous Pg releases. However, with the new
>> cache replacement system being used in 7.5devel, the situation *may* be
>> different (wonder if anyone has tried this out yet?).
>
> Quite honestly, I suspect we may be wasting our time hacking the
> Postgres buffer replacement algorithm at all. There are a bunch of
> reasons why the PG shared buffer arena should never be more than a
> small fraction of physical RAM, and under those conditions the cache
> replacement algorithm that will matter is the kernel's, not ours.
>
> I stand ready to be proven wrong, of course ...
Let's see,
I'm not actually claiming you're wrong. But it seems that the new ARC
code together with the background writer is at least as good as the OS
buffer cache. Looking at these tests:
http://developer.postgresql.org/~wieck/PGvsOSbuffers/
I see a 9.8% and 18.9% improvement on the 90th percentile of transaction
response time comparing 1000 to 50000 shared buffers. And a 17.8% and
21.8% improvement comparing 10000 to 50000 shared buffers. The old
school "sweet spot" of shared_buffers is the loser, interesting, eh?
I guess that the higher buffer hit rate pays off in this particular test
scenario, which does not look at average response times or overall
throughput but aims to satisfy 90% of all requests within 5 seconds.
Note that the server is in all 6 test runs slightly overloaded as it
cannot meet this requirement ever. However, the total number of
processed transactions is highest for 50000 buffers, but only marginal.
Jan
--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #