Обсуждение: Running postgresql as a VMware ESx client
Hi all, I was wondering whether anyone has had any experience running postgresql in a vm under ESx. VMware provides significant HA/DR oppurtunities and we would like to use it if we can. The DBase would be on a EMC SAN hosted LUN and the ESx servers would be dual Quad CPU HP DL-380/G5s. At this stage we would use iSCSI for SAN connectivity as our testing with MS-SQL has not indicated that FC is needed. We are getting a bit of push back from the external support agency who seem more than a little bit nervous about the environment. I would appreciate any comments. -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Glen and Rosanne Eustace GodZone Internet Services, a division of AGRE Enterprises Ltd. P.O. Box 8020, Palmerston North, New Zealand 4446. Ph: +64 6 357 8168, Fax +64 6 357 8165, Mob: +64 21 424 015 http://www.godzone.net.nz "A Ministry specialising in providing low-cost Internet Services to NZ Christian Churches, Ministries and Organisations."
On Sun, Nov 23, 2008 at 5:28 PM, Glen Eustace <geustace@godzone.net.nz> wrote: > Hi all, > > I was wondering whether anyone has had any experience running postgresql in > a vm under ESx. VMware provides significant HA/DR oppurtunities and we > would like to use it if we can. The DBase would be on a EMC SAN hosted LUN > and the ESx servers would be dual Quad CPU HP DL-380/G5s. At this stage we > would use iSCSI for SAN connectivity as our testing with MS-SQL has not > indicated that FC is needed. > > We are getting a bit of push back from the external support agency who seem > more than a little bit nervous about the environment. I would appreciate > any comments. Generally speaking, virtualization allows you to take a bunch of low powered servers and make them live in one big box saving money on electricity and management. Generally speaking, database sers are big powerful boxes with lots of hard disks and gigs upon gigs of ram to handle terabytes of data. Those two things seem at odds to me. When I saw you post that the ESx servers would be dual quad core machines I immediately thought how nice that would be to run pgsql on, without vmware between it and the hardware. On to the real issue. PostgreSQL, like any database, requires certain guarantees about the data it's writing to the disks actually being written when the OS says it was. With a SAN and vmware, you now have a few extra layers that could like to your database. In order to see if things are working properly, you need to do some semi-destructive testing. Meaning you need to start up a few hundred clients running read/write transactions, force a checkpoint, and pull the plug on various parts (or all) of your system and see if it can come back up. Do not trust UPSes, generators, power conditioners etc. They can and do ALL fail sometimes. I've seen it happen a few times myself, when systems that could not possibly lose power lost power. If your system can't guarantee data integrity in case of power loss then I would push back on it too. What, exactly, as you looking to gain by running pgsql under vmware on such hardware?
> Generally speaking, virtualization allows you to take a bunch of low > powered servers and make them live in one big box saving money on > electricity and management. Generally speaking, database sers are big > powerful boxes with lots of hard disks and gigs upon gigs of ram to > handle terabytes of data. Those two things seem at odds to me. If one is handling databases with Terabytes of data and 1000s of connections, I would agree. We will be looking at 100s of Megabytes max and possible several hundred connections. A much smaller workload. > When I saw you post that the ESx servers would be dual quad core > machines I immediately thought how nice that would be to run pgsql on, > without vmware between it and the hardware. To pilot this project we were going to run just the single VM on the server, if there are performance issues we can always rebuild and put the OS directly on the hardware rather than the ESx hypervisor. > What, exactly, as you looking to gain by running pgsql under vmware on > such hardware? Mobility, in the HA/DR sense. Being able to vmotion the server while it is running to rectify hardware issues or perform upgrades. To bring the server up at a DR site automagically using SRM. And yes, we have still to investigate the whole crash consistency stuff with SRM. This database is heavily biased toward reads rather than writes. -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Glen and Rosanne Eustace GodZone Internet Services, a division of AGRE Enterprises Ltd. P.O. Box 8020, Palmerston North, New Zealand 4446. Ph: +64 6 357 8168, Fax +64 6 357 8165, Mob: +64 21 424 015 http://www.godzone.net.nz "A Ministry specialising in providing low-cost Internet Services to NZ Christian Churches, Ministries and Organisations."
On Sun, Nov 23, 2008 at 6:47 PM, Glen Eustace <geustace@godzone.net.nz> wrote: > >> Generally speaking, virtualization allows you to take a bunch of low >> powered servers and make them live in one big box saving money on >> electricity and management. Generally speaking, database sers are big >> powerful boxes with lots of hard disks and gigs upon gigs of ram to >> handle terabytes of data. Those two things seem at odds to me. > > If one is handling databases with Terabytes of data and 1000s of > connections, I would agree. We will be looking at 100s of Megabytes max and > possible several hundred connections. A much smaller workload. Yeah, you're not really looking at heavy lifting here then. Should be fine. > >> What, exactly, as you looking to gain by running pgsql under vmware on >> such hardware? > > Mobility, in the HA/DR sense. Being able to vmotion the server while it is > running to rectify hardware issues or perform upgrades. To bring the server > up at a DR site automagically using SRM. And yes, we have still to > investigate the whole crash consistency stuff with SRM. This database is > heavily biased toward reads rather than writes. Given the light load you're looking at running, you'll likely be ok as long as there aren't any issues with crash recovery.
Glen Eustace wrote: > Hi all, > > I was wondering whether anyone has had any experience running postgresql > in a vm under ESx. VMware provides significant HA/DR oppurtunities and > we would like to use it if we can. The DBase would be on a EMC SAN > hosted LUN and the ESx servers would be dual Quad CPU HP DL-380/G5s. At > this stage we would use iSCSI for SAN connectivity as our testing with > MS-SQL has not indicated that FC is needed. > > We are getting a bit of push back from the external support agency who > seem more than a little bit nervous about the environment. I would > appreciate any comments. I've done that a number of times, never had any problems. As has been mentioned elsewhere in the thread, for low or medium load databases of course. Just be sure to mount the PostgreSQL filesystem(s) (both xlog and data, if they are separate) directly to the SAN, and *don't* use the vmfs files on the host disk. You *can* use vmfs files, but they need to be specially configured IIRC to disable write caching, and will not perform very well. But if you use a directly mounted SAN volume, that issue goes away. And obviously you need someone in your organization that knows vmware ESX well (not just the point-click-installed level) to make sure things are set up in a reliable way for databases. //Magnus
Glen Eustace escribió: > >> Generally speaking, virtualization allows you to take a bunch of low >> powered servers and make them live in one big box saving money on >> electricity and management. Generally speaking, database sers are big >> powerful boxes with lots of hard disks and gigs upon gigs of ram to >> handle terabytes of data. Those two things seem at odds to me. > > If one is handling databases with Terabytes of data and 1000s of > connections, I would agree. We will be looking at 100s of Megabytes max > and possible several hundred connections. A much smaller workload. You're not gonna get "several hundred connections" on a resource-starved machine. Consider using a pooler (pgbouncer, pgpool), and reducing the number of actual connections to the DB to a very low number of dozens. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support