Обсуждение: Replication solutions high volume db
Folks, I'm thinking of replication tools for high volume site. Postgres on dual Opteron box. 1/2 GB of data. Big overnight load of data. Some inserts during the data but mostly selects. I think I've narrowed it down to Slony and Mammoth. Any pro's or con's on one vs the other? There is the obvious price difference but thats managable if Mammoth substancially better. Some area where our environement might stretch a solution. Some records can be large. Limitied to one table but they can be 40K+ of chraracters in a column. High volume of inserts coming during nightly load (millions of rows). These loads are coming in commited blocks of 100. That cannot slow down the replication. Our call center agents needs access to them asap. Any locking issues on the master during the replication process? Anything else I should be concerned about or things I need to consider? Personal experiences you have had running either. tia David
> > I think I've narrowed it down to Slony and Mammoth. > Any pro's or con's on one vs the other? Slony is more complicated to setup. Slony requires triggers and cron. Slony is asynchronous Slony is Open Source If Slony breaks you can call Command Prompt It used to be that Slony was timed replication. I do not know if that is still the case. Mammoth is about as easy as it can be to setup. Mammoth does not require triggers or cron Mammoth is asynchronous Mammoth costs $$$ Mammoth is set to be optionally synchronous in June. Mammoth replicates on a per comitted transaction basis. Mammoth Replicator is PostgreSQL with integrated replication. If Mammoth breaks you can call Command Prompt ;) > > Some area where our environement might stretch a solution. > Some records can be large. Limitied to one table but they can be 40K+ > of chraracters in a column. As long as that is not the primary key, there wouldn't be a problem. The length of the primary key can not be greater than 8k with Mammoth. > > High volume of inserts coming during nightly load (millions of rows). > These loads are coming in commited blocks of 100. No problem. > > That cannot slow down the replication. > Our call center agents needs access to them asap. > > Any locking issues on the master during the replication process? Neither Mammoth or Slony require any exclusive locks to replicate the data. Sincerely, Joshua D. Drake > Anything else I should be concerned about or things I need to consider? > Personal experiences you have had running either. > > tia > David > > ---------------------------(end of broadcast)--------------------------- > TIP 7: don't forget to increase your free space map settings -- Command Prompt, Inc., Your PostgreSQL solutions company. 503-667-4564 Custom programming, 24x7 support, managed services, and hosting Open Source Authors: plPHP, pgManage, Co-Authors: plPerlNG Reliable replication, Mammoth Replicator - http://www.commandprompt.com/ ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
David B wrote: > Folks, > > I'm thinking of replication tools for high volume site. > Postgres on dual Opteron box. 1/2 GB of data. Did you really mean 1/2 GB? Seems like a mighty small database. Maybe 1/2 TB or if you were really wild and crazy 1/2 PB. Sorry but I couldn't resist. I've funny fingered too many times myself. Rod -- --- [This E-mail scanned for viruses by Declude Virus]
On Tue, 2005-03-29 at 17:34, David B wrote: > Folks, > > I'm thinking of replication tools for high volume site. > Postgres on dual Opteron box. 1/2 GB of data. > > Big overnight load of data. > Some inserts during the data but mostly selects. > > I think I've narrowed it down to Slony and Mammoth. > Any pro's or con's on one vs the other? > > There is the obvious price difference but thats managable if Mammoth > substancially better. > > Some area where our environement might stretch a solution. > Some records can be large. Limitied to one table but they can be 40K+ > of chraracters in a column. > > High volume of inserts coming during nightly load (millions of rows). > These loads are coming in commited blocks of 100. > > That cannot slow down the replication. > Our call center agents needs access to them asap. > > Any locking issues on the master during the replication process? > > > Anything else I should be concerned about or things I need to consider? > Personal experiences you have had running either. I don't see any obvious problems for either replication engine here. Since slony is free and pretty easy to setup, I'd make a test setup and see how it handles the load. Generally speaking I'd expect Mammoth to handle high loads a little better since it's a log shipper and not trigger based. So, if slony proves too slow for this situation, then I'd recommend asking the folks at Command Prompt for a demo of their system to see if it can.
On Tue, 2005-03-29 at 17:50, Joshua D. Drake wrote: > > > > I think I've narrowed it down to Slony and Mammoth. > > Any pro's or con's on one vs the other? > > Slony is more complicated to setup. > Slony requires triggers and cron. Triggers yes, cron no... > It used to be that Slony was timed replication. I do not know if that is > still the case. Not the case now. I didn't use it "back in the day" (you know, last year :)
On Tue, 2005-03-29 at 18:12 -0600, Scott Marlowe wrote: > On Tue, 2005-03-29 at 17:50, Joshua D. Drake wrote: > > > > > > I think I've narrowed it down to Slony and Mammoth. > > > Any pro's or con's on one vs the other? > > > > Slony is more complicated to setup. > > Slony requires triggers and cron. > > Triggers yes, cron no... Oh? Good to know. > > > It used to be that Slony was timed replication. I do not know if that is > > still the case. > > Not the case now. I didn't use it "back in the day" (you know, last > year :) I know.. that is so 2004 :) Sincerely, Joshua D. Drake > -- Command Prompt, Inc., Your PostgreSQL solutions company. 503-667-4564 Custom programming, 24x7 support, managed services, and hosting Open Source Authors: plPHP, pgManage, Co-Authors: plPerlNG Reliable replication, Mammoth Replicator - http://www.commandprompt.com/
> > Generally speaking I'd expect Mammoth to handle high loads a little > better since it's a log shipper and not trigger based. So, if slony > proves too slow for this situation, then I'd recommend asking the folks > at Command Prompt for a demo of their system to see if it can. http://www.commandprompt.com/download Sincerely, Joshua D. Drake > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster -- Command Prompt, Inc., Your PostgreSQL solutions company. 503-667-4564 Custom programming, 24x7 support, managed services, and hosting Open Source Authors: plPHP, pgManage, Co-Authors: plPerlNG Reliable replication, Mammoth Replicator - http://www.commandprompt.com/ ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
> > Generally speaking I'd expect Mammoth to handle high loads a little > better since it's a log shipper and not trigger based. So, if slony > proves too slow for this situation, then I'd recommend asking the folks > at Command Prompt for a demo of their system to see if it can. http://www.commandprompt.com/download Sincerely, Joshua D. Drake > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster -- Command Prompt, Inc., Your PostgreSQL solutions company. 503-667-4564 Custom programming, 24x7 support, managed services, and hosting Open Source Authors: plPHP, pgManage, Co-Authors: plPerlNG Reliable replication, Mammoth Replicator - http://www.commandprompt.com/ ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
> > I think I've narrowed it down to Slony and Mammoth. > Any pro's or con's on one vs the other? Slony is more complicated to setup. Slony requires triggers and cron. Slony is asynchronous Slony is Open Source If Slony breaks you can call Command Prompt It used to be that Slony was timed replication. I do not know if that is still the case. Mammoth is about as easy as it can be to setup. Mammoth does not require triggers or cron Mammoth is asynchronous Mammoth costs $$$ Mammoth is set to be optionally synchronous in June. Mammoth replicates on a per comitted transaction basis. Mammoth Replicator is PostgreSQL with integrated replication. If Mammoth breaks you can call Command Prompt ;) > > Some area where our environement might stretch a solution. > Some records can be large. Limitied to one table but they can be 40K+ > of chraracters in a column. As long as that is not the primary key, there wouldn't be a problem. The length of the primary key can not be greater than 8k with Mammoth. > > High volume of inserts coming during nightly load (millions of rows). > These loads are coming in commited blocks of 100. No problem. > > That cannot slow down the replication. > Our call center agents needs access to them asap. > > Any locking issues on the master during the replication process? Neither Mammoth or Slony require any exclusive locks to replicate the data. Sincerely, Joshua D. Drake > Anything else I should be concerned about or things I need to consider? > Personal experiences you have had running either. > > tia > David > > ---------------------------(end of broadcast)--------------------------- > TIP 7: don't forget to increase your free space map settings -- Command Prompt, Inc., Your PostgreSQL solutions company. 503-667-4564 Custom programming, 24x7 support, managed services, and hosting Open Source Authors: plPHP, pgManage, Co-Authors: plPerlNG Reliable replication, Mammoth Replicator - http://www.commandprompt.com/ ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq