Обсуждение: Replication: Slony-I vs. Mammoth Replicator vs. ?

HI all, I'm trying to implement a highly-scalable, high-performance,
real-time database replication system to back-up my Postgres database
as data gets written.

So far, Mammoth Replicator is looking pretty good but it costs $1000+ .

Has anyone tried Slony-I and other replication systems?  Slony-I is
pretty new so I'm a little unsure if it's ready for a prime-time
commercial system yet.

So... wanted to put this out to the experts.  Has anyone got any
recommendations or had experiences with real-time database replication
solutions that don't rely on RAID?  The reason why I don't want to
rely on a hardware solution is because we are renting dedicated
servers and we don't have access to the boxes, only to software that
gets installed on the boxes.

Thanks,
Chris

Chris Cheston wrote:
> HI all, I'm trying to implement a highly-scalable, high-performance,
> real-time database replication system to back-up my Postgres database
> as data gets written.
>
> So far, Mammoth Replicator is looking pretty good but it costs $1000+ .

Yes but it includes 30 days of support and 12 months of upgrades/updates :)


> Has anyone tried Slony-I and other replication systems?  Slony-I is
> pretty new so I'm a little unsure if it's ready for a prime-time
> commercial system yet.

It really depends on your needs. They are both good systems. Slony-I is
a bit more of a beast to get up and running, and it is a batch
replication system that uses triggers. Once it is up and running it
works well though.

Mammoth Replicator is easy to setup and is integrated into PostgreSQL.
However replicator is 1000+ and doesn't support promoting of slaves
automatically (you can do it by hand) like Slony does. Replicator is
also live replication.

Sincerely,

Joshua D. Drake



>
> So... wanted to put this out to the experts.  Has anyone got any
> recommendations or had experiences with real-time database replication
> solutions that don't rely on RAID?  The reason why I don't want to
> rely on a hardware solution is because we are renting dedicated
> servers and we don't have access to the boxes, only to software that
> gets installed on the boxes.
>
> Thanks,
> Chris
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: the planner will ignore your desire to choose an index scan if your
>       joining column's datatypes do not match


--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL

On 8/13/2004 9:39 PM, Joshua D. Drake wrote:

> Chris Cheston wrote:
>> HI all, I'm trying to implement a highly-scalable, high-performance,
>> real-time database replication system to back-up my Postgres database
>> as data gets written.
>>
>> So far, Mammoth Replicator is looking pretty good but it costs $1000+ .
>
> Yes but it includes 30 days of support and 12 months of upgrades/updates :)

The lead developer of Slony (me) is working for a company that has it
deployed in production already and will move more mission critical
systems to it very shortly. Guess what will be my top work priority if
you spot a bug?

>
>
>> Has anyone tried Slony-I and other replication systems?  Slony-I is
>> pretty new so I'm a little unsure if it's ready for a prime-time
>> commercial system yet.
>
> It really depends on your needs. They are both good systems. Slony-I is
> a bit more of a beast to get up and running, and it is a batch
> replication system that uses triggers. Once it is up and running it
> works well though.
>
> Mammoth Replicator is easy to setup and is integrated into PostgreSQL.
> However replicator is 1000+ and doesn't support promoting of slaves
> automatically (you can do it by hand) like Slony does. Replicator is
> also live replication.

Once again, Joshua, would you please explain what you mean with "batch"
and "live" replication system? Slony does group multiple "master"
transactions into one replication transaction to improve performance
(fewer commits on the slaves). The interval of these groups is
configurable and for high volume DBs it is recommended to use about one
second, which means that all commits that fall into an interval of one
second are replicated in one transaction on the slave. On normal running
systems this results in a replication lag of 600 to 800 milliseconds in
average. On overloaded systems the asynchronous nature of course allows
the slaves to fall behind.

What is a usual average replication lag of Mammoth Replicator?

What happens to the other existing slaves when you promote by hand? In
Slony they accept the new master and continue replicating without the
need of rebuilding from scratch. Slony has mechanisms to ensure the new
master will be ahead or equal in the replication process at the time it
takes over and allows client application updates.

The Slony documentation is an issue at the moment and the administrative
tools around it are immature. The replication engine itself exceeds my
own expectations and performs very robust.


Jan

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #

>
> Once again, Joshua, would you please explain what you mean with
> "batch" and "live" replication system? Slony does group multiple
> "master" transactions into one replication transaction to improve
> performance (fewer commits on the slaves). The interval of these
> groups is configurable and for high volume DBs it is recommended to
> use about one second, which means that all commits that fall into an
> interval of one second are replicated in one transaction on the slave.
> On normal running systems this results in a replication lag of 600 to
> 800 milliseconds in average. On overloaded systems the asynchronous
> nature of course allows the slaves to fall behind.


Your description above is what I considered batch... you are taking a
"batch" of transactions and replicating them versus each transaction. I
am not saying it is bad in any way. I am just saying it is different
that replicator.

> What is a usual average replication lag of Mammoth Replicator?
>
Obviously it depends on the system, the network connectivity between the
systems etc... In our test systems it takes less than 100 ms to
replicate the data. Again it depends on the size of the transaction (the
data being moved).

> What happens to the other existing slaves when you promote by hand?

This is something that Slony has over replicator. Currently the new
master will force a full dump to the slaves. Of course this is already
on the road map, thanks to Slony :) and should be resolved by months end.

> The Slony documentation is an issue at the moment and the
> administrative tools around it are immature. The replication engine
> itself exceeds my own expectations and performs very robust.
>
I have never suggested otherwise. My only comment about maturity is that
their are actually many companies using replicator in production. We
have already dealt with the 1.0 blues as they say.

I hope you understand that I, in no way have ever suggested (purposely)
anything negative about Slony. Only that I believe they serve different
technical solutions.

Sincerely,

Joshua D. Drake



>
> Jan
>


--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
PostgreSQL Replicator -- production quality replication for PostgreSQL

On 8/14/2004 12:22 PM, Joshua D. Drake wrote:

> I hope you understand that I, in no way have ever suggested (purposely)
> anything negative about Slony. Only that I believe they serve different
> technical solutions.

You know I never took anything you said negative. I think People here
need to know that we two have communicated and collaborated outside of
the public mailing lists, that we still do it and that we do not
consider each other as opponents. There is a natural overlap in the
systems we offer and therefore there is some competition. Software
development is a sport. For some professionals this sport happens to be
the main source of income. But that shall not spoil the spirit here.


Jan

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #

Centuries ago, Nostradamus foresaw when jd@commandprompt.com ("Joshua D. Drake") would write:
> I hope you understand that I, in no way have ever suggested
> (purposely) anything negative about Slony. Only that I believe they
> serve different technical solutions.

Stipulating that I may have some bias ;-), I still don't find it at
all clear what the different situations are "shaped like" that lead to
Mammoth being forcibly preferable to Slony-I.

(Note that I have a pretty decent understanding about how ERS and
Slony work, so I'm not too frightened of technicalities...  I set up
instances of both on Thursday, so I'm pretty up to speed :-).)

Win32 support may be true at the moment, although I have to discount
that as we only just got the start of a beta release of native Win32
support for PostgreSQL proper.  For that very reason, I had to point
my youngest brother who needed "something better than Access" to
Firebird last Saturday; I played with my niece while he was doing the
install.  And there is little reason to think that Slony-I won't be
portable to Win32 given a little interest and effort, particularly
once work to make it play well with "pgxs" gets done.
--
(format nil "~S@~S" "cbbrowne" "ntlug.org")
http://www3.sympatico.ca/cbbrowne/multiplexor.html
"At  Microsoft, it doesn't  matter which  file you're  compiling, only
which flags you #define."  -- Colin Plumb

Christopher Browne wrote:

Centuries ago, Nostradamus foresaw when jd@commandprompt.com ("Joshua D. Drake") would write: 

I hope you understand that I, in no way have ever suggested
(purposely) anything negative about Slony. Only that I believe they
serve different technical solutions.   

Stipulating that I may have some bias ;-), I still don't find it at
all clear what the different situations are "shaped like" that lead to
Mammoth being forcibly preferable to Slony-I. 

I would choose replicator if:

1. You want ease of setup
2. You want your each transaction to be replicated at time of commit
3. Your database is already laden with triggers
4. You are pushing a very high transactional load*

* Caveat I have no idea how well Slony performs on a system that does say 200,000 transactions
an hours that are heavily geared toward updates. Replicator performs very well in this scenario.

5. Replicators administrative tools are more mature than Slony (for example you know exactly what state your slaves are in with Replicator).

I would choose Slony if:

1. The fact that it is Open Source matters to you
2. The auto promotion of slaves is important*

*This will be fixed in a couple of weeks with Replicator

To be fair, in the real world ---

It doesn't make a bit of difference which one you choose it really comes down to this:

Replicator is dumb simple to setup. Any halfway talented person can setup replicator
in 30 minutes with a single master / slave configuration.

Slony is Open Source and thus a little easier on the pocket book initially.

Command Prompt, will support either one -- so the Replicator is commercially supported
argument is a little weak here.

Sincerely,

Joshua D. Drake




Sincerely,

Joshua D. Drake

(Note that I have a pretty decent understanding about how ERS and
Slony work, so I'm not too frightened of technicalities...  I set up
instances of both on Thursday, so I'm pretty up to speed :-).)

Win32 support may be true at the moment, although I have to discount
that as we only just got the start of a beta release of native Win32
support for PostgreSQL proper.  For that very reason, I had to point
my youngest brother who needed "something better than Access" to
Firebird last Saturday; I played with my niece while he was doing the
install.  And there is little reason to think that Slony-I won't be
portable to Win32 given a little interest and effort, particularly
once work to make it play well with "pgxs" gets done. 

-- 
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
PostgreSQL Replicator -- production quality replication for PostgreSQL

One more point for your list:

Choose Slony if Replicator doesn't support your platform. :-)


--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D.                Khera Communications, Inc.
Internet: khera@kciLink.com       Rockville, MD  +1-301-869-4449 x806
AIM: vivekkhera Y!: vivek_khera   http://www.khera.org/~vivek/