Обсуждение: postgresql-R
Hello, I was wondering if anybody has had much experience with postgresql-R, or being able to do database replication in a WAN environment. Basically, does anybody have any good links/docs on doing this with Solaris 9, postgresql v8.0? Thanks in advance.
Вложения
On Tue, Mar 07, 2006 at 03:29:37PM -0500, Louis Gonzales wrote: > Hello, > I was wondering if anybody has had much experience with postgresql-R, or > being able to do database replication in a WAN environment. > Basically, does anybody have any good links/docs on doing this with > Solaris 9, postgresql v8.0? If I'm not mistaken, pgreplication (which I'm guessing is what you mean by postgresql-R) is part of the basis for the upcomming Slony-II. Right now, depending on what you mean by 'WAN', your best bet is probably Slony-I. If you're not doing a large update volume and the connections to your remote sites are reliable it should work OK. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
Based on:<br /><p><b>Postgres based Replication Projects</b><ul><li><a href="http://gborg.postgresql.org/project/pgreplication/projdisplay.php"><b>PGReplication</b></a><li>Postgres-R: <a href="http://www.cs.mcgill.ca/%7Ekemme"><b>Dr.Kemme's Site</b></a>, <a href="http://www.cs.mcgill.ca/%7Ekemme/papers/vldb00.html"><b>Paper</b></a>,<a href="http://www.cs.mcgill.ca/%7Ekemme/publications.html"><b>Publications</b></a>,<a href="http://www.cs.mcgill.ca/%7Ekemme/disl/replication.html"><b>ReplicationWork</b></a>, <a href="http://www.cs.cornell.edu/Info/Projects/HORUS/"><b>TheHorus Project</b></a> and <a href="http://www.cs.cornell.edu/Info/Projects/Ensemble/"><b>Emsemble</b></a><li><a href="http://www.inf.ethz.ch/department/IS/iks/research/dragon.html"><b>DRAGON</b></a>:Database Replication based on GroupCommunication <li><a href="http://gborg.postgresql.org/project/slony1/projdisplay.php"><b>The Slony-1 Project</b></a><li><ahref="http://hiroshima.sraw.co.jp/people/mitani/jpug/pgcluster/en/"><b>PGCluster</b></a><li><a href="http://sourceforge.net/projects/dbbalancer/"><b>DBBalancer</b></a><li><a href="http://pgreplicator.sourceforge.net/"><b>PostgreSQLReplicator</b></a>: <a href="http://pgreplicator.sourceforge.net/tutorial.html"><b>Tutorial</b></a>,<a href="http://pgreplicator.sourceforge.net/5074-69225.pdf"><b>Paper</b></a>,<a href="http://techdocs.postgresql.org/installguides.php#replicator"><b>Techdocs</b></a><li><a href="http://gborg.postgresql.org/project/erserver/projdisplay.php"><b>eRServer:</b></a><a href="http://techdocs.postgresql.org/installguides.php#rserv"><b>Techdocs</b></a><li><a href="http://archives.postgresql.org/pgsql-patches/2002-02/msg00004.php"><b>DBMirror</b></a>:Will be in the 7.3 contrib directory.<li><a href="http://usogres.good-day.net/"><b>Usogres</b></a>: <a href="http://techdocs.postgresql.org/installguides.php#usogres"><b>Techdocs</b></a></ul>From: <a class="moz-txt-link-freetext" href="http://gborg.postgresql.org/project/pgreplication/genpage.php?replication_research">http://gborg.postgresql.org/project/pgreplication/genpage.php?replication_research</a><br />I was under the impression that "Postgres-R" - I misspelled it as "postgresql-R" - is a database clustering software. I downloaded the sources for this and it's supposed to work with "spread." I apologize if I'm mistaken?<br /><br/> I will certainly take a look into Slony-1, as you suggest. Did you have any good links to this? Have you gone throughan install on Solaris 9 w/Postgresl v8.0?<br /><br /> WAN != LAN, where my database server is not in the same cityas the target system to cluster with; for my worse case scenario, could be from east coast to west coast.<br /><br /> Kind regards,<br /><br /><br /> Jim C. Nasby wrote:<br /><blockquote cite="mid20060307203530.GG58405@pervasive.com"type="cite"><pre wrap="">On Tue, Mar 07, 2006 at 03:29:37PM -0500, Louis Gonzaleswrote: </pre><blockquote type="cite"><pre wrap="">Hello, I was wondering if anybody has had much experience with postgresql-R, or being able to do database replication in a WAN environment. Basically, does anybody have any good links/docs on doing this with Solaris 9, postgresql v8.0? </pre></blockquote><pre wrap=""> If I'm not mistaken, pgreplication (which I'm guessing is what you mean by postgresql-R) is part of the basis for the upcomming Slony-II. Right now, depending on what you mean by 'WAN', your best bet is probably Slony-I. If you're not doing a large update volume and the connections to your remote sites are reliable it should work OK. </pre></blockquote><br />
Louis Gonzales <louis.gonzales@linuxlouis.net> writes:
> I was wondering if anybody has had much experience with postgresql-R,
Postgres-R doesn't exist in any production-ready form. That software
is an academic project that's based on a very old, buggy release of
Postgres ... you *don't* want to use it.
regards, tom lane
Tom Lane wrote:<br /><blockquote cite="mid3834.1141765903@sss.pgh.pa.us" type="cite"><pre wrap="">Louis Gonzales <a
class="moz-txt-link-rfc2396E"href="mailto:louis.gonzales@linuxlouis.net"><louis.gonzales@linuxlouis.net></a>
writes:</pre><blockquote type="cite"><pre wrap="">I was wondering if anybody has had much experience with postgresql-R,
</pre></blockquote><pre wrap="">
Postgres-R doesn't exist in any production-ready form. That software
is an academic project that's based on a very old, buggy release of
Postgres ... you *don't* want to use it.
regards, tom lane </pre></blockquote> Tom,<br /> Thank you! I'll get rid of the source now. Jim Nasby,
actuallyreferred me to Slony... thanks Jim, that looks like what I want.<br />
On Mar 7, 2006, at 2:58 PM, Louis Gonzales wrote: > Based on: > Postgres based Replication Projects > > PG Replication > Postgres-R: Dr. Kemme's Site, Paper, Publications, Replication > Work, The Horus Project and Emsemble > DRAGON: Database Replication based on Group Communication > The Slony-1 Project > PGCluster > DBBalancer > PostgreSQL Replicator: Tutorial, Paper, Techdocs > eRServer: Techdocs > DBMirror: Will be in the 7.3 contrib directory. > Usogres: Techdocs > From: http://gborg.postgresql.org/project/pgreplication/ > genpage.php?replication_research > I was under the impression that "Postgres-R" - I misspelled it as > "postgresql-R" - is a database clustering software. I downloaded > the sources for this and it's supposed to work with "spread." I > apologize if I'm mistaken? Well, AfAIK it's quasi-clustering. I'd describe it as a mix between sync and async multi-master replication. Its concepts are the basis behind Slony-II AFAIK. > I will certainly take a look into Slony-1, as you suggest. Did you > have any good links to this? Have you gone through an install on > Solaris 9 w/Postgresl v8.0? > > WAN != LAN, where my database server is not in the same city as the > target system to cluster with; for my worse case scenario, could be > from east coast to west coast. The issues you'll run into are that in a high-load scenario, the latency on that link could become a real issue, as could bandwidth (actually, bandwidth is probably the bigger issue). The other downside is that slony is decidedly async; if you have to fail over you're probably going to lose some recently committed data. As long as that's not an issue then it's your best bet. If it is an issue, you'll probably be stuck rolling your own solution using something like 2PC. You should hop on the slony mailing list if you have any questions about it. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
Hey Jim,
Thanks again for the pointer to this. I've already compiled and
installed on one of the two Solaris nodes, that I needed to. Yeah upon
further reading, I can't wait for Slony-II to come out - is there truth
behind the "anyday" rumor? - is it also true that it's going to
implement the true multi-master scenario, where updates can be made at
any of the clustered nodes.
I'm going to deploy the "slon worker process" locally on every
participating node, rather than letting the master host the "slon"
processes for every cluster participant, for performance reasons.
I will also join the Slony mailing list, very cool stuff.
Kind Regards,
Jim Nasby wrote:
> On Mar 7, 2006, at 2:58 PM, Louis Gonzales wrote:
>
>> Based on:
>> Postgres based Replication Projects
>>
>> PG Replication
>> Postgres-R: Dr. Kemme's Site, Paper, Publications, Replication Work,
>> The Horus Project and Emsemble
>> DRAGON: Database Replication based on Group Communication
>> The Slony-1 Project
>> PGCluster
>> DBBalancer
>> PostgreSQL Replicator: Tutorial, Paper, Techdocs
>> eRServer: Techdocs
>> DBMirror: Will be in the 7.3 contrib directory.
>> Usogres: Techdocs
>> From: http://gborg.postgresql.org/project/pgreplication/
>> genpage.php?replication_research
>> I was under the impression that "Postgres-R" - I misspelled it as
>> "postgresql-R" - is a database clustering software. I downloaded
>> the sources for this and it's supposed to work with "spread." I
>> apologize if I'm mistaken?
>
>
> Well, AfAIK it's quasi-clustering. I'd describe it as a mix between
> sync and async multi-master replication. Its concepts are the basis
> behind Slony-II AFAIK.
>
>> I will certainly take a look into Slony-1, as you suggest. Did you
>> have any good links to this? Have you gone through an install on
>> Solaris 9 w/Postgresl v8.0?
>>
>> WAN != LAN, where my database server is not in the same city as the
>> target system to cluster with; for my worse case scenario, could be
>> from east coast to west coast.
>
>
> The issues you'll run into are that in a high-load scenario, the
> latency on that link could become a real issue, as could bandwidth
> (actually, bandwidth is probably the bigger issue). The other
> downside is that slony is decidedly async; if you have to fail over
> you're probably going to lose some recently committed data. As long
> as that's not an issue then it's your best bet. If it is an issue,
> you'll probably be stuck rolling your own solution using something
> like 2PC.
>
> You should hop on the slony mailing list if you have any questions
> about it.
> --
> Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
> Pervasive Software http://pervasive.com work: 512-231-6117
> vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
>
Jim Nasby wrote: > On Mar 7, 2006, at 2:58 PM, Louis Gonzales wrote: > >> Based on: >> Postgres based Replication Projects >> >> PG Replication >> Postgres-R: Dr. Kemme's Site, Paper, Publications, Replication Work, >> The Horus Project and Emsemble >> DRAGON: Database Replication based on Group Communication >> The Slony-1 Project >> PGCluster >> DBBalancer >> PostgreSQL Replicator: Tutorial, Paper, Techdocs >> eRServer: Techdocs >> DBMirror: Will be in the 7.3 contrib directory. >> Usogres: Techdocs >> From: http://gborg.postgresql.org/project/pgreplication/ >> genpage.php?replication_research >> I was under the impression that "Postgres-R" - I misspelled it as >> "postgresql-R" - is a database clustering software. I downloaded >> the sources for this and it's supposed to work with "spread." I >> apologize if I'm mistaken? > > > Well, AfAIK it's quasi-clustering. I'd describe it as a mix between > sync and async multi-master replication. Its concepts are the basis > behind Slony-II AFAIK. > >> I will certainly take a look into Slony-1, as you suggest. Did you >> have any good links to this? Have you gone through an install on >> Solaris 9 w/Postgresl v8.0? >> >> WAN != LAN, where my database server is not in the same city as the >> target system to cluster with; for my worse case scenario, could be >> from east coast to west coast. > > > The issues you'll run into are that in a high-load scenario, the > latency on that link could become a real issue, as could bandwidth > (actually, bandwidth is probably the bigger issue). The other > downside is that slony is decidedly async; if you have to fail over > you're probably going to lose some recently committed data. As long > as that's not an issue then it's your best bet. If it is an issue, > you'll probably be stuck rolling your own solution using something > like 2PC. Actually, one more question. Upon further reading, the documentation stated that any modification done on the master, will have to be committed to all cluster participants, otherwise it will not commit? Is it still possible to lose any committed data, with that being true? As a supplement, I was thinking about using some sort of PITR, in conjuction with Slony-I to ensure all bases are covered. > > You should hop on the slony mailing list if you have any questions > about it. > -- > Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com > Pervasive Software http://pervasive.com work: 512-231-6117 > vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 >
Clinging to sanity, louis.gonzales@linuxlouis.net (Louis Gonzales) mumbled into her beard:
> Hey Jim, Thanks again for the pointer to this. I've already
> compiled and installed on one of the two Solaris nodes, that I
> needed to. Yeah upon further reading, I can't wait for Slony-II to
> come out - is there truth behind the "anyday" rumor? - is it also
> true that it's going to implement the true multi-master scenario,
> where updates can be made at any of the clustered nodes.
I'm not holding my breath, at this point.
Performance results have indicated the Slony-II approach wouldn't work
out well for systems where there is heavy competition for locks on
some objects. The trouble is that you wind up having to distribute
locks, and if the application has common heavily-updated objects, the
cost winds up prohibitive...
> I'm going to deploy the "slon worker process" locally on every
> participating node, rather than letting the master host the "slon"
> processes for every cluster participant, for performance reasons.
I run quite a lot of Slony-I nodes, and let me be pointed...
Performance is NOT a good reason (or a reason at all) to spread slon
processes across a bunch of hosts.
Performance of replication is based on the I/O taking place in the
databases; the costs of distributing some of the slon network traffic
will be fairly immaterial.
The "savings" from hosting slons on specific DB nodes is an illusion.
The data will all have to cross the network to get from sources to
providers, so whether the slon is on one host or another, you'll have
the same traffic transmitted around.
Having slons centralized makes it way simpler to manage them; I can't
see there being anywhere near enough benefit from any savings of
network traffic to actually represent a material performance
improvement.
--
wm(X,Y):-write(X),write('@'),write(Y). wm('cbbrowne','cbbrowne.com').
http://linuxdatabases.info/info/slony.html
Never criticize anybody until you have walked a mile in their shoes,
because by that time you will be a mile away and have their shoes.
-- email sig, Brian Servis