Re: DBmirror replication - replacement for DBMirror.pl

Just to add - the replacement for DBMirror.pl automatically works out the schema in use and will replicate using DBmirror from 7.x or from 8.x (table names all changed!). This is done transparently.

I also wrote a short article at http://www.whitebeam.org/library/guide/TechNotes/replicate.rhtm

Pete

Peter Wilson wrote:

Not sure whether this is any use to anyone, or whether this is the right list to post to but...

I've just released a C++ implementation of the DBMirror.pl script as part of Whitebeam (http://www.whitebeam.org). We had *real* performance issues with the Perl implementation replicating large fields. It's an almost drop in replacement - except configuration file format is difffernt. It's also more fault tolerant than the Perl implementation and has some parameters to help distribute the replication load over time (if you delete 10,000 rows from a table, you don't really want you're main application to grind to a halt as replication soak up most of your resource!)

I needed to do this quickly - so it links to the utility classes in Whitebeam, things like a string class etc. It wouldn't be too difficult to decouple these and if there is any interest I'll do that when I get a little spare time. Once it's built though it's entirely generic and doesn't use anything else from Whitebeam.

If anyone has contact information for the original DBMirror author then I'd like to get in touch.

Would like to get feedback from anyone using DBmirror or thinks this would be useful..

Background
========

Our Whitebeam application server uses PostgreSQL for back-end data storage. We originally used IBM DB2, but ported to Postgres about 3 years ago, but we never sorted out replication (partly because we were using Large Objects). I recently sorted that out and looked around for a replication scheme to use - first choice being Slony-I. I couldn't get it to work - the configuration scheme just kept claiming half our schema tables didn't have primary keys (they did!). Also the documentation was virtually non-existent.

Dropping back to DBmirror (which comes with the PostgreSQL distribution in the 'contrib' directory) worked fine-ish. Unfortunately the replication Perl script was un-usably slow, taking minutes to replicate a 100K BYTEA field, which we used to store images. The replication Perl script seemed to be rather inefficient, using a lot of regular expressions to decode field values etc. Perl isn't something I felt too confident in - and I needed a solution quickly and hence the C++ implementation

Pete
--
Peter Wilson
YellowHawk : http://www.yellowhawk.co.uk
Whitebeam : http:/www.whitebeam.org
-----


---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend