Re: Replication options?

  > As far as I know (Joshua please clearify here) Mammoth Replicator
writes
> its own, binary journal containing the changes that need to be applied
> to the replica.


Yes that is correct.

>
> On a first look inserting into database tables might look more
> expensive. But there are some fine details that make it worth taking a
> second look. One side effect of doing this is that collecting the
> replication log together with changing the data on the origin (master)
> is automatically covered by the exact same ACID properties the database
> provides. I am not sure if or how Replicator guarantees that under all
> possible circumstances and server crash situations the replication log
> journal will contain exacly all committed transactions, and only those.

Replicator does not replicate until the transaction is committed.

> To make a transaction durable, the changes first have to be recorded in
> PostgreSQL's crash recovery WAL. Only after that data is flushed to the
> disk it can be assumed that the transaction will be redone in the case
> of an immediately following crash. If a replication system now logs the
> commit event before the WAL operation happens, it is possible that the
> transaction does not commit on the master due to a crash, but it will be
> replayed and committed on the slaves. On the other side if the
> replication logging of the commit is done after the WAL operation, it
> must be assured that WAL replay during crash recovery also causes
> replication log journal to be recovered or repeated. In short, the
> replication log must be covered by the same redo mechanism the crash
> recovery system uses.

This I will have to verify with our programmers as to exactly "when" the
replication occurs.

>
> This all is only important for the case that one does not immediately
> slam on the big red panic button and issues a full failover when the
> main server crashes, but rather tries to bring the main server back. If
> it can boot,

That is correct. It is important to remember that one should not just
"failover". You need to know why... what happen, happen. If you take two
minutes bring that master back up it is often apparent what needs to be
done quickly to get the machine in operational condition.


> has no FS inconsistencies and PostgreSQL's crash recovery
> succeeds too, there is no need to fail over any more and one will want
> to resume normal operation. If now the strict synchronization between
> what PostgreSQL's crash recovery mechanism does restore and what will be
> applied to the slave systems cannot be guaranteed, then there is the
> possibility of loss of synchronization between master and slaves. You
> just lost the data integrity of your backup server.

If Replicator looses synchronization, it will attempt to resync. If it
can not, it will completely wipe a slave and perform a full dump to make
sure the slave in question is in sync.

> Slony-I has the replication log journal covered by PostgreSQL's native
> ACID properties. I assume Joshua can explain how Mammoth Replicator
> solves this problem.
>
> Another important difference is automatic replication of schema changes.
> This is a feature often asked for, and I have no idea where that wish
> comes from. Certainly it does not stem from too much thinking about the
> problem. Slony-I does not attempt to go into that direction. A trigger
> based solution like Slony-I cannot do it anyway. Again, Joshua, what's
> the plans or status with Replicator on that?

We will not as it would be on many levels of crazy :) replicate schema
changes.

> The reason why I consider automatic schema replication a subintelligent
> idea is simply that it makes special purpose configurations impossible.
> If one only needs a full backup server for failover, it sure is usefull.

I don't even know if I agree with that. A database schema should be
relatively static once it goes into production. If you have to make a
database schema change then one should test, test, test on dev machines
and then schedule an outage for the schema changes.

Also even if we were to replicate schema changes it would almost
guarantee a full dump on every change you made.


>> Does the Slony-I replications operation on BSD? Is there a reviews of
>> the Mammoth online?
>

BSD is a platform that is coming for Replicator. The majority (95%) of
our demand has been on Linux and thus is obviously our most supported
platform.

Next would be Solaris, and then about half a dozen requests for BSD.

Sincerely,

Joshua D. Drake



>
> I use FreeBSD 4.9 for most of the development. In general Slony-I should
> run on every PostgreSQL supported Unix platform that provides pthreads.
>
>
> Jan
>


--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL