Re: PostgreSQL 64 Bit XIDs - Transaction IDs

On Fri, Mar 23, 2012 at 11:18 PM, Bret Stern
<bret_stern@machinemanagement.com> wrote:
> VoltDB maybe
> ----- Original Message -----

VoltDB has a completely different focus than PostgreSQL really.
PostgreSQL is a general purpose database that can achieve some very
impressive numbers using super fast hardware, while still being a
general purpose db that can do more than just shove through thousands
of transactions a second.

For instance, I've tested machines with spinning drives that could
sustain, for weeks, 5 to 7k tps.  Autovacuum suitably tuned to be fast
enough to keep up (i.e. no sleep time, greatly increased cost ilmit,
lots of checkpoint segments, and very aggressive bgwriter settings as
well.

I can imagine doubling that with a rack of SSDs behind it.  And all on
a single machine, that can be running PITR or streaming replication at
the same time.

VoltDB accomplishes this kind of throughput by running in memory,
doing a kind of auto-sharded, RAIDed database setup for redundancy.  I
imagine a farm of little 1U 8 or 16 core machines running multiple
instances of VoltDB (individual instances are single threaded) could
just stomp pg right into the ground as a transactional engine.
Because now you're scaling horizontally

No matter how fast the hardware underlying it gets, pg will have a
limit by the laws of physics on an individual machine that VoltDB
simply works around by having dozens or hundreds of individual boxes
doing the same job, albeit in a very specilalized manner.

I've not played with VoltDB and I don't know how stable and reliable
it is yet, but it was built by Michael Stonebraker, so I'd expect it
to at least hold the promise of stable operation as it matures, if not
being stable and reliable now.

But the % of db systems that need that level of performance are pretty
limited, and generally designed to have a large chunk of the world's
population working on them at the same time.

For a system that's more typical, i.e. 95% or more read, you can throw
memcache in front of even a pretty mediocre box and have it keep up
just fine for hundreds or even thousands of writes per second and
hundreds of thousands of reads to the cache layer per second.

There are real world problems with redundancy and reliability to work
out with VoltDB that are quite different from Postgresql, since you
need real synchronous replication from two geographically separated
data centers, because if something horrible happens in one, (it gets
hit by a meteor, or some junior electrician drops a piece of wire into
a power conditioner, one whole segment of your db fabric could go
dark.  And if it's not synchronous, then the replication is useless
for a true transactional banking style system.  Keep in mind a pg
server simply losing power is usually safe from data loss, and a
nearby machine with syncrhonous replication is likely to provide a
high enough level of redundancy for most transactional systems.  Since
VoltDB is in memory, you HAVE to have a live backup running somewhere
remote, or a simple power loss kills all your data / latest
transactions.