Re: Postgres and data warehouses

Поиск
Список
Период
Сортировка
От Brent Tubbs
Тема Re: Postgres and data warehouses
Дата
Msg-id CAAOBOv9jynL7=1GarS+STRczLfguQk25s7U9LuOExKO=HAk74A@mail.gmail.com
обсуждение исходный текст
Ответ на Re: Postgres and data warehouses  (Jim Nasby <Jim.Nasby@BlueTreble.com>)
Список pgsql-general
The logical replication features in 9.4 look very promising as an alternative to the binary replication that requires an identical schema on both ends. There's no reason that the thing you're replicating to couldn't be something like RabbitMQ or Storm.

I've been playing with a JSON-based decoder plugin so Postgres could emit changes to non-SQL systems like this.  See https://bitbucket.org/btubbs/decoder_json/src.  It should be still considered experimental and only used by people unafraid of hacking on C at this point.  More eyeballs would be welcome.

I blogged about more options on streaming updates from Postgres at http://btubbs.com/streaming-updates-from-postgres.html.

On Tue, Mar 10, 2015 at 8:11 AM, Jim Nasby <Jim.Nasby@bluetreble.com> wrote:
On 3/8/15 11:02 PM, Rob Sargent wrote:
I expected the latter but … some proprietors like to brag :)

Standing down.

On Mar 8, 2015, at 9:56 PM, John R Pierce <pierce@hogranch.com> wrote:

On 3/8/2015 8:24 PM, Rob Sargent wrote:
I strongly suspect many would like further details on your implementation. A user conference session well recorded perhaps?

the details are proprietary and quite specific to our workload and requirements.

You could build something similar with Skytools; it's what Skype actually created it for. They ran many "real time" datamarts/warehouses using it.

The big difference between it and Slony is that Skytools is modular, so it's designed to allow for this kind of flexibility.
--
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com



--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

В списке pgsql-general по дате отправления:

Предыдущее
От: Tomas Vondra
Дата:
Сообщение: Re: Benchmarking partitioning triggers and rules
Следующее
От: Tom Lane
Дата:
Сообщение: Re: PostgreSQL-related legal question