Re: Postgres and data warehouses
От
John R Pierce
Тема
Re: Postgres and data warehouses
Дата
Msg-id
54FCD3F3.2010307@hogranch.com
Ответ на
Postgres and data warehouses (Nigel Gardiner)
Список
Дерево обсуждения
Postgres and data warehouses Nigel Gardiner <nigelgardiner@gmail.com>
Re: Postgres and data warehouses Stephen Frost <sfrost@snowman.net>
Re: Postgres and data warehouses Bill Moran <wmoran@potentialtech.com>
Re: Postgres and data warehouses John R Pierce <pierce@hogranch.com>
Re: Postgres and data warehouses Rob Sargent <robjsargent@gmail.com>
Re: Postgres and data warehouses John R Pierce <pierce@hogranch.com>
Re: Postgres and data warehouses Rob Sargent <robjsargent@gmail.com>
Re: Postgres and data warehouses Jim Nasby <Jim.Nasby@BlueTreble.com>
Re: Postgres and data warehouses Brent Tubbs <brent.tubbs@gmail.com>
Re: Postgres and data warehouses Jerry Sievers <gsievers19@comcast.net>
On 3/8/2015 7:40 AM, Nigel Gardiner wrote: > I'm looking at making a data warehouse to address our rapidly > spiralling report query times against the OLTP. I'm looking first at > what it would take to make this a real-time data warehouse, as opposed > to batch-driven. we use a hybrid architecture. we have a 'republisher' process that repeatedly slurps new data from the OLTP database and sends it to the back end databases, using a 'publish/subscribe' messaging bus. several back end databases subscribe to this data, and their subscriber processes insert the incoming data into those OLAP and various other reporting databases. this way the reporting databases can have completely different schemas optimized for their needs, and have different retention requirements than the OLTP database. this republisher is usually within a few seconds of live new data. in our case its made fairly easy to track 'new' because all our OLTP transactions are event-oriented. -- john r pierce 37N 122W somewhere on the middle of the left coast
В списке pgsql-general по дате отправления
