On 04/22/2014 06:39 AM, Andrew Dunstan wrote: > I agree, and indeed that was something like my first reaction to hearing > about this development - FDW seems like a very odd way to handle this. > But the notion of builtin columnar storage suggests to me that we really > need first to tackle how various storage engines might be incorporated > into Postgres. I know this has been a bugbear for many years, but maybe > now with serious proposals for alternative storage engines on the > horizon we can no longer afford to put off the evil day when we grapple > with it.
Yes. *IF* PostgreSQL already supported alternate storage, then the Citus folks might have released their CStore as a storage plugin instead of an FDW. However, if they'd waited for pluggable storage, they'd still be waiting.
I am sceptical - what I know about OLAP column store databases - they need a hardly different planner, so just engine or storage is not enough. Vector Wise try to merge Ingres with Monet engine more than four years - and still has some issues.
Our extensibility is probably major barrier against fast OLAP - I see a most realistic way to support better partitioning and going in direction higher parallelism and distribution - and maybe map/reduce support.
In GoodData we use successfully Postgres for BI projects to 20G with fast response - and most painfulness are missing MERGE, missing fault tolerant copy, IO expensive update of large tables with lot of indexes and missing simple massive partitioning. On second hand - Postgres works perfectly on thousands databases with thousands tables without errors with terrible simple deploying in cloud environment.