On 04/22/2014 08:04 AM, Simon Riggs wrote:
> On 22 April 2014 00:24, Josh Berkus <josh@agliodbs.com> wrote:
>> On 04/21/2014 03:41 PM, Simon Riggs wrote:
>>> Storage Efficiency
>>> * Compression
>>> * Column Orientation
>> You might look at turning this:
>>
>> http://citusdata.github.io/cstore_fdw/
>>
>> ... into a more integrated part of Postgres.
> Of course I'm aware of that work - credit to them. Certainly, many
> people feel that it is now time to do as you suggest and include
> column store features within PostgreSQL.
>
> As to turning it into a more integrated part of Postgres, we have a
> few problems there
>
> 1. cstore_fdw code has an incompatible licence
>
> 2. I don't think FDWs are the right place for complex new
> architectures such as column store, massively parallel processing or
> sharding. The fact that it is probably the best place to implement it
> in user space doesn't mean it transfers well into core code. That's a
> shame and I don't know what to do about it, because it would be nice
> to simply ask for change of licence and then integrate it, but it
> seems more work than that (to me).
>
I agree, and indeed that was something like my first reaction to hearing
about this development - FDW seems like a very odd way to handle this.
But the notion of builtin columnar storage suggests to me that we really
need first to tackle how various storage engines might be incorporated
into Postgres. I know this has been a bugbear for many years, but maybe
now with serious proposals for alternative storage engines on the
horizon we can no longer afford to put off the evil day when we grapple
with it.
cheers
andrew