On Tue, Aug 26, 2014 at 4:01 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Josh Berkus <josh@agliodbs.com> writes:
>> On 08/26/2014 11:40 AM, Tom Lane wrote:
>>> I was hoping you'd get some useful data from that, but so far it seems
>>> like a rehash of points made in the on-list thread :-(
>
>> Unfortunately even the outside commentors don't seem to understand that
>> storage size *is* related to speed, it's exchanging I/O speed for CPU speed.
>
> Yeah, exactly. Given current hardware trends, data compression is
> becoming more of a win not less as time goes on: CPU cycles are cheap
> even compared to main memory access, let alone mass storage. So I'm
> thinking we want to adopt a compression-friendly data format even if
> it measures out as a small loss currently.
>
> I wish it were cache-friendly too, per the upthread tangent about having
> to fetch keys from all over the place within a large JSON object.
What about my earlier proposal?
An in-memory compressed representation would greatly help cache
locality, more so if you pack keys as you mentioned.