Re: amazon ec2

On Wed, 4 May 2011, Josh Berkus wrote:

> Date: Wed, 04 May 2011 17:02:53 -0700
> From: Josh Berkus 
> To: postgres performance list 
> Subject: Re: [PERFORM] amazon ec2
> 
>
>> FWIW, EnterpriseDB's "InfiniCache" provides the same caching benefit. The way that works is when PG goes to evict a page from shared buffers that page gets compressed and stuffed into a memcache cluster. When PG determines that a given page isn't in shared buffers it will then check that memcache cluster before reading the page from disk. This allows you to cache amounts of data that far exceed the amount of memory you could put in a physical server.
>
> So memcached basically replaces the filesystem?
>
> That sounds cool, but I'm wondering if it's actually a performance
> speedup.  Seems like it would only be a benefit for single-row lookups;
> any large reads would be a mess.

I think it would depend a lot on how well the page compresses.

if the network I/O plus uncompression time is faster than the seek and 
read from disk it should be a win.

I don't see why the benifit would be limited to single row lookups, 
anything within that page should be the same.

for multipage actions, the disk would have less of a disadvantage as 
readahead may be able to hide some of the work while the memcache approach 
would need to do separate transactions for each page.

this does bring up an interesting variation for hierarchical storage, 
using compressed pages in memcache rather than dedicated resources on a 
local system. thanks for the thought and I'll keep it in mind. I can think 
of lots of cases where the database stores a relativly small set of values 
that would compress well.

David Lang