Re: [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

I'm sorry I misunderstood about the extension you wrote.

Is there some way not to use shared memory for it?

Cheers,
David.

On Sat, Jan 04, 2014 at 11:46:25AM +0400, knizhnik wrote:
> Hi David,
> 
> Sorry, but I do not completely understand your suggestions:
> 
> 1. IMCS really contains single patch file sysv_shmem.patch.
> Applying this patch is not mandatory for using IMCS: it just solves
> the problem with support of > 256Gb of shared memory.
> Right now PostgreSQL is not able to use more than 256Gb shared
> buffers at Linux with standard 4kb pages.
> I have found proposal for using MAP_HUGETLB flag in commit fest:
> 
> http://www.postgresql.org/message-id/20131125032920.GA23793@toroid.org
> 
> but unfortunately it was rejected. Hugepages are intensively used by
> Oracle and I think that them will be useful for improving
> performance of PorstreSQL. So not just IMCS can benefit from this
> patch. My patch  is much more simple - I specially limited scope of
> this patch to one file. Certainly switch huge tlb on/off should be
> done through postgresql.conf configuration file.
> 
> In any case - IMCS can be used without this patch: you just could
> not use more than 256Gb memory, even if your system has more RAM.
> 
> 2. I do not understand "The add-on is not formatted as an EXTENSION"
> IMCS was created as standard extension - I just look at the examples
> of other PostgreSQL extensions included in PostgreSQL distribution
> (for example pg_stat_statements). It can be added using "create
> extension imcs" and removed "drop extension imcs" commands.
> 
> If there are some violations of PostgreSQL extensions rules, please
> let me know, I will fix them.
> But I thought that I have done everything in legal way.
> 
> 
> 
> 
> 
> 
> On 01/04/2014 03:21 AM, David Fetter wrote:
> >On Thu, Jan 02, 2014 at 08:48:24PM +0400, knizhnik wrote:
> >>I want to announce implementation of In-Memory Columnar Store
> >>extension for PostgreSQL.
> >>Vertical representation of data is stored in PostgreSQL shared memory.
> >Thanks for the hard work!
> >
> >I noticed a couple of things about this that probably need some
> >improvement.
> >
> >1.  There are unexplained patches against other parts of PostgreSQL,
> >which means that they may break other parts of PostgreSQL in equally
> >inexplicable ways.  Please rearrange the patch so it doesn't require
> >this.  This leads to:
> >
> >2.  The add-on is not formatted as an EXTENSION, which would allow
> >people to add it or remove it cleanly.
> >
> >Would you be so kind as to fix these?
> >
> >Cheers,
> >David.

-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate