Re: In-Memory Columnar Store

<div class="moz-cite-prefix">Thank you very much for reporting the problem.<br /> And sorry for this bug and lack of
negativetests.<br /><br /> Attempt to access unexisted value cause autoloading of data from the table to columnar store
(becauseautoload property is enabled by default) <br /> and as far as this entry is not present in the table, the code
fallsinto infinite recursion.<br /> Patched version of IMCS is available at <a class="moz-txt-link-freetext"
href="http://www.garret.ru/imcs-1.01.tar.gz">http://www.garret.ru/imcs-1.01.tar.gz</a><br/><br /> I am going to place
IMCSunder version control now. Just looking for proper place for repository...<br /><br /><br /> On 12/12/2013 04:06
AM,desmodemone wrote:<br /></div><blockquote
cite="mid:CAEs9oFn920CSw_0k+TTa79cdF6zQC+TdHz3xwvbdXmXa_iEMZQ@mail.gmail.com"type="cite"><div dir="ltr"><br /><div
class="gmail_extra"><br/><br /><div class="gmail_quote">2013/12/9 knizhnik <span dir="ltr"><<a
href="mailto:knizhnik@garret.ru"moz-do-not-send="true" target="_blank">knizhnik@garret.ru</a>></span><br
/><blockquoteclass="gmail_quote" style="margin:0px 0px 0px             0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">Hello!<br /><br /> I want to annouce my implementation of In-Memory Columnar Store
extensionfor PostgreSQL:<br /><br />      Documentation: <a href="http://www.garret.ru/imcs/user_guide.html"
moz-do-not-send="true"target="_blank">http://www.garret.ru/imcs/user_guide.html</a><br />      Sources: <a
href="http://www.garret.ru/imcs-1.01.tar.gz"moz-do-not-send="true"
target="_blank">http://www.garret.ru/imcs-1.01.tar.gz</a><br/><br /> Any feedbacks, bug reports and suggestions are
welcome.<br/><br /> Vertical representation of data is stored in PostgreSQL shared memory.<br /> This is why it is
importantto be able to utilize all available physical memory.<br /> Now servers with Tb or more RAM are not something
exotic,especially in financial world.<br /> But there is limitation in Linux with standard 4kb pages  for maximal size
ofmapped memory segment: 256Gb.<br /> It is possible to overcome this limitation either by creating multiple segments -
butit requires too much changes in PostgreSQL memory manager.<br /> Or just set MAP_HUGETLB flag (assuming that huge
pageswere allocated in the system).<br /><br /> I found several messages related with MAP_HUGETLB flag, the most recent
onewas from 21 of November:<br /><a href="http://www.postgresql.org/message-id/20131125032920.GA23793@toroid.org"
moz-do-not-send="true"target="_blank">http://www.postgresql.org/message-id/20131125032920.GA23793@toroid.org</a><br
/><br/> I wonder what is the current status of this patch?<span class=""><font color="#888888"><br /><br /><br /><br
/><br/><br /><br /> -- <br /> Sent via pgsql-hackers mailing list (<a href="mailto:pgsql-hackers@postgresql.org"
moz-do-not-send="true"target="_blank">pgsql-hackers@postgresql.org</a>)<br /> To make changes to your subscription:<br
/><ahref="http://www.postgresql.org/mailpref/pgsql-hackers" moz-do-not-send="true"
target="_blank">http://www.postgresql.org/mailpref/pgsql-hackers</a><br/></font></span></blockquote></div><br /><br
/></div><divclass="gmail_extra">Hello,<br /></div><div class="gmail_extra">           excellent work! I begin to do
testingand it's very fast, by the way I found a strange case of "endless" query with CPU a 100%  when the value used as
filterdoes not exists:<br /><br /></div><div class="gmail_extra">I am testing with postgres 9.3.1 on debian  and I used
defaultvalue for the extension except memory ( 512mb )<br /><br /></div><div class="gmail_extra">how to recreate the
testcase :<br /></div><div class="gmail_extra"><br /></div><div class="gmail_extra">## create a table :<br /><br
/></div><divclass="gmail_extra">create table endless ( col1 int , col2 char(30) , col3 int ) ;<br /><br /></div><div
class="gmail_extra">##insert some values:<br /></div><div class="gmail_extra"><br /> insert into endless values ( 1,
'ahahahaha',3);<br /><br /> insert into endless values ( 2, 'ghghghghg', 4);<br /><br /> ## create the column store
objects:<br/><br /> select cs_create('endless','col1','col2');<br />  cs_create <br /> -----------<br />  <br /> (1
row)<br/><br /></div><div class="gmail_extra">## try and test column store :<br /></div><div class="gmail_extra"><br />
selectcs_avg(col3) from  endless_get('ahahahaha');<br />  cs_avg <br /> --------<br />       3<br /> (1 row)<br /><br
/>select cs_avg(col3) from  endless_get('ghghghghg');<br />  cs_avg <br /> --------<br />       4<br /> (1 row)<br
/><br/></div><div class="gmail_extra">## now select with a value that does not exist :<br /></div><div
class="gmail_extra"><br/> select cs_avg(col3) from  endless_get('testing');<br /><br /></div><div class="gmail_extra">#
andnow  start to loop on cpu and seems to never ends , I had to terminate backend<br /></div><div
class="gmail_extra"><br/></div><div class="gmail_extra">Bye<br /><br /> Mat<br /></div></div></blockquote><br />