Re: Parallel Sort
От | Kohei KaiGai |
---|---|
Тема | Re: Parallel Sort |
Дата | |
Msg-id | CADyhKSUieOzpqS0erkTxTHtZr999f5jJR68DfT=nv=TVipbZ3g@mail.gmail.com обсуждение исходный текст |
Ответ на | Parallel Sort (Noah Misch <noah@leadboat.com>) |
Ответы |
Re: Parallel Sort
|
Список | pgsql-hackers |
<div dir="ltr">2013/5/13 Noah Misch <span dir="ltr"><<a href="mailto:noah@leadboat.com" target="_blank">noah@leadboat.com</a>></span><br/><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote"style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">* Planner &Similar Issues<br /><br /> We should decide whether to actually sort in parallel based on the comparator<br /> costand the data size. The system currently has no information on comparator<br /> cost: bt*cmp (and indeed almost all built-infunctions) all have procost=1,<br /> but bttextcmp is at least 1000x slower than btint4cmp. Let's improve the<br/> procost estimates of all core B-tree and hash operators. This will have other<br /> benefits, but we will needto be cognizant of the risk of upsetting setups<br /> that have tuned cpu_operator_cost based on the present situation.<br/><br /> The choice of whether to parallelize can probably be made a manner similar to<br /> the choice to doan external sort: the planner guesses the outcome for costing<br /> purposes, but the actual decision is made at executiontime. The planner<br /> would determine a tuple count cutoff at which parallelism becomes favorable,<br /> andtuplesort would check that to establish its actual decision.<br /><br /></blockquote><div style="style">It probably crossoversmy <span style="color:rgb(0,0,0);font-family:Arial,'Arial New','\00ff2d\00ff33 \00ff30 \0030b4\0030b7\0030c3\0030af',sans-serif;font-size:13px">problemconsciousness to off-load CPU bounds</span></div><div style="style"><spanstyle="color:rgb(0,0,0);font-family:Arial,'Arial New','\00ff2d\00ff33 \00ff30 \0030b4\0030b7\0030c3\0030af',sans-serif;font-size:13px">workloads;that I partially tried to implement on writable foreigntable feature.</span></div><div style="style"><font color="#000000" face="Arial, Arial New, MS P ゴシック, sans-serif">Notonly sorting stuff, I think it may be worthful to have capability to push</font></div><div style="style"><fontcolor="#000000" face="Arial, Arial New, MS P ゴシック, sans-serif">heavy workload (like sort, aggregate orcomplex target-list) out external</font></div><div style="style"><font color="#000000" face="Arial, Arial New, MS P ゴシック,sans-serif">computing resources.</font></div><div style="style">However, I doubt whether the decision to parallelizeshould be done in</div><div style="style">execution time, rather than plan stage. For example, in case when we</div><divstyle="style">have enough number of records and 10-core multiprocessor, the wise</div><div style="style">planmay take parallel data load by 10-processors, partial-sort by 10-</div><div style="style">processors individually,then merge-sort. It needs fundamental different</div><div style="style">tree structure from the traditionalsingle-processors based plan-tree.</div><div style="style">So, it seems to me we should take an enhancement toallow to inject</div><div style="style">plan-tree special purpose parallel processing plan node.</div><div style="style">Howabout your opinion?</div><div style="style"><br /></div><div style="style">Thanks,</div></div></div></div>
В списке pgsql-hackers по дате отправления: