Re: Relation extension scalability
| От | Dilip Kumar |
|---|---|
| Тема | Re: Relation extension scalability |
| Дата | |
| Msg-id | CAFiTN-s642YKPv1mTNCDYBtrZh2OPVz-bWM8TyXGKKKEZ7qNPg@mail.gmail.com обсуждение исходный текст |
| Ответ на | Re: Relation extension scalability (Andres Freund <andres@anarazel.de>) |
| Ответы |
Re: Relation extension scalability
(Dilip Kumar <dilipbalaut@gmail.com>)
|
| Список | pgsql-hackers |
On Sun, Jul 19 2015 9:37 PM Andres Wrote,
> extend to the same block, but after one of them succeeded the buffer is
> written out and reused for an independent page. So there is no in-memory
> state telling the slower backend that that page has already been used.
I was looking into this patch, and done some performance testing..
Currently i have done testing my my local machine, later i will perform on big machine once i get access to that.
Just wanted to share the current result what i get i my local machine
Machine conf (Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz, 8 core and 16GM of RAM).
Test Script:
./psql -d postgres -c "COPY (select g.i::text FROM generate_series(1, 10000) g(i)) TO '/tmp/copybinarywide' WITH BINARY";
./psql -d postgres -c "truncate table data"
./psql -d postgres -c "checkpoint"
./pgbench -f copy_script -T 120 -c$ -j$ postgres
Summary of the results:
1. When data fits into shared buffer improvement is not visible, but when it don't then some improvement is visible in my local machine (still does not seems to be scaling, may be we can see some different behaviour in big machine), Thats because in first case it need not to flush the buffer out.
2. As per Tom's analysis since we are doing extra read it will reduce performance in lower no of client where RelationExtensionLock is not bottleneck and same is visible in test result.
As discussed earlier, what about keeping the RelationExtentionLock as it is and just do the victim buffer search and buffer flushing outside the lock, that way we can save extra read. Correct me if i have missed something in this..
Shared Buffer 512 MB
-----------------------------
Client: Tps Base Tps Patch
1 145 126
2 211 246
4 248 302
8 225 234
Shared Buffer 5GB
-----------------------------
Client: Tps Base Tps Patch
1 165 156
2 237 244
4 294 296
8 253 247
Also observed one problem with patch,
@@ -433,63 +434,50 @@ RelationGetBufferForTuple(Relation relation, Size len,
+ while(true)
+ {
+ buffer = ExtendRelation(relation, MAIN_FORKNUM, bistate->strategy);
bistate can be NULL if direct insert instead of copy case
On Sun, Jul 19, 2015 at 9:37 PM, Andres Freund <andres@anarazel.de> wrote:
On 2015-07-19 11:56:47 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2015-07-19 11:28:25 -0400, Tom Lane wrote:
> >> At this point session 1 will go and create page 44, won't it, and you
> >> just wasted a page.
>
> > My local code now recognizes that case and uses the page. We just need
> > to do an PageIsNew().
>
> Er, what? How can you tell whether an all-zero page was or was not
> just written by another session?
The check is only done while holding the io lock on the relevant page
(have to hold that anyway), after reading it in ourselves, just before
setting BM_VALID. As we only can get to that point when there wasn't any
other entry for the page in the buffer table, that guarantees that no
other backend isn't currently expanding into that page. Others might
wait to read it, but those'll wait behind the IO lock.
The situation the read() protect us against is that two backends try to
extend to the same block, but after one of them succeeded the buffer is
written out and reused for an independent page. So there is no in-memory
state telling the slower backend that that page has already been used.
Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
В списке pgsql-hackers по дате отправления:
Предыдущее
От: Aleksander AlekseevДата:
Сообщение: Re: Patch: fix lock contention for HASHHDR.mutex