Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

Поиск
Список
Период
Сортировка
Искать
От
Andres Freund
Тема
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?
Дата
Msg-id
20230217173114.kxnzc7wt7xtusnlv@awork3.anarazel.de
Ответ на
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? (Michael Paquier)
Список
Дерево обсуждения
Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Thomas Munro <thomas.munro@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Thomas Munro <thomas.munro@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Nathan Bossart <nathandbossart@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Nathan Bossart <nathandbossart@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Nathan Bossart <nathandbossart@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Thomas Munro <thomas.munro@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Nathan Bossart <nathandbossart@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Thomas Munro <thomas.munro@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Nathan Bossart <nathandbossart@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Thomas Munro <thomas.munro@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Thomas Munro <thomas.munro@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Thomas Munro <thomas.munro@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Nathan Bossart <nathandbossart@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Thomas Munro <thomas.munro@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Nathan Bossart <nathandbossart@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Andres Freund <andres@anarazel.de>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Andres Freund <andres@anarazel.de>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Alvaro Herrera <alvherre@alvh.no-ip.org>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Andres Freund <andres@anarazel.de>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Andres Freund <andres@anarazel.de>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Andres Freund <andres@anarazel.de>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Andres Freund <andres@anarazel.de>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Andres Freund <andres@anarazel.de>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Andres Freund <andres@anarazel.de>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Andres Freund <andres@anarazel.de>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Andres Freund <andres@anarazel.de>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Thomas Munro <thomas.munro@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Thomas Munro <thomas.munro@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Andres Freund <andres@anarazel.de>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Thomas Munro <thomas.munro@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Julien Rouhaud <rjuju123@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Alvaro Herrera <alvherre@alvh.no-ip.org>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Thomas Munro <thomas.munro@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Andres Freund <andres@anarazel.de>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Andres Freund <andres@anarazel.de>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? John Naylor <john.naylor@enterprisedb.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Michael Paquier <michael@paquier.xyz>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? John Naylor <john.naylor@enterprisedb.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> 
Hi,

On 2023-02-17 16:19:46 +0900, Michael Paquier wrote:
> But it looks like I misunderstood what this quote meant compared to
> what v3 does.  It is true that v3 sets iov_len and iov_base more than
> needed when writing sizes larger than BLCKSZ.

I don't think it does for writes larger than BLCKSZ, it just does more for
writes larger than PG_IKOV_MAX * BLCKSZ. But in those cases CPU time is going
to be spent elsewhere.


> Seems like you think that it is not really going to matter much to track
> which iovecs have been already initialized during the first loop on
> pg_pwritev_with_retry() to keep the code shorter?

Yes. I'd bet that, in the unlikely case you're going to see any difference at
all, unconditionally initializing is going to win.

Right now we memset() 8KB, and iterate over 32 IOVs, unconditionally, on every
call. Even if we could do some further optimizations of what I did in the
patch, you can initialize needed IOVs repeatedly a *lot* of times, before it
shows up...

I'm inclined to go with my version, with the argument order swapped to
Bharath's order.

Greetings,

Andres Freund
В списке pgsql-hackers по дате отправления
Предыдущее
От: Pavel Stehule
Дата:
Следующее
От: Andres Freund
Дата:
FAQ