Re: [HACKERS] Adding hook in BufferSync for backup purposes

Поиск
Список
Период
Сортировка
От Andrey Borodin
Тема Re: [HACKERS] Adding hook in BufferSync for backup purposes
Дата
Msg-id 449A7A9D-DB58-40F8-B80E-4C4EE7DB47FD@yandex-team.ru
обсуждение исходный текст
Ответ на Re: [HACKERS] Adding hook in BufferSync for backup purposes  (Tom Lane <tgl@sss.pgh.pa.us>)
Список pgsql-hackers
Hi hackers!

> 8 авг. 2017 г., в 11:27, Tom Lane <tgl@sss.pgh.pa.us> написал(а):
>
> My point is not to claim that we mustn't put a hook there.  It's that what
> such a hook could safely do is tightly constrained, and you've not offered
> clear evidence that there's something useful to be done within those
> constraints.  Alvaro seems to think that the result might be better
> reached by hooking in at other places, and my gut reaction is similar.
>


I was studying through work done by Marco and Gabriel on the matter of tracking pages for the incremental backup, and I
havefound PTRACK patch by Yury Zhuravlev and PostgresPro [0]. This work seems to be solid and thoughtful. I have
drafteda new prototype for hooks enabling incremental backup as extension based on PTRACK calls. 

If you look at the original patch you can see that attached patch differs slightly: functionality to track end of
criticalsection is omitted. I have not included it in this draft because it changes very sensitive place even for those
whodo not need it. 

Subscriber of proposed hook must remember that it is invoked under critical section. But there cannot me more than
XLR_MAX_BLOCK_IDblocks for one transaction. Proposed draft does not add any functionality to track finished
transactionsor any atomic unit of work, just provides a flow of changed block numbers. Neither does this draft provide
anyassumption on where to store this information. I suppose subscribers could trigger asynchronous writes somewhere as
longas info for given segment is accumulated (do we need a hook on segment end then?). During inremental backup you can
skipscanning those WAL segments for which you have accumulated changeset of block numbers. 

The thing which is not clear to me: if we are accumulating blocknumbers under critical section, then we must place them
topreallocated array. When is the best time to take away these blocknumbers to empty that array and avoid overflow?
PTRACKhas array of XLR_MAX_BLOCK_ID length and saves these array during the end of each critical section. But I want to
avoidintervention into critical sections. 

Thank you for your attention, any thoughts will be appreciated.

Best regards, Andrey Borodin.

[0] https://gist.github.com/stalkerg/ab833d94e2f64df241f1835651e06e4b

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Вложения

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Dilip Kumar
Дата:
Сообщение: Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages
Следующее
От: Mithun Cy
Дата:
Сообщение: Re: [HACKERS] POC: Cache data in GetSnapshotData()