Preventing deadlock on parallel backup

Поиск
Список
Период
Сортировка
От Lucas
Тема Preventing deadlock on parallel backup
Дата
Msg-id CAEWGB6_tbkq6UpmHBjZu6rTEC+p0OLzCuWn-SeOv_ENizadx9Q@mail.gmail.com
обсуждение исходный текст
Ответы Re: Preventing deadlock on parallel backup  (Tom Lane <tgl@sss.pgh.pa.us>)
Список pgsql-hackers
<p dir="ltr">People,<p dir="ltr">I made a small modification in pg_dump to prevent parallel backup failures due to
exclusivelock requests made by other tasks. <p dir="ltr">The modification I made take shared locks for each parallel
backupworker at the very beginning of the job. That way, any other job that attempts to acquire exclusive locks will
waitfor the backup to finish.<p dir="ltr">In my case, each server was taking a day to complete the backup, now with
parallelbackup one is taking 3 hours and the others less than a hour.<p dir="ltr">The code below is not very elegant,
butit works for me. My whishlist for the backup is:<p dir="ltr">1) replace plpgsql by c code reading the backup toc and
assemblingthe lock commands.<br /> 2) create an timeout to the locks.<br /> 3) broadcast the end of copy to every
workerin order to release the locks as early as possible;<br /> 4) create a monitor thread that prioritize an copy job
basedon a exclusive lock acquired;<br /> 5) grant the lock for other connection of the same distributed transaction if
itis held by any connection of the same distributed transaction. There is some sideefect I can't see on that?<p
dir="ltr">1to 4 are within my capabilities and I may do it in the future. 4 is to advanced for me and I do not dare to
messwith something so fundamental rights now.<p dir="ltr">Anyone else is working on that?<br /><p dir="ltr">On,
Parallel.c,void RunWorker(...), add:<p dir="ltr">PQExpBuffer query;<br /> PGresult   *res;<p dir="ltr">query =
createPQExpBuffer();           <br /> resetPQExpBuffer(query);<br /> appendPQExpBuffer(query,<br /> "do language
'plpgsql'$$"<br /> " declare "<br /> "    x record;"<br /> " begin"<br /> "    for x in select * from pg_tables where
schemanamenot in ('pg_catalog','information_schema') loop"<br /> "        raise info 'lock table %.%', x.schemaname,
x.tablename;"<br/> "        execute 'LOCK TABLE '||quote_ident(x.schemaname)||'.'||quote_ident(x.tablename)||' IN
ACCESSSHARE MODE NOWAIT';"<br /> "    end loop;"<br /> "end"<br /> "$$" );<p dir="ltr">res = PQexec(AH->connection,
query->data);<pdir="ltr">if (!res || PQresultStatus(res) != PGRES_COMMAND_OK)<br />        
exit_horribly(modulename,"Couldnot lock the tables to begin the work\n\n");<br /> PQclear(res);<br />
destroyPQExpBuffer(query);<br/> 

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Stephen Frost
Дата:
Сообщение: Re: Add support for restrictive RLS policies
Следующее
От: Tom Lane
Дата:
Сообщение: Re: Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]