Re: [PERFORM] pg_dump and thousands of schemas

От: Stephen Frost
Тема: Re: [PERFORM] pg_dump and thousands of schemas
Дата: ,
Msg-id: 20120531040149.GC1267@tamriel.snowman.net
(см: обсуждение, исходный текст)
Ответ на: Re: [PERFORM] pg_dump and thousands of schemas  (Tom Lane)
Ответы: Re: [PERFORM] pg_dump and thousands of schemas  (Tom Lane)
Список: pgsql-hackers

Скрыть дерево обсуждения

Re: [PERFORM] pg_dump and thousands of schemas  (Tatsuo Ishii, )
 Re: [PERFORM] pg_dump and thousands of schemas  (Tom Lane, )
  Re: [PERFORM] pg_dump and thousands of schemas  (Stephen Frost, )
   Re: [PERFORM] pg_dump and thousands of schemas  (Tom Lane, )
  Re: [PERFORM] pg_dump and thousands of schemas  (Tatsuo Ishii, )
 Re: [PERFORM] pg_dump and thousands of schemas  (Bruce Momjian, )
  Re: [PERFORM] pg_dump and thousands of schemas  (Tom Lane, )
   Re: [PERFORM] pg_dump and thousands of schemas  (Bruce Momjian, )
   Re: [PERFORM] pg_dump and thousands of schemas  (Robert Haas, )
    Re: [PERFORM] pg_dump and thousands of schemas  (Tom Lane, )
     Re: [PERFORM] pg_dump and thousands of schemas  (Jeff Janes, )
      Re: [PERFORM] pg_dump and thousands of schemas  (Robert Haas, )

* Tom Lane () wrote:
> Tatsuo Ishii <> writes:
> > Shall I commit to master and all supported branches?
>
> I'm not excited by this patch.  It dodges the O(N^2) lock behavior for
> the initial phase of acquiring the locks, but it does nothing for the
> lock-related slowdown occurring in all pg_dump's subsequent commands.
> I think we really need to get in the server-side fix that Jeff Janes is
> working on, and then re-measure to see if something like this is still
> worth the trouble.  I am also a tad concerned about whether we might not
> have problems with parsing memory usage, or some such, with thousands of
> tables being listed in a single command.

I can't imagine a case where it's actually better to incur the latency
penalty (which is apparently on the order of *minutes* of additional
time here..) than to worry about the potential memory usage of having to
parse such a command.

If that's really a concern, where is that threshold, and could we simply
cap pg_dump's operations based on it?  Is 1000 alright?  Doing a 'lock'
w/ 1000 tables at a time is still going to be hugely better than doing
them individually and the amount of gain between every-1000 and
all-at-once is likely to be pretty minimal anyway...

The current situation where the client-to-server latency accounts for
multiple minutes of time is just ridiculous, however, so I feel we need
some form of this patch, even if the server side is magically made much
faster.  The constant back-and-forth isn't cheap.

    Thanks,

        Stephen


В списке pgsql-hackers по дате сообщения:

От: Heikki Linnakangas
Дата:
Сообщение: Re: GiST buffering build, bug in levelStep calculation
От: Simon Riggs
Дата:
Сообщение: Re: Uh, I change my mind about commit_delay + commit_siblings (sort of)