Re: pg_dump and thousands of schemas

От: Tom Lane
Тема: Re: pg_dump and thousands of schemas
Дата: ,
Msg-id: 15138.1338243996@sss.pgh.pa.us
(см: обсуждение, исходный текст)
Ответ на: Re: pg_dump and thousands of schemas  (Jeff Janes)
Ответы: Re: pg_dump and thousands of schemas  ("Hugo <Nabble>")
Список: pgsql-performance

Скрыть дерево обсуждения

pg_dump and thousands of schemas  ("Hugo ", )
 Re: pg_dump and thousands of schemas  (Craig James, )
  Re: pg_dump and thousands of schemas  (Jeff Janes, )
   Re: pg_dump and thousands of schemas  (Bruce Momjian, )
    Re: pg_dump and thousands of schemas  ("Hugo <Nabble>", )
     Re: pg_dump and thousands of schemas  (Ondrej Ivanič, )
     Re: pg_dump and thousands of schemas  (Tom Lane, )
      Re: pg_dump and thousands of schemas  (Bruce Momjian, )
       Re: pg_dump and thousands of schemas  (Jeff Janes, )
      Re: pg_dump and thousands of schemas  (Tatsuo Ishii, )
       Re: pg_dump and thousands of schemas  (Tatsuo Ishii, )
        Re: pg_dump and thousands of schemas  (Tom Lane, )
         Re: pg_dump and thousands of schemas  (Tatsuo Ishii, )
         Re: pg_dump and thousands of schemas  (Tatsuo Ishii, )
          Re: pg_dump and thousands of schemas  (Tatsuo Ishii, )
           Re: pg_dump and thousands of schemas  (Tom Lane, )
            Re: pg_dump and thousands of schemas  (Stephen Frost, )
             Re: pg_dump and thousands of schemas  (Tom Lane, )
            Re: pg_dump and thousands of schemas  (Tatsuo Ishii, )
           Re: pg_dump and thousands of schemas  (Bruce Momjian, )
            Re: pg_dump and thousands of schemas  (Tom Lane, )
             Re: pg_dump and thousands of schemas  (Bruce Momjian, )
             Re: [HACKERS] pg_dump and thousands of schemas  (Robert Haas, )
              Re: [HACKERS] pg_dump and thousands of schemas  (Tom Lane, )
               Re: [HACKERS] pg_dump and thousands of schemas  (Jeff Janes, )
                Re: [HACKERS] pg_dump and thousands of schemas  (Robert Haas, )
                 Re: [HACKERS] pg_dump and thousands of schemas  (Denis, )
                  Re: [HACKERS] pg_dump and thousands of schemas  (Tom Lane, )
                   Re: [HACKERS] pg_dump and thousands of schemas  (Denis, )
                    Re: [HACKERS] pg_dump and thousands of schemas  (Tom Lane, )
                     Re: [HACKERS] pg_dump and thousands of schemas  (Denis, )
                      Re: [HACKERS] pg_dump and thousands of schemas  (Tom Lane, )
                       Re: [HACKERS] pg_dump and thousands of schemas  (Denis, )
                        Re: [HACKERS] pg_dump and thousands of schemas  (Jeff Janes, )
                         Re: [HACKERS] pg_dump and thousands of schemas  (Denis, )
                          Re: [HACKERS] pg_dump and thousands of schemas  (Jeff Janes, )
                          Re: [HACKERS] pg_dump and thousands of schemas  (Andrew Dunstan, )
          Re: pg_dump and thousands of schemas  (Jeff Janes, )
           Re: pg_dump and thousands of schemas  (Jeff Janes, )
            Re: pg_dump and thousands of schemas  (Tatsuo Ishii, )
             Re: pg_dump and thousands of schemas  (Jeff Janes, )
              Re: pg_dump and thousands of schemas  (Tatsuo Ishii, )
               Re: pg_dump and thousands of schemas  ("Hugo <Nabble>", )
        Re: pg_dump and thousands of schemas  (Tatsuo Ishii, )
         Re: pg_dump and thousands of schemas  (Robert Klemme, )
          Re: pg_dump and thousands of schemas  (Tatsuo Ishii, )
           Re: pg_dump and thousands of schemas  (Robert Klemme, )
            Re: pg_dump and thousands of schemas  (Claudio Freire, )
             Re: [HACKERS] pg_dump and thousands of schemas  (Tom Lane, )
              Re: [HACKERS] pg_dump and thousands of schemas  (Robert Haas, )
               Re: [HACKERS] pg_dump and thousands of schemas  (Tom Lane, )
                Re: [HACKERS] pg_dump and thousands of schemas  (Bruce Momjian, )
                Re: [HACKERS] pg_dump and thousands of schemas  (Claudio Freire, )
                 Re: [HACKERS] pg_dump and thousands of schemas  (Tom Lane, )
                  Re: [HACKERS] pg_dump and thousands of schemas  (Claudio Freire, )
                Re: [HACKERS] pg_dump and thousands of schemas  (Robert Haas, )
                 Re: [HACKERS] pg_dump and thousands of schemas  (Bruce Momjian, )
      Re: pg_dump and thousands of schemas  ("Hugo <Nabble>", )
       Re: pg_dump and thousands of schemas  (Jeff Janes, )
        Re: pg_dump and thousands of schemas  (Tom Lane, )
         Re: pg_dump and thousands of schemas  ("Hugo <Nabble>", )
    Re: pg_dump and thousands of schemas  (Jeff Janes, )
     Re: pg_dump and thousands of schemas  (Tom Lane, )
      Re: pg_dump and thousands of schemas  (Jeff Janes, )
     Re: pg_dump and thousands of schemas  (Tom Lane, )
      Re: pg_dump and thousands of schemas  (Jeff Janes, )

Jeff Janes <> writes:
> There is a quadratic behavior in pg_dump's "mark_create_done".  This
> should probably be fixed, but in the mean time it can be circumvented
> by using -Fc rather than -Fp for the dump format.  Doing that removed
> 17 minutes from the run time.

Hmm, that would just amount to postponing the work from pg_dump to
pg_restore --- although I suppose it could be a win if the dump is for
backup purposes and you probably won't ever have to restore it.
inhibit_data_for_failed_table() has the same issue, though perhaps it's
less likely to be exercised; and there is a previously noted O(N^2)
behavior for the loop around repoint_table_dependencies.

We could fix these things by setting up index arrays that map dump ID
to TocEntry pointer and dump ID of a table to dump ID of its TABLE DATA
TocEntry.  The first of these already exists (tocsByDumpId) but is
currently built only if doing parallel restore.  We'd have to build it
all the time to use it for fixing mark_create_done.  Still, the extra
space is small compared to the size of the TocEntry data structures,
so I don't see that that's a serious objection.

I have nothing else to do right now so am a bit tempted to go fix this.

> I'm working on a patch to reduce the LockReassignCurrentOwner problem
> in the server when using pg_dump with lots of objects.

Cool.

            regards, tom lane


В списке pgsql-performance по дате сообщения:

От: Tom Lane
Дата:
Сообщение: Re: pg_dump and thousands of schemas
От: "Hugo "
Дата:
Сообщение: Re: pg_dump and thousands of schemas