Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

Список
Период
Сортировка
От Anastasia Lubennikova
Тема Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits
Дата
Msg-id 75f3760f-1077-07c1-aea4-975d5c127d6d@postgrespro.ru
обсуждение исходный текст
Ответ на Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Tomas Vondra)
Ответы Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Tomas Vondra)
Список pgsql-hackers
Дерево обсуждения
COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Pavan Deolasee, )
 Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Kuntal Ghosh, )
  Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Simon Riggs, )
   Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Kuntal Ghosh, )
 Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Jeff Janes, )
  Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Pavan Deolasee, )
 Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Masahiko Sawada, )
  Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Pavan Deolasee, )
   Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Masahiko Sawada, )
    Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Pavan Deolasee, )
     Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Masahiko Sawada, )
      Re: Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (David Steele, )
      Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Pavan Deolasee, )
       Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Darafei Praliaskouski, )
       Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Masahiko Sawada, )
        Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Pavan Deolasee, )
 Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Pavan Deolasee, )
  Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Tomas Vondra, )
  Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Andres Freund, )
   Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Alvaro Herrera, )
    Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Andres Freund, )
     Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Andres Freund, )
      Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Alvaro Herrera, )
       Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Andres Freund, )
        Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Pavan Deolasee, )
       Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Andres Freund, )
        Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Pavan Deolasee, )
         Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Andres Freund, )
        Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Tom Lane, )
         Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Andres Freund, )
          Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Andres Freund, )
           Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Andres Freund, )
            Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Pavan Deolasee, )
             Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Pavan Deolasee, )
              Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Amit Kapila, )
               Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Ibrar Ahmed, )
         Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Darafei "Komяpa" Praliaskouski, )
          Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Andres Freund, )
       Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Ibrar Ahmed, )
        Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Ibrar Ahmed, )
         Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Daniel Gustafsson, )
          Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Anastasia Lubennikova, )
           Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Robert Haas, )
            Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Anastasia Lubennikova, )
             Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Ibrar Ahmed, )
              Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Hamid Akhtar, )
               Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Ibrar Ahmed, )
              Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Alvaro Herrera, )
               Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Anastasia Lubennikova, )
                Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Ibrar Ahmed, )
                 Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Anastasia Lubennikova, )
                  Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Ibrar Ahmed, )
                   Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Anastasia Lubennikova, )
                    Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Tatsuo Ishii, )
                     Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Tomas Vondra, )
                      Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Anastasia Lubennikova, )
 Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Tomas Vondra, )
  Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Anastasia Lubennikova, )
   Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Tomas Vondra, )
    Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Anastasia Lubennikova, )
     Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Tomas Vondra, )
      Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Anastasia Lubennikova, )
       Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Tomas Vondra, )
        Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Tomas Vondra, )
         Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Tatsuo Ishii, )
         Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Pavan Deolasee, )
          Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Tom Lane, )
           Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Tomas Vondra, )
            Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits  (Tomas Vondra, )
On 11.01.2021 01:35, Tomas Vondra wrote:
> Hi,
>
> I started looking at this patch again, hoping to get it committed in 
> this CF, but I think there's a regression in handling TOAST tables 
> (compared to the v3 patch submitted by Pavan in February 2019).
>
> The test I'm running a very simple test (see test.sql):
>
> 1) start a transaction
> 2) create a table with a text column
> 3) copy freeze data into it
> 4) use pg_visibility to see how many blocks are all_visible both in the
>    main table and it's TOAST table
>
> For v3 patch (applied on top of 278584b526 and 
> s/hi_options/ti_options) I get this:
>
>            pages           NOT all_visible
>   ------------------------------------------
>   main       637                         0
>   toast    50001                         3
>
> There was some discussion about relcache invalidations causing a 
> couple TOAST pages not be marked as all_visible, etc.
>
> However, for this patch on master I get this
>
>            pages           NOT all_visible
>   ------------------------------------------
>   main       637                         0
>   toast    50001                     50001
>
> So no pages in TOAST are marked as all_visible. I haven't investigated 
> what's causing this, but IMO that needs fixing to make ths patch RFC.
>
> Attached is the test script I'm using, and a v5 of the patch - rebased 
> on current master, with some minor tweaks to comments etc.
>

Thank you for attaching the test script. I reproduced the problem. This 
regression occurs because TOAST internally uses heap_insert().
You have asked upthread about adding this optimization to heap_insert().

I wrote a quick fix, see the attached patch 0002. The TOAST test passes 
now, but I haven't tested performance or any other use-cases yet.
I'm going to test it properly in a couple of days and share results.

With this change a lot of new code is repeated in heap_insert() and 
heap_multi_insert(). I think it's fine, because these functions already 
have a lot in common.

-- 
Anastasia Lubennikova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company


Вложения

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Tom Lane
Дата:
Сообщение: Re: pg_dump, ATTACH, and independently restorable child partitions
Следующее
От: "tsunakawa.takay@fujitsu.com"
Дата:
Сообщение: RE: POC: postgres_fdw insert batching