RE: Force streaming every change in logical decoding
От | shiy.fnst@fujitsu.com |
---|---|
Тема | RE: Force streaming every change in logical decoding |
Дата | |
Msg-id | OSZPR01MB6310DFBDC1B254A00AF5DC4BFDFB9@OSZPR01MB6310.jpnprd01.prod.outlook.com обсуждение исходный текст |
Ответ на | Re: Force streaming every change in logical decoding (Masahiko Sawada <sawada.mshk@gmail.com>) |
Список | pgsql-hackers |
On Thu, Dec 22, 2022 3:17 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote: > > I think > > instead of adding new tests with this patch, it may be better to > > change some of the existing tests related to streaming to use this > > parameter as that will clearly show one of the purposes of this patch. > > +1. I think test_decoding/sql/stream.sql and spill.sql are good > candidates and we change logical replication TAP tests in a separate > patch. > Hi, I tried to reduce the data size in test_decoding tests by using the new added GUC "logical_decoding_mode", and found that the new GUC is not suitable for some cases. For example, in the following cases in stream.sql (and there are some similar cases in twophase_stream.sql), the changes in sub transaction exceed logical_decoding_work_mem, but they won't be streamed because the it is rolled back. (But the transaction is marked as streamed.) After the sub transaction, there is a small amount of inserts, as logical_decoding_work_mem is not exceeded, it will be streamed when decoding COMMIT. If we use logical_decoding_mode=immediate, the small amount of inserts in toplevel transaction will be streamed immediately. This is different from before, so I'm not sure we can use logical_decoding_mode in this case. ``` -- streaming test with sub-transaction BEGIN; savepoint s1; SELECT 'msg5' FROM pg_logical_emit_message(true, 'test', repeat('a', 50)); INSERT INTO stream_test SELECT repeat('a', 2000) || g.i FROM generate_series(1, 35) g(i); TRUNCATE table stream_test; rollback to s1; INSERT INTO stream_test SELECT repeat('a', 10) || g.i FROM generate_series(1, 20) g(i); COMMIT; ``` Besides, some cases in spill.sql can't use the new GUC because they want to serialize when processing the specific toplevel transaction or sub transaction. For example, in the case below, the changes in the subxact exceed logical_decoding_work_mem and are serialized, and the insert after subxact is not serialized. If logical_decoding_mode is used, both of them will be serialized. This is not what we want to test. ``` -- spilling subxact, non-spilling main xact BEGIN; SAVEPOINT s; INSERT INTO spill_test SELECT 'serialize-subbig-topsmall--1:'||g.i FROM generate_series(1, 5000) g(i); RELEASE SAVEPOINT s; INSERT INTO spill_test SELECT 'serialize-subbig-topsmall--2:'||g.i FROM generate_series(5001, 5001) g(i); COMMIT; ``` Although the rest cases in spill.sql can use new GUC, but it needs set and reset logical_decoding_mode many times. Besides, the tests in this file were added before logical_decoding_work_mem was introduced, I am not sure if we can modify these cases by using the new GUC. Also, it looks the tests for toast in stream.sql can't be changed, too. Due to the above reasons, it seems that currently few tests can be modified to use logical_decoding_mode. If later we find some tests can changed to use it, we can do it in a separate thread. I will close the CF entry. Regards, Shi yu
В списке pgsql-hackers по дате отправления:
Следующее
От: Bharath RupireddyДата:
Сообщение: Re: Add a new pg_walinspect function to extract FPIs from WAL records