Use read streams in CREATE DATABASE command when the strategy is wal_log

Hi,

I am working on using read streams in the CREATE DATABASE command when the strategy is wal_log. RelationCopyStorageUsingBuffer() function is used in this context. This function reads source buffers then copies them to the destination buffers. I used read streams only when reading source buffers because the destination buffers are read by 'RBM_ZERO_AND_LOCK' option, so it is not important.

I created a ~6 GB table [1] and created a new database with the wal_log strategy using the database that table was created in as a template [2]. My benchmarking results are:

a. Timings:

patched:
12955.027 ms
12917.475 ms
13177.846 ms
12971.308 ms
13059.985 ms

master:
13156.375 ms
13054.071 ms
13151.607 ms
13152.633 ms
13160.538 ms

There is no difference in timings, the patched version is a tiny bit better but it is negligible. I actually expected the patched version to be better because there was no prefetching before, but the read stream API detects sequential access and disables prefetching.

b. strace:

patched:
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 68.02    3.749359           2   1285782           pwrite64
 18.54    1.021734          21     46730           preadv
  9.49    0.522889         826       633           fdatasync
  2.55    0.140339          59      2368           pwritev
  1.14    0.062583         409       153           fsync

master:
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 59.71    3.831542           2   1288365           pwrite64
 29.84    1.914540           2    747936           pread64
  7.90    0.506843         837       605           fdatasync
  1.58    0.101575          54      1856           pwritev
  0.75    0.048431         400       121           fsync

There are fewer (~1/16) read system calls in the patched version.

c. perf:

patched:

-   97.83%     1.13%  postgres  postgres           [.] RelationCopyStorageUsingBuffer                                                    
   - 97.83% RelationCopyStorageUsingBuffer                                                                                                
      - 44.28% ReadBufferWithoutRelcache                                                                                                  
         + 42.20% GetVictimBuffer                                                                                                        
           0.81% ZeroBuffer                                                                                                              
      + 31.86% log_newpage_buffer                                                                                                        
      - 19.51% read_stream_next_buffer                                                                                                    
         - 17.92% WaitReadBuffers                                                                                                        
            + 17.61% mdreadv                                                                                                              
         - 1.47% read_stream_start_pending_read                                                                                          
            + 1.46% StartReadBuffers      

master:

-   97.68%     0.57%  postgres  postgres           [.] RelationCopyStorageUsingBuffer                                                    
   - RelationCopyStorageUsingBuffer                                                                                                      
      - 65.48% ReadBufferWithoutRelcache                                                                                                  
         + 41.16% GetVictimBuffer                                                                                                        
         - 20.42% WaitReadBuffers                                                                                                        
            + 19.90% mdreadv                                                                                                              
         + 1.85% StartReadBuffer                                                                                                          
           0.75% ZeroBuffer                                                                                                              
      + 30.82% log_newpage_buffer

Patched version spends less CPU time in read calls and more CPU time in other calls such as write.

There are three patch files attached. First two are optimization and adding a way to create a read stream object by using SMgrRelation, these are already proposed in the streaming I/O thread [3]. The third one is the actual patch file.

Any kind of feedback would be appreciated.

[1] CREATE TABLE t as select repeat('a', 100) || i || repeat('b', 500) as filler from generate_series(1, 9000000) as i;
[2] CREATE DATABASE test_1 STRATEGY 'wal_log' TEMPLATE test;
[3] https://www.postgresql.org/message-id/CAN55FZ1yGvCzCW_aufu83VimdEYHbG_zuOY3J9JL-nBptyJyKA%40mail.gmail.com

--
Regards,
Nazir Bilal Yavuz
Microsoft