Re: Map WAL segment files on PMEM as WAL buffers

Поиск
Список
Период
Сортировка
От Takashi Menjo
Тема Re: Map WAL segment files on PMEM as WAL buffers
Дата
Msg-id CAOwnP3PSDxKhBUGS=97_b44s65DxaGexJB2-Nbmf7SGiLUJ2pg@mail.gmail.com
обсуждение исходный текст
Ответ на Re: Map WAL segment files on PMEM as WAL buffers  (Justin Pryzby <pryzby@telsasoft.com>)
Ответы Re: Map WAL segment files on PMEM as WAL buffers  (Justin Pryzby <pryzby@telsasoft.com>)
Список pgsql-hackers
Hi Justin,

Thank you for your build test and comments. The v7 patchset attached
to this email fixes the issues you reported.


> The cfbot showed issues compiling on linux and windows.
> http://cfbot.cputube.org/takashi-menjo.html
>
> https://cirrus-ci.com/task/6125740327436288
> [02:30:06.538] In file included from xlog.c:38:
> [02:30:06.538] ../../../../src/include/access/xlogpmem.h:32:42: error: unknown type name ‘tli’
> [02:30:06.538]    32 | PmemXLogEnsurePrevMapped(XLogRecPtr ptr, tli)
> [02:30:06.538]       |                                          ^~~
> [02:30:06.538] xlog.c: In function ‘GetXLogBuffer’:
> [02:30:06.538] xlog.c:1959:19: warning: implicit declaration of function ‘PmemXLogEnsurePrevMapped’
[-Wimplicit-function-declaration]
> [02:30:06.538]  1959 |    openLogSegNo = PmemXLogEnsurePrevMapped(endptr, tli);
>
> https://cirrus-ci.com/task/6688690280857600?logs=build#L379
> [02:33:25.752] c:\cirrus\src\include\access\xlogpmem.h(33,1): error C2081: 'tli': name in formal parameter list
illegal(compiling source file src/backend/access/transam/xlog.c) [c:\cirrus\postgres.vcxproj] 
>
> I'm attaching a probable fix.  Unfortunately, for patches like this, most of
> the functionality isn't exercised unless the library is installed and
> compilation and runtime are enabled by default.

I got the same error when without --with-libpmem. Your fix looks
reasonable. My v7-0008 fixes this error.


> In 0009: recaluculated => recalculated

v7-0011 fixes this typo.


> 0010-Update-document should be squished with 0003-Add-wal_pmem_map-to-GUC (and
> maybe 0002 and 0001).  I believe the patches after 0005 are more WIP, so it's
> fine if they're not squished yet.

As you say, the patch updating document should melt into a related
fix, probably "Add wal_pmem_map to GUC".  For now I want it to be a
separate patch (v7-0014).


> I'm not sure what the point is of this one: 0008-Let-wal_pmem_map-be-constant-unl

If USE_LIBPMEM is not defined (that is, no --with-libpmem),
wal_pmem_map is always false and is never used essentially. Using
#if(n)def everywhere is not good for code readability, so I let
wal_pmem_map be constant. This may help compilers optimize conditional
branches.

v7-0005 adds the comment above.


> +               ereport(ERROR,
> +                               (errcode_for_file_access(),
> +                                errmsg("could not pmem_map_file \"%s\": %m", path)));
>
> => The outer parenthesis are not needed since e3a87b4.

v7-0009 fixes this.


> But in this case it really doesn't work :(
>
> running bootstrap script ... 2022-01-05 23:17:30.244 CST [12088] FATAL:  file not on PMEM: path
"pg_wal/000000010000000000000001"

Do you have a real PMEM device such as NVDIMM-N or Intel Optane PMem?
If so, please use a PMEM mounted with Filesystem DAX option for
pg_wal, or the FATAL error will occur.

If you don't, you have two alternatives below. Note that neither of
them ensures durability. Each of them is just for testing.

1. Emulate PMEM with memmap=nn[KMG]!ss[KMG]. This can be used only on
Linux. Please see [1][2] for details; or
2. Set the environment variable PMEM_IS_PMEM_FORCE=1 to tell libpmem
to treat any devices as if they were PMEM.


Regards,
Takashi


[1]
https://www.intel.com/content/www/us/en/developer/articles/training/how-to-emulate-persistent-memory-on-an-intel-architecture-server.html
[2] https://nvdimm.wiki.kernel.org/

--
Takashi Menjo <takashi.menjo@gmail.com>

Вложения

В списке pgsql-hackers по дате отправления:

Предыдущее
От: "houzj.fnst@fujitsu.com"
Дата:
Сообщение: RE: row filtering for logical replication
Следующее
От: Japin Li
Дата:
Сообщение: Re: Support tab completion for upper character inputs in psql