Re: [HACKERS] Supporting huge pages on Windows

On Tue, Jan 10, 2017 at 8:49 AM, Tsunakawa, Takayuki
<tsunakawa.takay@jp.fujitsu.com> wrote:
> Hello, Amit, Magnus,
>
> I'm sorry for my late reply.  Yesterday was a national holiday in Japan.
>
>
> From: pgsql-hackers-owner@postgresql.org
>> [mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Amit Kapila
>> PGSharedMemoryReAttach is called after the startup of new process whereas
>> pgwin32_ReserveSharedMemoryRegion is called before the new process could
>> actually start.  Basically, pgwin32_ReserveSharedMemoryRegion is used to
>> reserve shared memory for each backend, so calling VirtualAlloc there should
>> follow spec for huge pages.  If you have some reason for not using, then
>> it is not clear from your reply, can you try to explain in detail.
>
> OK.  The processing takes place in the following order:
>
> 1. postmaster calls CreateProcess() to start a child postgres in a suspended state.
> 2. postmaster calls VirtualAlloc(MEM_RESERVE) in pgwin32_ReserveSharedMemoryRegion() to reserve the virtual address
spacein the child to secure space for the shared memory.  This call just affects the virtual address space and does not
allocatephysical memory.  So the large page is still irrelevant. 

Okay, today again reading VirtualAlloc specs, I could see that
MEM_LARGE_PAGES is not not required to reserve the memory region.  It
is only required during allocation.


>
> I succeeded by following the same procedure using secpol.msc on Win10, running 64-bit PostgreSQL.  I started
PostgreSQLas a Windows service because it's the normal way, with the service account being a regular Windows user
account(i.e.not LocalSystem). 
>
> But... I failed to start PostgreSQL by running "pg_ctl start" from a command prompt, receiving the same error message
asyou.  On the other hand, I could start PostgreSQL on a command prompt with administrative privileges (right-click on
"Commandprompt" from the Start menu and select "Run as an administrator" in the menu. 


Hmm.  It doesn't work even on a command prompt with administrative
privileges. It gives below error:

waiting for server to start....2017-01-17 11:20:13.780 IST [4788] FATAL:  could
not create shared memory segment: error code 1450
2017-01-17 11:20:13.780 IST [4788] DETAIL:  Failed system call was CreateFileMap
ping(size=148897792, name=Global/PostgreSQL:E:/WorkSpace/PostgreSQL/master/Data)
.
2017-01-17 11:20:13.780 IST [4788] LOG:  database system is shut downstopped waiting
pg_ctl: could not start server
Examine the log output.


Now, error code 1450 can occur due to insufficient system resources,
so I have tried by increasing the size of shared memory (higher value
of shared_buffers) without your patch and it works.  This indicates
some problem with the patch.

>  It seems that Windows removes many privileges, including "Lock Pages in Memory", when starting the normal command
prompt. As its evidence, you can use the attached priv.c to see what privileges are assigned and and enabled/disabled.
Buildit like "cl priv.c" and just run priv.exe on each command prompt.  Those runs show different privileges. 
>

This is bad.

> Should I need to do something, e.g. explain in the document that the user should use the command prompt with
administrativeprivileges when he uses huge_pages? 
>

I think it is better to document in some way if we decide to go-ahead
with the patch.


--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com