Re: Initializing LWLock Array from Server Code

Поиск
Список
Период
Сортировка
От Souvik Bhattacherjee
Тема Re: Initializing LWLock Array from Server Code
Дата
Msg-id CAANrPSdEu-bkuxOypPvNcWD-7ASa4L-s-yMAeJ3KqfJGHJKOvw@mail.gmail.com
обсуждение исходный текст
Ответ на Initializing LWLock Array from Server Code  (Souvik Bhattacherjee <kivuosb@gmail.com>)
Ответы Re: Initializing LWLock Array from Server Code  (Robert Haas <robertmhaas@gmail.com>)
Список pgsql-hackers
Hi Robert,

Thank you for your reply and sorry that I couldn't reply earlier.
Since I didn't get any response within a couple of days, I took the longer route -- changed the lwlock.h and lwlock.c
for accommodating the lw locks for the shared hash table.

I'll describe what I modified in the lwlock.h and lwlock.c and it seems to be working fine. Although I haven't got an opportunity
to test it extensively. If you could let me know if I missed out anything that might cause problems later that would be great.

In lwlock.h, the following modifications were made:

/* the number of partitions of the shared hash table */
#define NUM_FILTERHASH_PARTITIONS 64

/* Offsets for various chunks of preallocated lwlocks. */
#define BUFFER_MAPPING_LWLOCK_OFFSET    NUM_INDIVIDUAL_LWLOCKS
#define LOCK_MANAGER_LWLOCK_OFFSET        \
    (BUFFER_MAPPING_LWLOCK_OFFSET + NUM_BUFFER_PARTITIONS)
#define PREDICATELOCK_MANAGER_LWLOCK_OFFSET \
    (LOCK_MANAGER_LWLOCK_OFFSET + NUM_LOCK_PARTITIONS)

/* offset for shared filterhash lwlocks in the MainLWLockArray */
#define FILTERHASH_OFFSET    \
    (PREDICATELOCK_MANAGER_LWLOCK_OFFSET + NUM_PREDICATELOCK_PARTITIONS)
/* modified NUM_FIXED_LOCKS */
#define NUM_FIXED_LWLOCKS    \
    (FILTERHASH_OFFSET + NUM_FILTERHASH_PARTITIONS)

/* added LWTRANCHE_FILTERHASH in the BuiltinTrancheIds */
typedef enum BuiltinTrancheIds
{
    LWTRANCHE_CLOG_BUFFERS = NUM_INDIVIDUAL_LWLOCKS,
    LWTRANCHE_COMMITTS_BUFFERS,
    LWTRANCHE_SUBTRANS_BUFFERS,
    LWTRANCHE_MXACTOFFSET_BUFFERS,
    LWTRANCHE_MXACTMEMBER_BUFFERS,
    LWTRANCHE_ASYNC_BUFFERS,
    LWTRANCHE_OLDSERXID_BUFFERS,
    LWTRANCHE_WAL_INSERT,
    LWTRANCHE_BUFFER_CONTENT,
    LWTRANCHE_BUFFER_IO_IN_PROGRESS,
    LWTRANCHE_REPLICATION_ORIGIN,
    LWTRANCHE_REPLICATION_SLOT_IO_IN_PROGRESS,
    LWTRANCHE_PROC,
    LWTRANCHE_BUFFER_MAPPING,
    LWTRANCHE_LOCK_MANAGER,
    LWTRANCHE_PREDICATE_LOCK_MANAGER,
    LWTRANCHE_FILTERHASH,
    LWTRANCHE_PARALLEL_HASH_JOIN,
    LWTRANCHE_PARALLEL_QUERY_DSA,
    LWTRANCHE_SESSION_DSA,
    LWTRANCHE_SESSION_RECORD_TABLE,
    LWTRANCHE_SESSION_TYPMOD_TABLE,
    LWTRANCHE_SHARED_TUPLESTORE,
    LWTRANCHE_TBM,
    LWTRANCHE_PARALLEL_APPEND,
    LWTRANCHE_FIRST_USER_DEFINED
}            BuiltinTrancheIds;


In lwlock.c, the following modifications were made:

In function, static void InitializeLWLocks(void), the following lines were added:

/* Initialize filterhash LWLocks in main array */
    lock = MainLWLockArray + NUM_INDIVIDUAL_LWLOCKS +
        NUM_BUFFER_PARTITIONS + NUM_LOCK_PARTITIONS + NUM_FILTERHASH_PARTITIONS;
    for (id = 0; id < NUM_FILTERHASH_PARTITIONS; id++, lock++)
        LWLockInitialize(&lock->lock, LWTRANCHE_FILTERHASH);

In function void RegisterLWLockTranches(void), the following line was added:

LWLockRegisterTranche(LWTRANCHE_FILTERHASH, "filter_hash");

All of this was done after allocating the required of space in the shared memory.

Thanks,
-SB


On Mon, Apr 29, 2019 at 1:59 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Apr 26, 2019 at 2:58 PM Souvik Bhattacherjee <kivuosb@gmail.com> wrote:
> I have created a shared hash table in partitioned mode inside the postgres server code. In order to guard the partitions, I'm trying to initialize an array of LWLocks. The code that I'm trying to use for that is
>
> void RequestNamedLWLockTranche(const char *tranche_name, int num_lwlocks);
> LWLockPadded *GetNamedLWLockTranche(const char *tranche_name);
>
> I'm not sure where exactly should this code be called from the server code. So I had placed it in
>
> void CreateSharedMemoryAndSemaphores(bool makePrivate, int port);
>
> within ipic.c. However, I'm getting the following error message when starting the database:
>
> FATAL:  requested tranche is not registered
>
> So at this point, I'm a little confused as to where the methods should be called from inside the server code. Any pointers would be appreciated.

RequestNamedLWLockTranche() changes the behavior of LWLockShmemSize()
and CreateLWLocks(), so must be called before either of those.
GetNamedLWLockTranche() must be called after CreateLWLocks().

This machinery works for pg_stat_statements, so see that as an example
of how to make this work from an extension.  If you're hacking the
core server code, then look at the places where the corresponding bits
of pg_stat_statements code get called.  IIRC, _PG_init() gets called
from process_shared_preload_libraries(), so you might look at the
placement of the calls to that function.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Tom Lane
Дата:
Сообщение: Re: Calling PrepareTempTablespaces in BufFileCreateTemp
Следующее
От: Andres Freund
Дата:
Сообщение: Re: REINDEX INDEX results in a crash for an index of pg_class since9.6