Re: [HACKERS] Regarding Postgres Dynamic Shared Memory (DSA)

Поиск
Список
Период
Сортировка
От Mahendranath Gurram
Тема Re: [HACKERS] Regarding Postgres Dynamic Shared Memory (DSA)
Дата
Msg-id 15cd33d5aa9.d704fa438272.5447729703625904129@zohocorp.com
обсуждение исходный текст
Ответ на Re: [HACKERS] Regarding Postgres Dynamic Shared Memory (DSA)  (Thomas Munro <thomas.munro@enterprisedb.com>)
Список pgsql-hackers
Hi Thomas,

When you create or attach to a DSA area, a detach callback is
automatically registered on the control segment (or containing
segment, for an in-place DSA area). See the code like this in dsa.c:

/* Clean up when the control segment detaches. */
on_dsm_detach(segment, &dsa_on_dsm_detach_release_in_place,

PointerGetDatum(dsm_segment_address(segment)));

So the reference counting is automatically looked after and you don't
need to do anything. There are four possibilities: (1) you explicitly
detach from the area, (2) the ResourceManager active when you created
or attached goes out of scope, detaching you automatically (you
probably want to disable that with dsa_pin_mapping(area)), (3) your
backend exits normally and it's automatically detached (4) you crash
and the postmaster restarts the whole cluster on the basis that shared
memory could be arbitrarily corrupted.
The third possibility is our use case. We want the dsa_area to be alive through out the lifetime of the cluster and we want each backend to hold the attached dsa through out the lifetime of the backend irrespective of ResourceManager scope.

Hence we are calling dsa_pin(area) after dsa_create() and dsa_pin_mapping(area) after dsa_create() and dsa_attach().
DSA implementation in autovacuum.c helped me in understanding these concepts :)

So in my case, i could safely assume that postgres will automatically takes care of dsa detaching and reference count decrements during backend exit.

It's so nice of you, taking time and explaining and helping to solve the use cases.

Best Regards,
-Mahi
Teamwork divides the task and multiplies the success.




---- On Thu, 22 Jun 2017 17:08:01 +0530 Thomas Munro <thomas.munro@enterprisedb.com> wrote ----

On Thu, Jun 22, 2017 at 10:59 PM, Mahendranath Gurram
> I'm implementing the In-Memory index as per your suggestion. So far it's
> good.

Great news.

> As of now, only one thing is unclear for me. How could i detach the
> dsa(dsa_detach() call) in backend (typically during backend quit).
>
> Of-course when process quits, all it's associated memory will be
> cleared/destroyed. But we have to decrement dsm reference counts as part of
> the potgres dsa framework implementation. which can not be done now.
>
> I have gone through the postgres source code. But unfortunately, i couldn't
> see anything in handling this case.

When you create or attach to a DSA area, a detach callback is
automatically registered on the control segment (or containing
segment, for an in-place DSA area). See the code like this in dsa.c:

/* Clean up when the control segment detaches. */
on_dsm_detach(segment, &dsa_on_dsm_detach_release_in_place,

PointerGetDatum(dsm_segment_address(segment)));

So the reference counting is automatically looked after and you don't
need to do anything. There are four possibilities: (1) you explicitly
detach from the area, (2) the ResourceManager active when you created
or attached goes out of scope, detaching you automatically (you
probably want to disable that with dsa_pin_mapping(area)), (3) your
backend exits normally and it's automatically detached (4) you crash
and the postmaster restarts the whole cluster on the basis that shared
memory could be arbitrarily corrupted.

Note that if you call dsa_pin(area) after creating the DSA area the
reference count cannot reach zero until you either call
dsa_unpin(area) or the cluster exits. That's the right thing to do
for a case where backends might come and go and it's possible for no
one to be attached for periods of time, but you want the DSA area to
continue to exist. I think that's probably what you need for a
DSA-backed in-memory index.

--
Thomas Munro


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Tom Lane
Дата:
Сообщение: Re: [HACKERS] Guarding against bugs-of-omission in initdb's setup_depend
Следующее
От: Yugo Nagata
Дата:
Сообщение: Re: [HACKERS] [POC] hash partitioning