Re: [HACKERS] Error: dsa_area could not attach to a segment that hasbeen freed

Поиск
Список
Период
Сортировка
От Craig Ringer
Тема Re: [HACKERS] Error: dsa_area could not attach to a segment that hasbeen freed
Дата
Msg-id CAMsr+YFXusTcm2waoczSr2KF+6MO_5uUsdD1_jR7cgOAJbJU-A@mail.gmail.com
обсуждение исходный текст
Ответ на Re: [HACKERS] Error: dsa_area could not attach to a segment that hasbeen freed  (Thomas Munro <thomas.munro@enterprisedb.com>)
Ответы Re: [HACKERS] Error: dsa_area could not attach to a segment that hasbeen freed  (Craig Ringer <craig@2ndquadrant.com>)
Re: [HACKERS] Error: dsa_area could not attach to a segment that has been freed  (Tom Lane <tgl@sss.pgh.pa.us>)
Список pgsql-hackers
Дерево обсуждения
On 20 September 2017 at 16:55, Thomas Munro <thomas.munro@enterprisedb.com> wrote:
On Wed, Sep 20, 2017 at 6:14 PM, Gaddam Sai Ram
<gaddamsairam.n@zohocorp.com> wrote:
> Thank you very much! That fixed my issue! :)
> I was in an assumption that pinning the area will increase its lifetime but
> yeah after taking memory context into consideration its working fine!

So far the success rate in confusing people who first try to make
long-lived DSA areas and DSM segments is 100%.  Basically, this is all
designed to ensure automatic cleanup of resources in short-lived
scopes.

90% ;)

I got it working with no significant issues for a long lived segment used to store a pool of shm_mq pairs used for a sort of "connection listener" bgworker. Though I only used DSM+ToC, not DSA. But TBH that may well be luck, as I tend to routinely use memory contexts scoped to the operational lifetime of a subsystem, making most problems like this just vanish without my realising they were there in the first place. Usually.

I pretty much shamelessly cribbed from test_shm_mq for the ToC stuff though. It's simple enough when you read it in use, but I'd be lucky to do it without an example.

I had lots more problems with shm_mq than DSM. shm_mq is very obviously designed for short-lived scopes, and falls down badly if you have a pool of queues you want to re-use after the peer detaches. You have to track "in use" flags separately to the shm_mq's own, because it doesn't clear its stored PGPROC entries for receiver/sender on detach. Once you know neither sender nor receiver is still attached, you can memset() the area and create a new queue in it.

You can't just reset the queue for a new peer, and have to do quite a dance to make sure it's safe detach from, overwrite, re-create and re-attach to.
 
Good luck for your graph project.  I think you're going to have to
expend a lot of energy trying to avoid memory leaks if your DSA lives
as long as the database cluster, since error paths won't automatically
free any memory you allocated in it.

Yeah, that's going to be hard. You might land up having lots and lots of little DSM segments.

 
There is a kind of garbage collection for palloc'd memory and
also for other resources like file handles, but if you're using a big
long lived DSA area you have nothing like that.

We need, IMO, a DSA-backed heirachical MemoryContext system.

We can't use the exact MemoryContext API as-is due to the need for far pointers though :(

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Alvaro Herrera
Дата:
Сообщение: Re: [HACKERS] src/test/subscription/t/002_types.pl hanging onparticular environment
Следующее
От: Craig Ringer
Дата:
Сообщение: Re: [HACKERS] Error: dsa_area could not attach to a segment that hasbeen freed