RE: Copy data to DSA area
От | Ideriha, Takeshi |
---|---|
Тема | RE: Copy data to DSA area |
Дата | |
Msg-id | 4E72940DA2BF16479384A86D54D0988A7DB51DBD@G01JPEXMBKW04 обсуждение исходный текст |
Ответ на | RE: Copy data to DSA area ("Ideriha, Takeshi" <ideriha.takeshi@jp.fujitsu.com>) |
Список | pgsql-hackers |
Hi, >>From: Thomas Munro [mailto:thomas.munro@gmail.com] >>Subject: Re: Copy data to DSA area >> >>On Wed, May 8, 2019 at 5:29 PM Ideriha, Takeshi >><ideriha.takeshi@jp.fujitsu.com> >>wrote: >>> >From: Ideriha, Takeshi [mailto:ideriha.takeshi@jp.fujitsu.com] >>> >Sent: Friday, April 26, 2019 11:50 PM Well, after developing PoC, I >>> >realized that this PoC doesn't solve the local process is crashed >>> >before the context becomes shared because local process keeps track >>> >of pointer to chunks. >>> >Maybe all of you have already noticed and pointed out this case :) >>> >So it needs another work but this poc is a good step for me to advance more. >>> >>> I think the point to prevent memory leak is allocating memory and >>> storing its address into a structure at the same time. This structure >>> should be trackable from other process. >> >>I'm not sure that it's necessarily wrong to keep tracking information in private >memory. >>If any backend crashes, the postmaster will terminate all backends and >>reinitialise everything anyway, because shared memory might be corrupted. > >I'm going to put keep tracking information in private >memory and send a patch. I updated a PoC patch. This has memory tracking buffer in local process. The old version also has this system but I refactored the code. Memory leak while allocating memory seems to be solved thanks to memory tracking buffer. What I haven't addressed is memory leak while freeing objects. In current sequence a cache (e.g. relcache) is freed after removed from its hash table. If cache and hash table gets shared, memory leak is likely to happen between removal from hash table and free. We lose track of cache objects if error happens after cache is unlinked from the hash table. And also a cache consists of graph structure. So we also take care of freeing cache partially. Maybe we need to remember pointers of objects before unlink from the hash. Also, we need to free them all at once after we can make sure that all the pointers are registered to local buffer. Followings are some idea to implement this: - change the order of removal from hash table and deletion - pfree in shared memory context doesn't dsa_free but just add pointer to the local buffer. - remove link from hash table after all pfree() is done - then, call a function, which does actual dsa_free taking a look at the local Buffer But I'm not sure this solution is good one. Do you have any thoughts? Regards, Takeshi Ideriha
Вложения
В списке pgsql-hackers по дате отправления: