Re: dynamically allocating chunks from shared memory
От | Markus Wanner |
---|---|
Тема | Re: dynamically allocating chunks from shared memory |
Дата | |
Msg-id | 4C45ECC7.7080802@bluegap.ch обсуждение исходный текст |
Ответ на | Re: dynamically allocating chunks from shared memory (Alvaro Herrera <alvherre@commandprompt.com>) |
Ответы |
Re: dynamically allocating chunks from shared memory
(Alvaro Herrera <alvherre@commandprompt.com>)
|
Список | pgsql-hackers |
Hello Alvaro, thank you for looking through this code. On 07/20/2010 07:50 PM, Alvaro Herrera wrote: > Interesting, thanks. > > I gave it a skim and found that it badly needs a lot more code comments. Hm.. yeah, the dynshmem stuff could probably need more comments. (The bgworker stuff is probably a better example). > I'm also unconvinced that spinlocks are the best locking primitive here. > Why not lwlocks? It's derived from a completely lock-free algorithm, as proposed by Maged M. Michael in: Scalable Lock-Free Dynamic Memory Allocator. I dropped all of the CAS primitives with their retry loop around and did further simplifications. Spinlocks simply looked like the simplest thing to fall-back to. But yeah, splitting into read and write accesses and using lwlocks might be a win. Or it might not. I honestly don't know. And it's probably not the best performing allocator ever. But it's certainly better than nothing. I did recently release the lock-free variant as well as a lock based one, see http://www.bluegap.ch/projects/wamalloc/ for more information. > I'm not sure what kind of resistance you'll see to the idea of a > dynamically allocatable shmem area. So far neither resistance nor applause. I'd love to hear more of an echo. Even if it's resistance. > Maybe we could use this in other > areas ..which is why I've published this separately from Postgres-R. > such as allocating space for heavyweight lock objects. Right now > the memory usage for them could grow due to a transitory increase in > lock traffic, leading to out-of-memory conditions later in other > modules. We've seen reports of that problem, so it'd be nice to be able > to fix that with this infrastructure. Maybe, yes. Sounds like a nice idea. > I didn't look at the imessages patch (except to notice that I didn't > very much like the handling of out-of-memory, but you already knew that). As all of the allocation problem has now been ripped out, the imessages patch got quite a bit smaller. imsg.c now consists of only around 370 lines of code. The handling of out-of-(shared)-memory situation could certainly be improved, yes. Note that I've already separated out a IMessageCreateInternal() method, which simply returns NULL in that case. Is that the API you'd prefer? Getting back to the dynshmem stuff: I don't mind much *which* allocator to use. I also looked at jemalloc, but haven't been able to integrate it into Postgres. So I've extended my experiment with wamalloc and turned it into something usable for Postgres. Regards Markus Wanner
В списке pgsql-hackers по дате отправления: