Re: Spinlock performance improvement proposal

On Wed, 26 Sep 2001, Alex Pilosov wrote:

> On Wed, 26 Sep 2001, D. Hageman wrote:
> 
> > When you need data that is specific to a thread you use a TSD (Thread 
> > Specific Data).  

> Which Linux does not support with a vengeance, to my knowledge.

I am not sure what that means.  If it works it works. 

> As a matter of fact, quote from Linus on the matter was something like
> "Solution to slow process switching is fast process switching, not another
> kernel abstraction [referring to threads and TSD]". TSDs make
> implementation of thread switching complex, and fork() complex.

Linus does have some interesting ideas.  I always like to hear his 
perspective on matters, but just like the government - I don't always 
agree with him.  I don't see why TSDs would make the implementation of 
thread switching complex - seems to me that would be something that is 
implemented in the userland side part of the pthreads implemenation and 
not the kernel side.  I don't really like to talk specifics, but both the 
lightweight process and the system call fork() are implemented using the 
__clone kernel function with the parameters slightly different (This is 
in the Linux kernel, btw since you wanted to use that as an example).  The 
speed improvements the kernel has given the fork() command (like copy on 
write) only lasts until the process writes to memmory.  The next time it 
comes around - it is for all intents and purposes a full context switch 
again.  With threads ... the cost is relatively consistant.

> The question about threads boils down to: Is there far more data that is
> shared than unshared? If yes, threads are better, if not, you'll be
> abusing TSD and slowing things down. 

I think the question about threads boils down to if the core members of 
the PostgreSQL team want to try it or not.  At this time, I would have to 
say they pretty much agree they like things the way they are now, which is 
completely fine.  They are the ones that spend most of the time on it and 
want to support it.

> I believe right now, postgresql' model of sharing only things that need to
> be shared is pretty damn good. The only slight problem is overhead of
> forking another backend, but its still _fast_.

Oh, man ... am I reading stuff into what you are writing or are you 
reading stuff into what I am writing?  Maybe a little bit of both?  My 
original contention is that I think that the best way to get the full 
potential out of SMP machines is to use a threads model.  I didn't say the 
present way wasn't fast.  

>  Actually, if I remember, there was someone who ported postgresql (I think
> it was 6.5) to be multithreaded with major pain, because the requirement
> was to integrate with CORBA. I believe that person posted some benchmarks
> which were essentially identical to non-threaded postgres...

Actually, it was 7.0.2 and the performance gain was interesting.  The 
posting can be found at:

http://candle.pha.pa.us/mhonarc/todo.detail/thread/msg00007.html

The results are:

20 clients, 900 inserts per client, 1 insert per transaction, 4 different
tables.

7.0.2    About    10:52 average completion
multi-threaded    2:42 average completion
7.1beta3          1:13 average completion

If the multi-threaded version was 7.0.2 and threads increased performance 
that much - I would have to say that was a bonus.  However, the 
performance increases that the PostgreSQL team implemented later ... 
pushed the regular version ahead again.  That kinda says to me that 
potential is there.

If you look at Myron Scott's post today you will see that it had other 
advantages going for it (like auto-vacuum!) and disadvantages ... rogue 
thread corruption (already debated today).

-- 
//========================================================\\
||  D. Hageman                    <dhageman@dracken.com>  ||
\\========================================================//