On Tue, 27 May 2003, Andrew Sullivan wrote:
> On Tue, May 27, 2003 at 12:07:06PM -0600, scott.marlowe wrote:
> > I wonder if that's a performance issue with Solaris and shared memory that
> > isn't a problem with BSD or Linux on Sparc Hardware?
>
> I wonder that, also. I haven't had a chance to try it out.
>
> The problem in our case is that we couldn't use Linux or BSD anyway,
> because we need the 8- and 10-way scalability we have, and we need
> the cleverness about disabling processors, &c., that's built into
> Solaris. So even if Solaris is the problem, we're stuck.
>
> The file system buffers, on the other hand, are pretty fast, so it's
> not too big a penalty.
>
> One thing that is very interesting about all of this is that the
> large shared buffers only exact their performance penalty over time.
> My hypothesis is that it has something to do with expiring buffers.
> In one test I performed, I set the buffer to 1G. I then did a bunch
> of work on a data set that was close to 1G. Speedy. But when I
> finally went over 1G, everything slowed to a crawl. This makes me
> believe that the problem is in the way records are added to or
> expired from the buffer.
>
> It was only one test, mind: I didn't have time to repeat it. So it's
> just a bit of gossip and not a result.
Thanks for the gossip. I tested Postgresql with ~1 gig of shared space
under RH 7.2 and found that while it WAS slower than say with 512 Meg of
ram, it was only slower when you weren't using more than 512Meg. If I was
doing something that was larger than 512Meg and smaller than 1 Gig, using
1 gig shared was still faster, but not by much. It didn't seem to slow
down a lot going over the max shared_buffers memory, but we don't have any
really huge datasets, so I didn't test all that thourougly at that
setting, as the performance gain wasn't all that great for large sets, and
noticeably slower for medium size datasets.
I just think SYSV shared memory isn't built for handling large amounts of
memory.
I could stand out here at the fence and chat all day... :-)