Re: Linux max on shared buffers?

On Thu, 11 Jul 2002, Glen Parker wrote:

> When are PG's buffers *not* in series with the OS's?

Never. That's the problem.

> [Disk] => <read> => [OS cache] => <mem copy> => [PG cache]
>
> And when its in only the OS cache, you get:
> [OS cache] => <mem copy> => [PG cache]

Right.

> The bigger your PG cache, the less often you have to ask the OS for a
> page.  That <mem copy> (plus syscall overhead) that happens between OS
> cache and PG cache is expensive compared to not having to do it.  Not
> horribly so, I realize, but every little bit helps...

Yes, but it's much, much cheaper than reading a page from disk.

> Unless I'm just completely missing something here, the bigger the PG
> cache the better, within reason.

Let's walk through an example. I have four pages total for caching.
Let's look at a read scenario based on two for postgres and two for the
OS, and one for postgres and three for the OS. Pn is a postgres buffer
and OSn is an OS buffer; the numbers below those show which disk blocks
are in which caches. We'll use an LRU algorithm for both caches and read
the blocks in this order: 1 2 3 2 1 2 3.

    OS1 OS2 P1  P2    Action
    -   -   -   -    Start with all caches empty.
    1   -   1   -       Read block 1 (replaces -/-). disk + memory copy.
    1   2   1   2    Read block 2 (replaces -/-). disk + memory copy.
    3   2   3   2    Read block 3 (replaces 1/1). disk + memory copy.
    3   2   3   2    Read block 2 (in cache). no copies.
    3   1   3   1    Read block 1 (replaces 2/2). disk + memory copy.
    2   1   2   1    Read block 2 (replaces 3/3). disk + memory copy.
    2   3   2   3    Read block 3 (replaces 1/1). disk + memory copy.

Now with postgres getting one buffer and the OS getting three:

    OS1 OS2 OS3 P1    Action
    -   -   -   -    Start with all caches empty.
    1   -   -   1       Read block 1 (replaces -/-). disk + memory copy.
    1   2   -   2    Read block 2 (replaces -/1). disk + memory copy.
    1   2   3   3    Read block 3 (replaces -/2). disk + memory copy.
    1   2   3   2    Read block 2 (in cache, replaces 3). memory copy.
    1   2   3   1    Read block 1 (in cache, replaces 2). memory copy.
    1   2   3   2    Read block 2 (in cache, replaces 1). memory copy.
    1   2   3   3    Read block 3 (in cache, replaces 2). memory copy.

So with 2 and 2 buffers for the OS and postgres, we end up doing
six disk reads and six memory copies. With 3 and 1, we do three
disk reads and seven memory copies. This is why you do not want to
take buffers from the OS in order to give them to postgres.

So, you ask, why not devote almost all your memory to postgres cache,
and minimize the OS's caching? Answer: if you can do that without
causing any of your programs to swap, that's fine. But I doubt you can,
unless you have, say, perfect control over how many sorts will ever be
ocurring at once, and so on.

cjs
--
Curt Sampson  <cjs@cynic.net>   +81 90 7737 2974   http://www.netbsd.org
    Don't you know, in this new Dark Age, we're all light.  --XTC