On 09/03/2014 07:17 PM, Kevin Goess wrote:
> Debian squeeze, still on 2.6.32.
Interesting. Unfortunately that kernel suffers from the newer task
scheduler they added to 3.2, and I doubt much of the fixes have been
back-ported. I don't know if that affects the memory handling, but it might.
> Darn, really? I just learned about the "mysql swap insanity" problem and
> noticed that all the free memory is concentrated on one of the two nodes.
>
> $ numactl --hardware
> available: 2 nodes (0-1)
> node 0 cpus: 0 2 4 6
> node 0 size: 32768 MB
> node 0 free: 9105 MB
> node 1 cpus: 1 3 5 7
> node 1 size: 32755 MB
> node 1 free: 259 MB
And that's the kind of behavior we were seeing until we upgraded to 3.8.
A 8GB gap between your nodes is definitely bad, but it's not the same
thing they described in the MySQL swap insanity posts. MySQL has a much
bigger internal cache than we do, so expects a good proportion of system
memory. It's not uncommon for dedicated MySQL systems to have more than
75% of system memory dedicated to database use. Without NUMA
interleaving, that's a recipe for a broken system.
> $ free
> total used free shared buffers cached
> Mem: 66099280 56565804 9533476 0 11548 51788624
And again, this is what we started seeing with 3.2 when we upgraded
initially. Unfortunately it looks like at least one of the bad memory
aging patches got backported to the kernel you're using. If everything
were working properly, that excess 9GB would be in your cache.
Check /proc/meminfo for a better breakdown of how the memory is being
used. This should work:
grep -A1 Active /proc/meminfo
I suspect your inactive file cache is larger than the active set,
suggesting an overly aggressive memory manager.
--
Shaun Thomas
OptionsHouse, LLC | 141 W. Jackson Blvd. | Suite 800 | Chicago IL, 60604
312-676-8870
sthomas@optionshouse.com
______________________________________________
See http://www.peak6.com/email_disclaimer/ for terms and conditions related to this email