Another update (let me know if I'm mailing too much).
> 59.73% postmaster [kernel.kallsyms] [k]
> compaction_alloc
The compaction_alloc function seemed to point to issues with transparent
huge pages (THP.) I tried to turn this off with:
echo never > /sys/kernel/mm/transparent_hugepage/enabled
This change yielded immediate results. The "migration/xx" processes
disappeared from "top", the function "compaction_alloc" disappeared from
"perf top", and system load dropped from 5 to 0.3.
We've called off the 9.4 -> 9.3 downgrade for now.
For anyone else finding this thread, we are running CentOS 6.5 with
kernel 2.6.32 (which was released in Dec 2009.)
Cheers,
Andomar