Thanks All for the response, finally we figure it out. The slow is due to high number of dead rows on main table, repack these tables wipe out the issue.
On Mar 6, 2015 9:31 PM, "Merlin Moncure" <
mmoncure@gmail.com> wrote:
On Wed, Mar 4, 2015 at 1:31 PM, Soni M <diptatapa@gmail.com> wrote:
> Hello All,
>
> Master db size 1.5 TB
> All postgres 9.1.13 installed from RHEL package.
> It has streaming replica and slony replica to another servers.
>
> Server performance is slower than usual, before that, there's a big query
> got cancelled and then performance get slow.
>
> No sign of IO wait.
>
> on sar, it's %user and %system dominate the cpu usage
> 01:25:04 PM CPU %user %nice %system %iowait %steal
> %idle
> Average: all 51.91 0.00 12.03 0.66 0.00
> 35.39
>
> on perf top, i saw
> 18.93% postgres [.] s_lock
> 10.72% postgres [.] _bt_checkkeys
> almost always at top.
>
> I don't have any idea, what's causing it or how to resolve it ?
Can you post the entire 'perf top'? do you see (specifically I'm
wondering if you are bumping against the RecoveryInProgress s_lock
issue). If so, upgrading postgres might be the best way to resolve
the issue.
merlin