CPU overloaded but no proccess is using more than 1% [duplicate]
I am monitoring a Cpanel (centos) server which have a 2-core CPU (4 virtual CPU cores) and it seems to be overloaded because I got this values using top:
load average: 11.80, 13.30, 13.02
Cpu(s): 42.2%us, 11.7%sy, 0.0%ni, 35.6%id, 10.1%wa, 0.1%hi, 0.3%si, 0.0%st
But if I look at the proccess list (using top or ps) no proccess is using more that 1%
Also, the sum of the proccess CPU usage (%) equals 4 and if I even assume that 0% values are rounded numbers, and change it to 0.04 (which rounded using 1 decimal digit is 0) the sum is 11 (still less than 100%).
How can I correctly interpret this data?, are there some hidden proccesses which are overloading my cpu.
On Linux, blocked processes also contribute to the load averages. The command ps -Al lists all processes. In the second column (S for State) of its output you will find the process states. Most often I have processes waiting for disk "D", which are counted towards the load averages.
The full list of states from the ps man page is
D Uninterruptible sleep (usually IO)
R Running or runnable (on run queue)
S Interruptible sleep (waiting for an event to complete)
T Stopped, either by a job control signal or because it is being
traced.
W paging (not valid since the 2.6.xx kernel)
X dead (should never be seen)
Z Defunct ("zombie") process, terminated but not reaped by its
parent.
Sample output
F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD 4 S 0 1 0 0 80 0 - 4906 poll_s ? 00:00:23 init 1 S 0 2 0 0 80 0 - 0 kthrea ? 00:00:02 kthreadd 1 R 0 3 0 99 80 0 - 0 ? 01:00:02 runner 1 D 0 4 0 1 80 0 - 0 ? 01:00:02 loader
If these were your only processes you we see a load of about 2, 1 for the CPU hog "runner" and another 1 for the loader which is waiting for disk.
Very precise is the information available on Wikipedia
An idle computer has a load number of 0. Each process using or waiting for CPU (the ready queue or run queue) increments the load number by 1. Most UNIX systems count only processes in the running (on CPU) or runnable (waiting for CPU) states. However, Linux also includes processes in uninterruptible sleep states (usually waiting for disk activity), which can lead to markedly different results if many processes remain blocked in I/O due to a busy or stalled I/O system.1 This, for example, includes processes blocking due to an NFS server failure or to slow media (e.g., USB 1.x storage devices). Such circumstances can result in an elevated load average, which does not reflect an actual increase in CPU use (but still gives an idea on how long users have to wait).
The top info you provided doesn't necessarily mean overloading:
- the CPU is 35% idle
- the load averages aren't necessarily too big (depends on the server's intended usage)
- the RAM and swap info are missing
Or rather, if by overloading you mean hitting some limitation, there can be various aspects of it: CPU limitation, network and/or disk I/O limitations, memory usage limitations, etc.
You shouldn't try to match various CPU load/usage views - they typically mean different things and the views are also collected at different timestamps (stats collection is not atomic):
- load average means number of jobs in the running queue, not CPU usage: https://stackoverflow.com/questions/21617500/understanding-load-average-vs-cpu-usage
- the CPU % usage numbers in the process context don't have to add up to 100% for various reasons, here are just a few:
- the CPU doesn't spend all its cycles in process space
- various cycles spent in a process context is counted differently on the overall CPU % usage line (the same process could have been in both running or waiting on I/O states during the accounting interval, thus contributing to both %us and %wa numbers on the overall CPU % usage line)
- the CPU could have spent cycles on processes nu longer running, those would be counted in the overall CPU % usage line, but not present on any of the process lines