How do I measure separate CPU core usage for a process?

Is there any way to measure a specific process CPU usage by cores?

I know top is good for measuring the whole system's CPU usage by cores and taskset can provide information about which CPU core is allowed for the process to run on.

But how do I measure a specific process' CPU usage by CPU cores?

Solution 1:

You can still do this in top. While top is running, press '1' on your keyboard, it will then show CPU usage per core.

Limit the processes shown by having that specific process run under a specific user account and use Type 'u' to limit to that user

Solution 2:

You can use:

 mpstat -P ALL 1

It shows how much each core is busy and it updates automatically each second. The output would be something like this (on a quad-core processor):

10:54:41 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest   %idle
10:54:42 PM  all    8.20    0.12    0.75    0.00    0.00    0.00    0.00    0.00   90.93
10:54:42 PM    0   24.00    0.00    2.00    0.00    0.00    0.00    0.00    0.00   74.00
10:54:42 PM    1   22.00    0.00    2.00    0.00    0.00    0.00    0.00    0.00   76.00
10:54:42 PM    2    2.02    1.01    0.00    0.00    0.00    0.00    0.00    0.00   96.97
10:54:42 PM    3    2.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00   98.00
10:54:42 PM    4   14.15    0.00    1.89    0.00    0.00    0.00    0.00    0.00   83.96
10:54:42 PM    5    1.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00   99.00
10:54:42 PM    6    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
10:54:42 PM    7    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00

This command doesn't answer original question though i.e. it does not show CPU core usage for a specific process.

Solution 3:

you can use ps.
e.g. having python process with two busy threads on dual core CPU:

$ ps -p 29492 -L -o pid,tid,psr,pcpu
  PID   TID PSR %CPU
29492 29492   1  0.0
29492 29493   1 48.7
29492 29494   1 51.9

(PSR is CPU id the thread is currently assigned to)

you see that the threads are running on the same cpu core (because of GIL)

running the same python script in jython, we see, that the script is utilizing both cores (and there are many other service or whatever threads, which are almost idle):

$ ps -p 28671 -L -o pid,tid,psr,pcpu
  PID   TID PSR %CPU
28671 28671   1  0.0
28671 28672   0  4.4
28671 28673   0  0.6
28671 28674   0  0.5
28671 28675   0  2.3
28671 28676   0  0.0
28671 28677   1  0.0
28671 28678   1  0.0
28671 28679   0  4.6
28671 28680   0  4.4
28671 28681   1  0.0
28671 28682   1  0.0
28671 28721   1  0.0
28671 28729   0 88.6
28671 28730   1 88.5

you can process the output and calculate the total CPU for each CPU core.

Unfortunately, this approach does not seem to be 100% reliable, sometimes i see that in the first case, the two working threads are reported to be separated to each CPU core, or in the latter case, the two threads are reported to be on the same core..

Solution 4:

htop gives a nice overview of individual core usage

Solution 5:

The ps solution was nearly what I needed and with some bash thrown in does exactly what the original question asked for: to see per-core usage of specific processes

This shows per-core usage of multi-threaded processes too.

Use like: cpustat `pgrep processname` `pgrep otherprocessname` ...

#!/bin/bash

pids=()
while [ $# != 0 ]; do
        pids=("${pids[@]}" "$1")
        shift
done

if [ -z "${pids[0]}" ]; then
        echo "Usage: $0 <pid1> [pid2] ..."
        exit 1
fi

for pid in "${pids[@]}"; do
        if [ ! -e /proc/$pid ]; then
                echo "Error: pid $pid doesn't exist"
                exit 1
        fi
done

while [ true ]; do
        echo -e "\033[H\033[J"
        for pid in "${pids[@]}"; do
                ps -p $pid -L -o pid,tid,psr,pcpu,comm=
        done
        sleep 1
done

Note: These stats are based on process lifetime, not the last X seconds, so you'll need to restart your process to reset the counter.