Kworker, what is it and why is it hogging so much CPU?

I recently upgraded to the Kubuntu Natty Beta 1 and I've been having a lot of issues with the process kworker. At moments it uses almost half my CPU. Also, strangely enough it seems to affect my USB ports; whenever I plug in an USB drive, the process kworker goes into hyperdrive, leaving me unable to work.

I have thought about filing a bug but since I haven't even found any reasonable explanation on what kworker is I figured I should find out first.

Solution 1:

"kworker" is a placeholder process for kernel worker threads, which perform most of the actual processing for the kernel, especially in cases where there are interrupts, timers, I/O, etc. These typically correspond to the vast majority of any allocated "system" time to running processes. It is not something that can be safely removed from the system in any way, and is completely unrelated to nepomuk or KDE (except in that these programs may make system calls, which may require the kernel to do something).

There were some reports of excessive kworker activity for relatively idle systems starting during 2.6.36 development (example discussion), and wide reports of confusion and problems with 2.6.38 (although many of these reports include the word "Natty", so I presume these people not to have used any kernel between 2.6.35 (distributed in Ubuntu 10.10) and 2.6.38 (distributed in Ubuntu 11.04).

I've found many reports of something that "fixed" this for one or another user. Most "fixes" seem to be related to updates of the kernel of various sorts. Where the update can be tracked to a specific issue, it seems to often be some driver or kernel service that has been patched to not misbehave: I have the impression that there are a very large number of things in the kernel that can cause a behaviour which is observed as excessive kworker usage.

If you find the system unusable due to excessive kworker activity, I would recommend trying to do fewer things. If you think you're not doing anything, try shutting down long-running services or timers (RSS readers, mail readers, file indexers, activity trackers, etc.). If this doesn't work, try restarting. If your system allows you to enable or disable hardware in a pre-boot environment, try turning off hardware you aren't using. If it happens on every restart before you do anything, you could try uninstalling things, but at this point you'll want to be running syscall profiling tools to track down specific applications that seem to be causing this overload.

It is to be hoped that your specific system will stop expressing this behaviour with a future kernel upgrade (and many of the most common causes of this have been solved).

Solution 2:

What is kworker? kworker means a Linux kernel process doing "work" (processing system calls). You can have several of them in your process list: kworker/0:1 is the one on your first CPU core, kworker/1:1 the one on your second etc..

Why does kworker hog your CPU? To find out why a kworker is wasting your CPU, you can create CPU backtraces: watch your processor load (with top or something) and in moments of high load through kworker, execute echo l > /proc/sysrq-trigger to create a backtrace. (On Ubuntu, this needs you to login with sudo -s). Do this several times, then watch the backtraces at the end of dmesg output. See what happens frequently in the CPU backtraces, it hopefully points you to the source of your problem.

Example: e1000e. In my case, I found a backtrace like this nearly every time:

Call Trace:
 delay_tsc+0x4a/0x80
 __const_udelay+0x2c/0x30
 e1000_acquire_swflag_ich8lan+0xa2/0x240 [e1000e]
 e1000e_read_phy_reg_igp+0x29/0x80 [e1000e]
 e1000e_phy_has_link_generic+0x85/0x120 [e1000e]
 e1000_check_for_copper_link_ich8lan+0x48/0x930 [e1000e]
 e1000e_has_link+0x55/0xd0 [e1000e]
 e1000_watchdog_task+0x5e/0x960 [e1000e]

It hinted me to a problem in the e1000e Ethernet card module, and indeed a sudo rmmod e1000e made the high CPU load go away immediately [e1000e bug #26].