Why doesn't prioritizing processes produce a speed improvement?

I have got 2 applications that are both using a lot of system resources. When I decrease the priority of one in Task Manager, while increasing the other, I don't notice any significant improvement in speed in the application with the higher priority.

Why is this? Is there more going on or is there more that needs to be done?

Solution 1:

Priority does not help when the bottleneck is the CPU itself. What priority actually does is affect the scheduling algorithm the Operating System uses to determine which process runs next since there are not enough processors in most systems to run every process continuously.

A higher-priority task will make its way to the top of the queue quicker, so this helps with general latency, but if your process is exhausting the entire timeslice it is allocated on actual computation then scheduling is not going to change anything there. Changing the priority is more useful when you have a process that is waiting on I/O and you want it to be more responsive.

Solution 2:

Priority is CPU time. Are all cores being 100% utilized all the time? If not, then priority would have no effect. Quite often the CPU is not the bottleneck, and it's memory, disk, or GPU resources.

Solution 3:

Priority only matters when there are more runnable threads than available CPU cores. When that happens, priority controls which threads get to run. In most systems, there is not enough computation happening for any contention on the CPU: the threads are all blocked, waiting for something to happen. That might be waiting for you to type something, move the mouse, touch the screen, or for data to arrive from the disk, the network, some other device you've plugged in, or for another thread to finish working on a critical data structure. It might be waiting on part of the program to be read from the disk or some memory that was swapped out to be read back, rather than explicitly reading a file.

In Windows, the scheduler keeps a queue of runnable threads at each priority level. When it makes a scheduling decision - either that a thread has exhausted its quantum (permitted time before something else needs to run), meaning another thread should get a turn, or the thread has blocked and is no longer runnable, or a higher-priority thread has become unblocked - the next thread in the queue at the top priority level with any runnable threads will be scheduled. If the thread that was running has used up its quantum, it is put on the end of the queue. If it's the only thread at its priority level that is runnable, and there are no other higher-priority runnable, but not running, threads, it will get another turn.

In multicore/multiprocessor systems, there may be restrictions on which cores a thread can run on. Also, the system tries to keep threads on their ideal core and within their NUMA node so that the thread's data is likely to still be in that core's cache and it has fast access to data it created. Threads will still be run on non-ideal cores if there's no choice of what to run next.

The system makes use of various dynamic priority boosts and dynamic quantum sizes so that the foreground application gets more time (if it needs it) than background processes, and so that processes can react quickly when I/O operations complete (including mouse, keyboard and touchscreen input). In addition, priority boosting is used to get around priority inversions, where a high-priority thread is waiting for a resource that a low-priority thread is currently holding. If there's a medium-priority thread also running, it will starve the low-priority thread of processor time, holding up the high-priority thread. So the low-priority thread is temporarily boosted to the higher priority so it gets time and hopefully releases the resource that the high-priority thread needs.

Before Windows Vista, the thread priority had no effect on how quickly I/O operations completed. Since Windows Vista, I/Os can also have a priority, which by default comes from the thread priority.

Summary: you largely won't see any effect of changing thread priorities unless your CPU is heavily loaded, and even then the effect will typically be minimal. If the process has to wait for I/O or it's not contending with other processes for CPU time, it is already running the fastest it can and changing priority isn't going to make it any faster.