Hard drive throttling?
You're copying a lot of stuff in same time and computer performance just dies. Try for example to copy 10+ ISO images same time.
There's QoS in networking, affinities and priorities in CPUs, but is there anything for hard drives to throttle I/O queues? Virtual filesystems? OS drivers?
Of course you can go around with this with for example
- Making list of files to copy and running script which copies files one by one
- Using software like TeraCopy
Windows and GNU/Linux solutions are welcome.
For Linux there's ionice, Windows has IO priorities since quite some time, but at least since Vista it's also fully available in a sane implementation for Workstations.
ionice only works with the CFQ scheduler, which is usually the default. If you are using another scheduler or if ionice isn't doing the job for you, you might want to periodically pause execution to let your io buffers flush. Here is a script we use for that:
#!/bin/sh
# Throttles the execution of a process by stoping it after every second of
# execution time
#
# Usage: io_nice <sleeptime> <pid>
# Where "sleeptime" is the number of seconds of sleep for each second of runtime
# and "pid" is the process ID of the process to throttle
PID=$2
SLEEPTIME=$1
while [ true ]; do
kill -s STOP $PID &> /dev/null|| exit 0
sleep $SLEEPTIME
kill -s CONT $PID &> /dev/null || exit 0
sleep 1
done
At a lower level, Linux (and I assume Windows, and *BSD, and everything else in serious use) has an IO scheduler that tries to balance the best overall utilization of each device with providing good service to each process. Linux's current default scheduler is known as Completely Fair Queueing, with the current generally available alternatives being called Anticipatory and Deadline.
On a somewhat related note, the program iotop is useful for monitoring and examining what's going on in a system under heavy IO load.