Bash: Dynamically traffic shaping rsync using variables for times schedule and slowing during runtime?
What are some solutions to having dynamic traffic shaping run during a bash script, after a command has already started? Is this even possible?
My use case is, I have rsync
running on a huge set of disks to a remote backup location and this takes many many hours over Internet, so I'd like to apply traffic shaping to the line where rsync
is invoked, but only during specified times.
So say for instance, current schedule is to reduce uploads to 1000 kilobytes per second (1.0MB/s) during 5am to 10am AND 3pm to 9pm.
I've looked at --bwlimit=1000
for rsync
but this shapes rsync
for the entire time it runs, not just the desired shaping timeframes.
Is it possible to slow rsync
once it's started? Because if that's possible somehow, that would be my solution.
I've seen people suggest wondershaper
but I'm wondering if this could be "switched on and off" programmatically? And if yes, how to do that?
Here's a mock up that I've hacked together using generous contributions from Stackoverflow, that gives the functionality of specifying and checking a time schedule (based on the hour, which is fine for my needs at the moment)...
#!/bin/bash
declare -A shapingTimes
declare -A keycount
useTrafficShaping=1
# schedule
shapingTimes=([0-from]=5 [0-to]=10) # 5am to 10am
shapingTimes+=([1-from]=15 [1-to]=21) # 3pm to 9pm
# because keys and array content differs in order from the way they are defined, we need to tidy this up by setting up the shaping times order in the way in which it is defined above
# to do this, count how many keys used in array entry (https://stackoverflow.com/questions/63532910)
# we use this result to to dynamically calculate the total number of entries ($totalshapingTimes)
for i in "${!shapingTimes[@]}"; do keycount[${i//[0-9]-/}]=""; done
totalshapingTimes=$((${#shapingTimes[*]} / ${#keycount[@]}))
x=1; while [[ $x -le "$totalshapingTimes" ]]; do shapingTimes_order+="$(( x-1 )) "; x=$(( $x + 1 )); done
# 'shapingTimes_order' array is used later in this script to process which shapingTimes are handled in what order
if [[ -n $useTrafficShaping ]] && [[ $useTrafficShaping == 1 ]]; then
echo "Traffic shaping: ON"
# get current time (hour) to determine if we're inside peak times
currentHour=$(date +%H)
# display our traffic shaping time windows as defined in above array
echo "Defined schedule:"
for i in ${shapingTimes_order[*]}; do
echo "${shapingTimes[$i-from]}hrs to ${shapingTimes[$i-to]}hrs"
# work out which peak times we're using, thanks to the help of Glenn Jackman (https://stackoverflow.com/questions/18128573)
if (( ${shapingTimes[$i-from]} <= 10#$currentHour && 10#$currentHour < ${shapingTimes[$i-to]} )); then
# current time matches a specified shaping timeframe, so rsync should go SLOW!
shape=1
fi
done
echo "The current hour is $currentHour"
if [[ -z $shape ]]; then
echo "Not in a shaping window, rsync can run as normal."
# some command here, run rsync as normal
else
echo "Matches a shaping schedule. *** SHAPING NETWORK TRAFFIC ***"
# command here to kick in traffic shaping
fi
else
echo "Traffic shaping: OFF"
# run rsync as normal
fi
rsync
cannot have its bandwidth limit parameter changed dynamically.
However you could
- Split the job into several smaller tasks where each will have a specific bw limit based on the time it starts, eg
for instance
rsync a b c d e target:/dir/
to
rsync --bw-limit=bw1 a target:/dir/
...
rsync --bw-limit=bw2 e target:/dir/
- Use the
--time-limit=MINS
option
rsync
is smart and will not redo what it just did, if the files are synced, so you could run rsync
for 1 hour with a bw limit bw1, then restart it with bw2, etc (and pause it if it needs to be), or combine with the above solution.
rsync --bw-limit=bw1 --time-limit=60 ...
rsync --bw-limit=bw2 --time-limit=60 ...
- Change rsync
If you feel like it, rsync
is open-source, and you could add some code to have rsync
answer dynamically to some signal (signal and kill) that would change the internal options (maight have to change the target rsync daemon as well depending on the code)
rsync --my-new-version ...
kill -SIGNALTOSLOW rsyncpid
...
kill -SIGNALTOSPEED rsyncpid