How can I read URLs on separate lines in a .txt file and ping each one

I have a .txt file with 5 urls in it. I want to read it line by line, and perform 4 ping requests for each site, and extract the time. The code I wrote is below, but it isn't working.

Can anyone give me a hint?

#!/bin/bash

cat /home/akis/Desktop/sites.txt
declare -i var=1
while read -r line
do
    while $var <= 5 
    do
        var2=$((var2+1)) 
        name="$line" | ping -c 4 $name > $var.txt | awk '{ print $8 }' < $var.txt | awk '/time/' > $var2.txt | tr '=' '\t' < $var2.txt | awk '{ print $2 }' > $var2.txt
    done

    var=$((var+1))  
done 

Solution 1:

^{This as well as Sergiy's answer came from Loop over text file with URLs and execute ping on each.}

Assuming the file urls resides in the current directory and contains your urls only, one per line:

#!/bin/bash
while IFS='' read -r l || [ -n "$l" ]; do
  avg=$(ping -q -w 10 "$l" | sed '$!d;s_.*=[^/]*/\([^/]*\)/.*_\1_')
  echo "Average respond time for $l: $avg ms" >> avg_time
done < "urls"

Example run

Above script was named avg here.

$ cat urls
askubuntu.com
unix.stackexchange.com
stackoverflow.com
$ ./avg
$ cat avg_time 
Average respond time for askubuntu.com: 37.742 ms
Average respond time for unix.stackexchange.com: 35.966 ms
Average respond time for stackoverflow.com: 38.982 ms

Explanations

• #!/bin/bash
  this so-called shebang defines which shell your script has to be started with, here it's bash
• while IFS='' read -r l || [ -n "$l" ]; do stuff; done < "urls"
read the file named urls line by line, assigning the content of the currently processed line to variable l and doing stuff with it
• ping -q -w 10 "$l" (= ping -qw10 "$l")
  call ping with the content of variable l as the argument and the options -quiet (less output we don't need) and -w 10 (timeout in seconds), so that the currently processed URL is pinged for exactly 10 seconds – you may adjust this value to your needs as well as specify other options (see man ping) for the list)

• avg=$(ping … | sed …)
  pipe the output of ping to sed which cuts out the average respond time and save this value in variable avg

  • sed '$!d;s_.*=[^/]*/\([^/]*\)/.*_\1_'
  • sed '$!d
    don't (!) delete the last line ($), but the whole rest
  • sed 's/a/b/'
substitute a by b – can also be written as e.g. sed 's_a_b_' if useful, this way we don't need to escape literal slash characters in the strings
    • .* – take everything
    • = and / – a literal “=” and “/”
    • [^/]* – take everything that's not (^) a slash
    • \([^/]*\) – take everything that's not (^) a slash and save it in a group
    • \1 – the first group saved with \(…\) before
  • the same work could also be done with awk -F/ '/^rtt/{print$5}':
    • -F/ – define / as the Field delimiter
    • /^rtt/{…} – select the line beginning (^) with “rtt” and do … with it
    • print$5 – print only field no. 5

• echo "Average respond time for $l: $avg ms" >> avg_time
  print the text inserting the content of the variables l and avg and redirect this output to the file avg_time appending to its content

Solution 2:

For a single simple command such as ping , one can use xargs. The cool thing about xargs is that it has --arg-file option which allows you to specify file which xargs will use to provide positional parameters to the command you're trying to run; in your case, that would be one site per line in the text file.

Specifically the command you want is this:

xargs -I% --arg-file=./input.txt  ping -c 4 %

• -I allows us to choose place holder for each argument, which xargs internally will
• --arg-file is the file from which positional parameters will come from; note that we're using ./ to indicate that the file is located in current working directory
• ping -c 4 will attempt to ping each site that comes from input.txt with 4 packets

Note that we can also use -q option to ping that will prevent printing a line for each packet received/transmitted, with only statistics being outputted. From there we can clean up output even further with awk or any other text-processing utility (keep in mind that output will be buffered while going via pipe and waiting for ping to actually finish and output statistics, so will show up on the screen a bit slow):

$ xargs -I% --arg-file=./input.txt  ping -c 4 % -q | awk '/^---/||/avg/'                                                                                     
--- askubuntu.com ping statistics ---
rtt min/avg/max/mdev = 49.677/73.403/94.312/19.276 ms
--- unix.stackexchange.com ping statistics ---
rtt min/avg/max/mdev = 40.015/59.099/115.545/32.590 ms
--- stackoverflow.com ping statistics ---
rtt min/avg/max/mdev = 40.130/40.878/42.685/1.056 ms

$ xargs -I% --arg-file=./input.txt  ping -c 4 % -q | awk -F'[ /]' '/^---/{print $2};/avg/{print $8}'                                                         
askubuntu.com
39.809
unix.stackexchange.com
189.557
stackoverflow.com
161.974