How to get real-time network statistics in Linux with KB/MB/Bytes format and for specific port or application processID?
I used IPTraf, Iftop, vnstat, bwm-ng, ifconfig -a. None of them is helping me to find real-time packets that are being sent/received from my application in KB or MB format. The reason is I am writing an application, where I need to be very sure my compression is correct, but I can't test to move forward.
What I can use to track very specific and accurate real-time network statistics?
Your application is probably sending packets to a specific UDP or TCP port number or to a specific IP-address.
You can therefore use something like TCPdump to capture that traffic.
TCPdump doesn't give you the real-time stats you desire but you can feed it's output to something that does (I'll try to update this answer with an answer later).
Update:
$ sudo tcpdump -i eth1 -l -e -n | ./netbps
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
11:36:53 2143.33 Bps
11:37:03 1995.99 Bps
11:37:13 2008.35 Bps
11:37:23 1999.97 Bps
11:37:33 2083.32 Bps
131 packets captured
131 packets received by filter
0 packets dropped by kernel
I interrupted that after a minute by pressing Ctrl+C.
You'd need to add a suitable filter expression at the end of the tcpdump command to only include the traffic generated by your app (e.g. port 123)
The program netbps is this:
#!/usr/bin/perl
use strict;
use warnings;
use Time::HiRes;
my $reporting_interval = 10.0; # seconds
my $bytes_this_interval = 0;
my $start_time = [Time::HiRes::gettimeofday()];
STDOUT->autoflush(1);
while (<>) {
if (/ length (\d+):/) {
$bytes_this_interval += $1;
my $elapsed_seconds = Time::HiRes::tv_interval($start_time);
if ($elapsed_seconds > $reporting_interval) {
my $bps = $bytes_this_interval / $elapsed_seconds;
printf "%02d:%02d:%02d %10.2f Bps\n", (localtime())[2,1,0],$bps;
$start_time = [Time::HiRes::gettimeofday()];
$bytes_this_interval = 0;
}
}
}
It's just an example, adjust to taste.
Usage as below from same folder:
To check packer per interface: ./netpps.sh eth0
To check speed per interface: ./netspeed.sh eth0
Measure Packets per Second on an Interface netpps.sh as file name
#!/bin/bash
INTERVAL="1" # update interval in seconds
if [ -z "$1" ]; then
echo
echo usage: $0 [network-interface]
echo
echo e.g. $0 eth0
echo
echo shows packets-per-second
exit
fi
IF=$1
while true
do
R1=`cat /sys/class/net/$1/statistics/rx_packets`
T1=`cat /sys/class/net/$1/statistics/tx_packets`
sleep $INTERVAL
R2=`cat /sys/class/net/$1/statistics/rx_packets`
T2=`cat /sys/class/net/$1/statistics/tx_packets`
TXPPS=`expr $T2 - $T1`
RXPPS=`expr $R2 - $R1`
echo "TX $1: $TXPPS pkts/s RX $1: $RXPPS pkts/s"
done
Measure Network Bandwidth on an Interface netspeed.sh as file name
#!/bin/bash
INTERVAL="1" # update interval in seconds
if [ -z "$1" ]; then
echo
echo usage: $0 [network-interface]
echo
echo e.g. $0 eth0
echo
exit
fi
IF=$1
while true
do
R1=`cat /sys/class/net/$1/statistics/rx_bytes`
T1=`cat /sys/class/net/$1/statistics/tx_bytes`
sleep $INTERVAL
R2=`cat /sys/class/net/$1/statistics/rx_bytes`
T2=`cat /sys/class/net/$1/statistics/tx_bytes`
TBPS=`expr $T2 - $T1`
RBPS=`expr $R2 - $R1`
TKBPS=`expr $TBPS / 1024`
RKBPS=`expr $RBPS / 1024`
echo "TX $1: $TKBPS kB/s RX $1: $RKBPS kB/s"
done
Please refer this site for more info http://xmodulo.com/measure-packets-per-second-throughput-high-speed-network-interface.html
Easiest to use and easiest to control output and redirect to file for continuous logging:
ifstat
Probably comes with most linux distributions, and can be installed with brew on mac
I think you can use the proc interface to get the information you need. I created this little shell script called rt_traf.sh:
#!/bin/bash
cat /proc/$1/net/netstat | grep 'IpExt: ' | tail -n 1 | awk '{ print $8 "\t" $9 }'
This will print the in and out octets separated by a tab. Octets multiplied by 8 will give you bits/second and then divided by 10^6 will give you megabits/second. Of course you can add this to the shell script to format the output how you want it. You can call this with the PID of your application like so ./rt_traf.sh <PID> which will give you an instantaneous reading of your application since startup. To watch real time stats per second you can wrap the shell script in the watch command:
watch -n 1 ./rt_traf.sh <PID>
The -n parameter can be adjusted all the way down to tenths of a second. To do a calculation over time I would do something like this:
PID=<PID>; START=`./rt_traf.sh $PID`;IN_START=`echo $START | awk '{ print $1 }'`; OUT_START=`echo $START | awk '{ print $2 }'`; sleep 10; END=`./rt_traf.sh $PID`; IN_END=`echo $END | awk '{ print $1 }'`; OUT_END=`echo $END | awk '{ print $2 }'`; IN_BPS=`echo "scale=2; (($IN_START-$IN_END)/10)/8" | bc`; OUT_BPS=`echo "scale=2; (($OUT_START-$OUT_END)/10)/8" | bc`; echo "In: " $IN_BPS "Bits/second"; echo "Out: " $OUT_BPS "Bits/second"
Again the math can be adjusted for the size/times you need. Not the most elegant or shrink wrapped solution but it should work in a pinch.
I just needed to measure how much mysql traffic is going in and out on an old system where perl didn't work properly, so I couldn't resist writing few lines of awk that serve the same goal of measuring total traffic seen by tcpdump:
# tcpdump -l -e -n port 3306 | \
awk '{
t=substr($1, 0, 8);
n=substr($9, 0, length($9)-1);
if(t != pt){
print t, sum;
sum = 0;
} else {
sum += n
}
pt=t;
}'
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
14:41:54
14:41:55 466905
14:41:56 765220
14:41:57 741511
14:41:58 688219
14:41:59 492322
14:42:00 800087
14:42:01 1248608
14:42:02 1276476
14:42:03 755586
14:42:04 1029453
14:42:05 818298
^C32515 packets captured
32633 packets received by filter
107 packets dropped by kernel
And if you like one-liners better, here's one for you:
tcpdump -l -e -n port 3306 | awk '{t=substr($1,0,8);n=substr($9,0,length($9)-1);if(t!=pt){print t,sum;sum=0;}else{sum+=n}pt=t;}'