Logging bandwidth use?

In an attempt to make my life easier I want to log all bandwidth used on my server for a day, and then somehow export it. It doesn't matter much how it exports it because I can parse that stuff myself, but I need software to be able to accurately log traffic going in and out of eth0.

If anyone knows of such software, I'd be grateful if you could provide a link. I have already Googled for such things yet found nothing suitable.

Cheers.

My recommendation is vnStat:

vnStat is a console-based network traffic monitor for Linux and BSD that keeps a log of network traffic for the selected interface(s)

It works as advertised and seems like a good match for your needs.

To facilitate parsing you'd probably like to take a look at the --xml and --dumpdb switch in the vnstat documentation:

• --xml : Show database content for selected interface or all interfaces in xml format. All traffic values in the output are in KiB.
• --dumpdb : Instead of showing the database with a formatted output, this output will dump the whole database in a format that should be easy to parse with most script languages.

Feature list:

• quick and simple to install and get running
• gathered statistics persists through system reboots
• can monitor multiple interfaces at the same time
• several output options
• summary, hourly, daily, monthly, weekly, top 10 days
• optional png image output (using libgd)
• months can be configured to follow billing period
• light, minimal resource usage
• same low cpu usage regardless of traffic
• can be used without root permissions

When using logstash with kibana this is easy to setup

What we did is log the total traffic to a log file with the following bash script.

LOGPATH="systemstatus.log"



timestamp() {
  date +"%Y-%m-%dT%T.%N"
}


while ( sleep 60 ) ; do
#clear log file 
truncate -s 0 $LOGPATH

#log timstamp and type of log line
echo -n "$(timestamp) linux::systemstatus::network " >> $LOGPATH
#remove tabs and spaces from grep result
grep "eth" /proc/net/dev |head -n1|sed -e 's/^[ \t]*//'| sed -n 's/  \+/ /gp' >> $LOGPATH

Done

Nxlog exports the log to logstash and kibana.

In logstash the string is parsed to individual values Kibana makes the traffic visible in a derivative histogram (this shows the difference from 1 value to the next)

This gives us a nice overal graph of the used bandwidth. On top of this we also log the CPU, Disk and Memory usage of the server.

Note: we parse the whole log line from /proc/net/dev into seperate values just in case we want to monitor on lost packeges or errors