Network traffic graphing with IPAC-NG and RRD

This is a little perl script I wrote. It reads from an ipac-ng config file, stores samples in RRD, and graphs the resulting network traffic. I wrote it because I couldn't find anything on the net that did the job just so.

Note: IPAC-NG already keeps a database and already provides graphing capabilities. This document is for those who prefer to bypass these built-ins and use RRD instead.

Here's how it works...

If you have a linux box acting as a router/NAT host and want to track traffic to/from each host by port (service), you might find this useful. It gives you one page of graphs with all your hosts:



...so you can see which hosts are currently using bandwidth. If you click on any of the hosts, you get a breakdown of traffic usage per-port (service) on that host:



Prerequisites: (assuming you already have Linux with iptables and perl)

IPAC-NG - IP accounting next generation
RRDtool - store and display time-series data
My rrd-ipac-ng.pl script - a quick download
My rules.conf config file - an example for you

How to install:

Install ipac-ng (I used gdbm for database storage) and rrdtool as normal (I prefer /usr/local, YMMV). The rrd-ipac-ng script can go anywhere.

In ipac.conf, I set my rules file to /etc/ipac-ng/rules.conf - you can use my example rules.conf above and edit for your hosts. The rules.conf file is quite verbose, if in doubt just stick to my format and remember the painful secret: the first field cannot be > 19 characters or RRD will barf.

See if your rules.conf works by running: fetchipac -S ...you can dump the rules to screen with fetchipac -R

Now you can start to customize the rrd-ipac-ng.pl script. There's a few variables to change right at the top of the file, some are obvious (paths to the ipacsum and rrdtool binaries, paths to rrd data dir and ipac config file), some are output directories that must be in the document root of your webserver for them to show up on the web:

$base_url = where the resulting index.html file is going to end up.
$graph_url = where the graphics will be written

Now run the rrd-ipac-ng.pl script by hand and make sure there are no errors, and make sure there are RRD files appearing in $rrd_data_dir, and you can check the URL with your web browser to make sure blank graphs are showing up.

If it's working so far, add the statistics collection and the graphing script to root's crontab. I have a little script called rrd-cron.sh which is run every 5 minutes and contains the following: 
#!/bin/sh
# we don't keep the DB around for more than one run because we're keeping
# the data in RRD
rm /var/lib/ipac/data.db
/usr/local/sbin/fetchipac 
/usr/local/etc/rrd-ipac-ng.pl >/tmp/rrd.log 2>&1
Note: These graphs will be generated every N minutes whether someone is watching them or not. You can save resources by using RRD's perl CGI interface to generate the graphs on-demand.

Anyway, once you've got it in cron, there's one more thing - you need to initialize the ipac-ng counters when your machine boots. I put a call to fetchipac in my /etc/rc.d/rc.local:

/usr/local/sbin/fetchipac -S

I hope this works for you. If you have any problems you can email me at ipac@fumanchu.com to chat about it.

Frequently asked questions

Here are some of the questions I am getting via email and my attempted answers.

The RRD files are not appearing in the destination directory (or, the RRD files appear but the graphs do not), how do I debug? 
There are some lines in the file that run the commands from perl,               
they look like this:

        `$cmd 1>/dev/null 2>&1`;

...if you remove the redirection of command output to null, to look             
like this instead:

        `$cmd`;

...they you will see the full output of the command and you should              
be able to tell why it is failing.  I think usually it is a path                
problem, it is looking for the rrd command in the wrong place, etc.

The script spits out errors with my rules.conf file, how can I fix it?

You may already be using ipac-ng and have a rules file set up that is not compatible with this script. Our parsing of the rules.conf file is not very smart or robust, you have to make your conf file strictly adhere to the format examples on this page, for instance: 
# ALL traffic to/from spun
spun recv|ipac~fo|+|all|0/0|10.10.10.2/32||
spun send|ipac~fo|+|all|10.10.10.2/32|0/0||

# SMTP traffic to/from spun
net spun smtp recv|ipac~fo|+|tcp|0/0|10.10.10.2/32 smtp||
spun net smtp recv|ipac~fo|+|tcp|0/0 smtp|10.10.10.2/32||
net spun smtp send|ipac~fo|+|tcp|10.10.10.2/32 smtp|0/0||
spun net smtp send|ipac~fo|+|tcp|10.10.10.2/32|0/0 smtp||
This example shows the 2 kinds of allowed rules - the "all traffic to host" rule and the "port traffic to host" rule. Note that all the fields except the last are filled in, not empty.

The ipac-ng database is removed at every run. Is it possible to use your script and keep the db?

Yes, simply change the RRD datatype in the script from ABSOLUTE to COUNTER, and remove the 'rm' in your crontab. If you have existing RRD graphing you wish to preserve, you can change the RRD data type in the RRD files by exporting to XML, searching and replacing the data types above, and reimporting to RRD. YMMV on a little trick like this but here's a script I used to change from COUNTER to ABSOLUTE (I used to keep the ipac-ng DB around but it grew very large and slow.)