Nfsen安装过程

Nfsen

Before I begin, let me remind you of the excellent documentation of nfdump and nfsen on the following pages:

• http://nfdump.sourceforge.net/
• http://nfsen.sourceforge.net/

The only reason I've compiled this step-by-step guide is because I tendency to always document the stuff I do. And I haven't yet found any online guide for nfsen which goes into such details. So if you're looking for a braindead do-this-do-that type of guide, then you've come to the right place.

Installation guide

This guide currently describes the procedure to get nfsen up and running on a Ubuntu server (or most likely on debian-based systems in general). Red Hat/Fedora/CentOS users should have no problem following the steps and make small changes to fit their system.

First off, lets get the necessary basic tools to be able to compile software on this system:

apt-get install build-essential

Install Apache (or your favourite web-server, whatever that may be. But this guide is based around Apache 2)

apt-get install apache2

A PHP interpreter is needed. It will integrate it self with Apache. I also install the command line version of PHP for practical reasons.

apt-get install php5 php5-cli

Check the version of Perl already installed on your system (it should be > 5)

perl --version

Nfsen requires the Mail-Header and Mail-Internet modules for Perl. I'm going to use CPAN to get them installed. First I initialize the CPAN shell.

perl -MCPAN -eshell

Then, install the needed modules.

cpan> install Mail::Header
cpan> install Mail::Internet

Some other needed packages gets installed

apt-get install libpng12-dev libfreetype6-dev libart-2.0-dev bison flex

Download rrdtool from its homepage, and compile it with the option enable-perl-site-install. Currently, per July 2008, rrdtool versjon 1.3 is not supported by NfSen, so you'll have to get the latest stable 1.2.x. In this guide, we use version 1.2.27, you may have to adjust some of the commands below to suit your system.

wget http://oss.oetiker.ch/rrdtool/pub/rrdtool.tar.gz
tar zxvf rrdtool.tar.gz
cd rrdtool-1.2.27
./configure --enable-perl-site-install
make && make install

Then we download nfdump (as nfsen, does not include it) and compile it with support fornfprofile (which nfsen uses). Again the path to rrdtool may have to be changed.

wget http://downloads.sourceforge.net/nfdump/nfdump-1.5.7.tar.gz
tar zxvf nfdump-1.5.7.tar.gz
cd nfdump-1.5.7
./configure --enable-nfprofile --with-rrdpath=/usr/local/rrdtool-1.2.27/
make && make install

Download nfsen

wget http://downloads.sourceforge.net/nfsen/nfsen-1.3.tar.gz
tar zxvf nfsen-1.3.tar.gz
cd nfsen-1.3

Make a copy of the default configuration file, and modify it as you'd like. You should look atthis diagram to get an overview of the directory structure of nfsen before you make your changes to the configuration file.

cp etc/nfsen-dist.conf etc/nfsen.conf
vi etc/nfsen.conf

Here are my configuration changes. If you have multiple flow sources, simply add more lines to the sources-array.

$BASEDIR = "/opt/nfsen";
$HTMLDIR    = "/var/www/nfsen/";
$PROFILEDATADIR="/var/log/netflow";
$USER    = "www-data";
$WWWUSER  = "www-data";
$WWWGROUP = "www-data";
%sources = (
   'flowsource01'    => { 'port' => '9999', 'col' => '#ff0000', 'type' => 'netflow' },
);

Now it's time to run the installation script and pointing to your config file.

./install.pl etc/nfsen.conf

For some reason the nfsen frontend does not have an index file, so lets quickly create one

echo -e "" > /var/www/nfsen/index.php

Start nfsen

/opt/nfsen/bin/nfsen start

And the web-based frontend to Nfsen is now available at http:///nfsen/

Standalone configuration

In some situations you may want to run nfcapd collectors and nfsen on separate machines. To be able to do this you need to make some minor configuration changes tonfsen.conf in addition to some os-level modifications. One nfsen.conf I've previously used is the following:

$BASEDIR = "/opt/nfsen";
$HTMLDIR    = "/var/www/html/nfsen/";
$PROFILEDATADIR="/var/log/nfsen/profiles-data";
$PROFILESTATDIR="/var/log/nfsen/profiles-stat"
$USER     = "www-data";
$WWWUSER  = "www-data";
$WWWGROUP = "www-data";
%sources = (
   'router01'  => { 'port'    => '0', 'col' => '#ff0000' },
   'router02'  => { 'port'    => '0', 'col' => '#00ff00' },
   'router03'  => { 'port'    => '0', 'col' => '#0000ff' },
);

Notice that the port parameter is set to 0 (zero). This make sure that nfsen does not automatically initiate nfcapd processes for each flow source defined in the sources-array when starting nfsen.

One the collector-machine /var/log/netflow is a separate disk-partition which the nfcapd processes writes flow data to. This partition is also read-only mounted on the nfsen-machine at/var/log/netflow over the network. Exactly how you do this depends on your system and setup. There are many ways to get the partition mounted on another server; NFS, common NAS, clustered filesystem like GFS or Lustre, or other solutions. In any case you need to get the partition nfcapd stores flow data on shared with the machine nfsen is running on. Also on the nfsen-machine the separate disk-partition/var/log/nfsen is used by nfsen to store profile and plugin data.

In such a configuration, it is also imperative that the ident-string in the flow data stored by nfcapd is identical with the configuration innfsen.conf. This means that for instance if the router01 source is defined in the nfsen configuration as:

'router01'  => { 'port'    => '0', 'col' => '#ff0000' }

then the nfcapd process for this flow source needs to use the following parameter:

-I router01

In this example the ident string "router01" set by nfcapd matches the nfsen configuration. If this doesn't match, nfprofile will not be able to process the flow data.

As the nfsen configuration expects to find the flow data in the live directory under$PROFILEDATADIR (defined in nfsen.conf), which in this example is/var/log/nfsen/profiles-data/, we need to symlink to the actual data which is located on the read-only mounted partition/var/log/netflow.

# pwd
/var/log/nfsen/profiles-data/live/router01
# ls -l
total 0
lrwxrwxrwx 1 root root 35 2008-05-22 13:36 2008 -> /var/log/netflow/router01/2008/

Here we have created the directory router01 (which complies with the ident-string for this source) under/var/log/nfsen/profiles-data/live. This is something that has to be done for each flow-source defined by%sources in nfsen.conf.

All this combined, we're now able to run nfsen on a completely separate server.

PortTracker

Included with nfsen is the plugin called PortTracker. This plugin tracks each TCP/UDP port and provides graphical insight to the activity observed on the most active ports.

cd /usr/local/src/nfsen/contrib/PortTracker/

Edit do_compile and define the NFDUMP, RRDINCLUDE andRRDLIB variables.

NFDUMP=/usr/local/src/nfdump-1.5.7
RRDINCLUDE=/usr/local/rrdtool-1.2.27/include
LIBRRD=/usr/local/rrdtool-1.2.27/lib

Currently there is a small bug which prevents compilation. Therefore add $NFDUMP/minilzo.o to the definition of NFDUMP_OBJ in do_compile.

NFDUMP_OBJ="$NFDUMP/util.o $NFDUMP/nftree.o $NFDUMP/grammar.o $NFDUMP/scanner.o $NFDUMP/nffile.o $NFDUMP/flist.o $NFDUMP/nf_common.o $NFDUMP/panonymizer.o $NFDUMP/rijndael.o $NFDUMP/ipconv.o $NFDUMP/minilzo.o"

Execute do_compile which now will create the binary 'nftrack'. Install this binary in the same directory as the rest of nfdump binaries.

./do_compile
cp nftrack /usr/local/bin/

Add the path to the RRD library to the LD configuration.

echo "/usr/local/rrdtool-1.2.27/lib" > /etc/ld.so.conf.d/librrd.conf
ldconfig

Create a directory for PortTracker to store its port data (requires approximately 10 GiB). The user$USER from nfsen.conf has to have write permission to this directory. In my case this user iswww-data-

mkdir /var/log/netflow/porttracker
chown www-data:www-data /var/log/netflow/porttracker
chmod 775 /var/log/netflow/porttracker

Edit PortTracker.pm and set the path to the directory you just created.

my $PORTSDBDIR = "/var/log/netflow/porttracker";

Copy backend and frontend files

cp PortTracker.pm /opt/nfsen/plugins/
cp PortTracker.php /var/www/nfsen/plugins/

Edit nfsen.conf and add the PortTracker plugin to the live profile

vi /opt/nfsen/etc/nfsen.conf

@plugins = (
        [ 'live',   'PortTracker'],
); 

Start nftrack as $USER

sudo -u www-data nftrack -I -d /var/log/netflow/porttracker/

Restart nfsen

/opt/nfsen/bin/nfsen reload

Check syslog to see if the plugin was successfully loaded

grep -i porttracker /var/log/syslog
...
Apr 13 16:48:13 hw01 nfsen[31474]: Loading plugin 'PortTracker': Success 
...
Apr 13 16:48:13 hw01 nfsen[31474]: Initializing plugin 'PortTracker': Success
...

After the next 5 minute processing interval, PortTracker will generate data and graphs in the web-interface.

Add a profile from CLI

One can maintain nfsen from the CLI as well as from the web-interface. Here follows an example on how to add a new profile.

List the profiles we have already defined.

root@server:/opt/nfsen/bin# ./nfsen -A
live
ssh

Now we'll add a new profile, called "tcp" with the start time at 00:00 June 12 and set to as a continuous profile (i.e by not defining an end time)

root@server:/opt/nfsen/bin# ./nfsen -a tcp -B 2008-06-12-00-00
# # 
name    tcp
group   (nogroup)
tcreate Thu Jun 12 21:23:30 2008
tstart  Thu Jun 12 00:00:00 2008
tend    Thu Jun 12 21:15:00 2008
updated Wed Jun 11 23:55:00 2008
expire  0 hours
size    0
maxsize 0
type    continuous
locked  0
status  new
version 130

Then add channels (flow sources). In my example here, I have only one single channel; router01. I also add my profile filter to extract only tcp traffic.

root@server:/opt/nfsen/bin# ./nfsen --add-channel tcp/router01 filter="proto 6"
...
channel router01     sign: + colour: #ff0000 order: 1        sourcelist: router01Files: 0 Size: 0

The profile has status new, i.e. it has not yet been activated.

root@server:/opt/nfsen/bin# ./nfsen --commit-profile tcp
...
status  built 0

If there is a lot of data to process for this profile, the status built <%> line will show and gradually increase. You may view a profile summary by running:

root@server:/opt/nfsen/bin# ./nfsen -l tcp
# # 
name    tcp
group   (nogroup)
tcreate Thu Jun 12 21:23:30 2008
tstart  Thu Jun 12 00:00:00 2008
tend    Thu Jun 12 21:15:00 2008
updated Thu Jun 12 21:15:00 2008
expire  0 hours
size    70.6 KB
maxsize 0
type    continuous
locked  0
status  OK
version 130
channel router01     sign: + colour: #ff0000 order: 1        sourcelist: router01Files: 257       Size: 1052672

In this case we see that the profile is completely built (OK) and currently is 70 KiB in size.