1.1   Obtaining FlowMon package

FlowMon package is available via WWW download page:

http://www.liberouter.org/clients

You can also check latest news at our web pages:

http://www.liberouter.org/

http://www.flowmon.org/

1.2   Package structure

/firmware      - COMBO6 card firmware (*.mcs files)
/base          - source code for software tools and drivers
  /mk                        - build system (makefiles)
  /sys_sw/drivers            - kernel drivers
  /sys_sw/hwtools            - necessary hardware tools for COMBO6 card
  /sys_sw/lib*               - libraries necessary for other tools
  /sys_sw/projects/flowmon   - exporters, testing programs, configuration
                               and documentation for using FlowMon
  /sys_sw/swtools/csxtool    - tool for handle COMBO6 XML files
  /vhdl_design/projects/netflow_ph1/test - FlowMon start up programs
README         - this file
RELNOTES       - differences against previous release

2.1   Host computer

The FlowMon hardware has been designed to work on any PC-AT compatible computer running GNU/Linux or other "Unix-like" operating system. It has been tested only on computers that use the x86 family of processors. The hardware requirements necessary to run FlowMon are:

• Intel Pentium or equivalent x86 based PC
• COMBO6 - 32-bit/33 MHz PCI bus

The "sandwich" consisting of COMBO6 and an interface card (COMBO-4MTX or COMBO-4SFP) fits into one PCI slot.

2.2   Supported hardware

2.3   Supported software

FlowMon software works on GNU/Linux OS with 2.4 and 2.6 kernels. The NetFlow v5 and v9 protocols are supported. The software has been tested on computers running Red Hat Enterprise Linux, Ubuntu and Debian.

More information about FlowMon probe you can find at http://www.flowmon.org

2.4   Installing COMBO6 card

After plugging Combo card into your PCI slot, you should test connection between the card and your PC. We use lspci(8) utility for this purpose. lspci(8) is a utility for displaying information about all PCI buses in the system and all devices connected to them. For correct recognition of the Combo card you need update PCI ID Database used by lspci(8) or download pciutils-2.2.2 (program collection containing lspci(8)) or later. If the lspci(8) output contains the following line your Combo card is connected properly.

$ lspci -d 18ec:
04:02.0 Ethernet controller: Cesnet, z.s.p.o. COMBO6 (rev 01)

2.5   Building package

Compilation and installation process is covered by the build system located in the base/mk directory. For more details about required tools necessary to build package see base/mk/REQUIREMENTS file.

To build kernel drivers needed by FlowMon package you will need fully configured Linux 2.4 or 2.6 kernel source tree. More installation and compilation details about kernel drivers are available in the README file inside base/sys_sw/drivers/linux directory.

pkgtool(1) helps to build, install and uninstall the FlowMon distribution package.

$ tar -xzvf flowmon-XX.YY.ZZ.tgz
$ cd flowmon-XX.YY.ZZ/base
$ ./pkgtool --build

Optionally you can define installation directory with the --prefix=path option. The --prefix option takes effect only during building package. If no prefix is set then /usr/local path is used.

$ ./pkgtool --build --prefix=/usr/local/flowmon

2.6   Installing package

The tools and kernel drivers will be installed to the installation directory (by default /usr/local).

# ./pkgtool --install

If you are going to use udev mechanism to creating device files, you can use pkgtool with --udev option. This option cause copying file with combo card rules (combo6.udev.rules) to the /etc/udev/rules directory.

# ./pkgtool --install --udev

There are necessary following post-install steps:

1. Set PATH variable to point to installed FlowMon's tools directory -> ${PREFIX}/bin

   $ export PATH=$PATH:/usr/local/bin
   

2. Set /etc/ld.so.conf to point to the FlowMon's library directory -> ${PREFIX}/lib

   Add /usr/local/lib entry to the /etc/ld.so.conf file.

   # ldconfig
   

3. If you want to load FlowMon's kernel modules when the PC starts up, add following lines to the /etc/modules file.

   combo6core
   combo6
   szedata
   netflow-ph1
   

   After this, you don't need to run netflow_ph1_lkm script any more.

4. The FlowMon probe driver needs device files in the /dev/combosix/ and /dev/szedata/ directory. To create them, run the flowdevices script in the /base/sys_sw/drivers/linux directory:

   # ./flowdevices
   

   To run FlowMon as a non-privileged user set r/w permissions on /dev/combosix/0 and /dev/szedata/0 devices.

   You have to change the permissions of the device files so that the non-privileged user that will execute the FlowMon tools has read-write access. For example, you can create a new group, say combo-rw, add that user to this group and change the file group ownership and permissions as follows:

   # chgrp combo-rw /dev/combosix/0 /dev/szedata/0
   # chmod 664 /dev/combosix/0 /dev/szedata/0
   

   Using recent 2.6 kernels it might be wanted to use the sysfs/udev mechanisms to create device files. For more details about udev see http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev.html

   Example udev rules are placed in the package in sys_sw/drivers/linux/combo6.udev.rules file. If you didn't use option --udev during package installation, copy this file to /etc/udev/rules.d directory and modify it (change group or permissions). Following line will create combo6 device files required by the FlowMon probe:

   ``KERNEL=="combosix[0-9]*", NAME="combosix/%n", GROUP="combo-rw", MODE="0664"``
   ``KERNEL=="szedata[0-9]*", NAME="szedata/%n", GROUP="combo-rw", MODE="0664"``
   

   NOTE:	Depending on your version of udevd you must adjust syntax of your udev rule file. Older versions of udevd require '=' character to match KERNEL key e.g. KERNEL="combosix[0-9]*". The newer one require '==' characters to match KERNEL key e.g. KERNEL=="combosix[0-9]*".

2.7   Upgrading package

If you have used some previous version of the FlowMon package, please follow all building and installing steps described above. Post-installation steps can be skipped if you performed these steps during previous installation (and an installation path was the same!). pkgtool(1) is able to detect your FlowMon Probe configuration file and it will ask you to decide if you wish to keep your own (but may be obsolete) configuration file or to overwrite them with our default (but up-to-date) configuration file.

2.8   Uninstalling package

It is not possible (for safety reasons) to automatically uninstall FlowMon package. pkgtool(1) will only show directories where the FlowMon tools are placed. By default the FlowMon package uses following files and directories:

/dev/combosix/
/dev/szedata/
/etc/liberouter/
/usr/local/bin/
/usr/local/etc/
/usr/local/include/
/usr/local/lib/
/usr/local/mcs/

3.1   flowmon.conf - FlowMon probe configuration file

flowmon.conf is placed in the /etc/liberouter/ directory. It is used to store default probe settings and some other useful settings for the flowmon project scripts. flowmon.conf is connected to the scripts by the . (dot) shell command, so it is, actually, set of variable definitions and these variables are then used in our scripts. You can freely modify this file and redefine default values of the probe settings.

During installation of the next flowmon package version you will be inquired if you prefer to keep your own configuration file or to rewrite it with our up-to-date version. So you don't have to be worry to lose your changes. Up-to-date file is during installation process always copied to the $PREFIX/etc/liberouter (by default /usr/local/etc/liberouter) directory. So you can use this file as a backup copy.

As we wrote earlier, flowmon.conf has the form of the list of variables. Here are main groups of variables defined in the flowmon.conf file:

probe parameters

parameters for setting up the FlowMon probe behavior, e.g.:

ACTIVE_TIMEOUT=30          # active timeout in seconds

exporter parameters

variables to set up exporter features, e.g.:

NETFLOW_PROTOCOL=9         # NetFlow protocol version (5,9)

combo6 card parameters

device file pointing to the combo6 card:

COMBO6_DEVICE="/dev/combosix/0"      # combo6 device file

firmware parameters

variables used to locate *.mcs files, e.g.:

FIRMWARE_PATH="/usr/local/mcs"       # firmware install path

4.1   Loading kernel modules

netflow_ph1_lkm script is used for loading/removing FlowMon's kernel modules. There are two main options for this script. Option -l is used for loading FlowMon's kernel modules and option -r is used for removing these modules. Script uses lspci(8) utility to detect Combo6 card.

When all kernel modules are loaded, csid(1) is able to detect type of plugged cards and provide more detailed information.

You should get following output on PC with installed

• combo6 mother card and mtx add-on card:

  $ csid
  combo6 mtx2 xcv2000
  

• combo6 mother card and sfp add-on card:

  $ csid
  combo6 sfp xcv1000
  

4.2   Setting up FlowMon probe

netflow_ph1 is main start up script for the FlowMon probe. Script boots firmware files (according to FIRMWARE_PATH variable from the /etc/liberouter/flowmon.conf), setting up probe behavior and is able to start NetFlow protocol exporter. By default, the probe settings are set according to variables from /etc/liberouter/flowmon.conf configuration file. Optionally you can define your own settings by options given to the script. More detailed information you can get by -h option:

$ netflow_ph1 -h

4.3   Logging probe information

netflow_ph1_log script logs some FlowMon hardware registers to a text file. Behavior of this script is set by given command line parameters. Main parameters are -p to set log period and -l to define log file, e.g.:

$ netflow_ph1_log -l ~/output.txt -p 30

4.4   Examples of using scripts

# ./netflow_ph1_lkm -l
$ ./netflow_ph1 -c collector.liberouter.org:60000

$ killall netflow_ph1_log
$ killall flowmon_nf5 flowmon_nf9

# ./netflow_ph1_lkm -r

$ csid -s
Board    : combo6
Addon    : mtx2
Chip     : xcv2000
LAN ports: 4
Firmware : ok
SW       : 0xf1010002
HW       : 0x00000006
Text     : NETFLOW_1Gbps_Probe

4.5   FlowMon probe init.d script

Startup scripts are used to start some service (or some script) at the machine's boot time. Our sample startup script is used to automatically start up the FlowMon probe (loads kernel modules, boots firmware and set up the probe behavior) anytime your PC is starting up. Sample script is stored in the package directory structure in the base/sys_sw/projects/flowmon/doc directory as a flowmon.rc file. It is prepared for use in the SysV init system (and tested on the Red Hat Linux distribution). Script is commented so you can get a lot of information directly from the script.

Startup script uses flowmon.conf to start Flow exporter(s).

7.1   Simple FlowMon testing

If you want to check FlowMon features you can simply mirror traffic from your router to the FlowMon probe.

+------------+
|  Internet  |
+------------+
      |
      |
      |
 +----------+      1 Gb/s       +---------------+           +-----------+
 |  Router  |-->------------->--| FlowMon Probe |-->----->--| Collector |
 +----------+ mirror     port 0 +---------------+    LAN    +-----------+
      |        port
      |
      |
+------------+
|  Intranet  |
+------------+

                Fig. 2 - Simple FlowMon probe connection schema

where

• mirror port : the copy of the traffic passing through the router
• port 0 : port 0 of MTX or SFP interface card (the closest one to the PCI slot)
• LAN : local network connection to collector

7.2   FlowMon Probe inserted in a line

You can also use the repeater functionality of the probe to connect it before your border router. This way you can monitor all the incoming or outgoing traffic.

 +------------+
 |  Internet  |
 +------------+
       |
       |
       | port 0
+---------------+           +-----------+
| FlowMon Probe |-->----->--| Collector |
+---------------+    LAN    +-----------+
       | port 1
       |
       |
 +------------+
 |   Router   |
 +------------+
       |
       |
       |
 +------------+
 |  Intranet  |
 +------------+

               Fig. 3 - FlowMon Probe inserted in a line

where

• port 0 : port 0 of MTX or SFP interface card (the closest one to the PCI slot)
• port 1 : port 1 of MTX or SFP interface card
• LAN : local network connection to collector

9.1   Mailing lists

For any questions or general technical support issues, please send mail to the netflow@liberouter.org general mailing list.

9.2   Submitting problem reports

Suggestions, bug reports and contributions of code are always valued. Please do not hesitate to report any problems you may find. If you encounter any suspicious behavior of your FlowMon probe (stops monitoring, reports non existing flows, runs too slow, reports only few flows, ... ) please run:

$ netflowctl_ph1 -c diagnose -F info

This will scan the line card and stores important information in following files (created in your current directory):

info_info
info_hsrch.mem
info_man.mem
info_sto.mem

Please attach these files to your bug report (usually they are too large to be send via email so please sends us the URL, so we can download them).

Bug reports with attached fixes are of course even more welcome.

9.3   Other COMBO6 based projects

Check our web pages for information about other projects, e.g. NIFIC (network interface card with packet filtering and forwarding) or intrusion detection system.

http://www.liberouter.org/