~ / sysadmin / networking / netflow

Troubleshooting a network with NetFlow

NetFlow is usually associated with large routers, but the same idea works on ordinary Unix hosts: capture flow records on the machine that sees the traffic, send them to a collector, and query the result.

When this helps

A link is saturated, a service is slow, or something is hammering a host and nobody knows what. Instead of guessing, the flows tell you which addresses and ports account for the traffic over a chosen window.

The probe side

The machine that sees the traffic runs a small agent that turns packet headers into flow records and forwards them to a collector. On Linux, one option is a probe that reads packets logged by the firewall.

apt install fprobe-ulog
# then set the collector address and port in its configuration

The firewall is then told to copy the traffic it sees to the probe.

iptables -A INPUT   -j ULOG --ulog-qthreshold 50 --ulog-cprange 48
iptables -A FORWARD -j ULOG --ulog-qthreshold 50 --ulog-cprange 48
iptables -A OUTPUT  -j ULOG --ulog-qthreshold 50 --ulog-cprange 48

The collector side

The collector receives the records and writes them to files, one per time interval, which can then be queried.

apt install nfdump
# enable the collector in its default configuration

Asking a useful question

The classic query lists the busiest conversations in a time window, ordered by throughput, which immediately shows the heavy hitters.

nfdump -R /var/cache/nfdump/nfcapd.202402271818 \\
  -s record/bps -n 20 -o extended \\
  -t 2024/02/27.02:00:00-2024/02/27.02:05:00

Care with the data

Flow records say who talked to whom. Treat the collected files as sensitive, keep them only as long as the investigation needs them, and restrict who can read them.

The collector and probe projects keep their own documentation; a starting point is on the resources page.