“Is the Internet slow? why is it taking so long for me to get to the file server?”, “Someone needs to reboot the internet!”. I know you have either heard or uttered those phrases in the last month or two; I know I have. But how do you measure ‘it is slow’? Slow network connections to one person, could be screaming fast for another person; and vise versa. Well, thankfully there is a tool for that; in comes iperf. Iperf was designed by the Distributed Applications Support Team (DAST), out of the National Laboratory for Applied Network Research(NLANR); designed not only measure network bandwidth, but also optimising the network by ‘tuning’ the TCP window size on the host operating system. This article will be focusing on the former; measuring network bandwidth. With Kernel 2.6.7, automatic send/receiver buffer tuning was put into place, thus mainly negating the need for fine-tuning the network on a host.
But what exactly does Iperf do? Well, it’s quite simple really. Iperf utilises the client/server architecture, sending a select amount of data from an iperf client to a listening iperf server, and measuring the time that it takes to transmit/receive the data. Not only can iPerf measure the total bandwidth from client to server, but it can also measure bandwidth being sent from the server back to the client in both a ‘trade-off’ (wait until client test runs, then run data back from server to client) or a dual test transfer of data, both sending and receiving of data at the same time. But what if you want to test multiple simultaneous connections to the server? Mimicking multiple clients connecting to a server at the same time? Iperf allows (if threading is enabled) for an iperf client to run multiple simultaneous connections with the -P flag, of which I will demonstrate later in this article. Not only can iperf measure network bandwidth, and optimising networks; but it can easily be a valuable tool for network administrators that are troubleshooting congestion and bottlenecks between various computers on the network, including wireless clients, Point-to-Point T1′s and VPN tunnels between various secured sites.
So where do you get such an awesome tool? Well, most modern Linux distributions contain the packages in their respective distributions package manager (be it apt-get, or rpm). Not only can you run the client or server from a Linux machine, but for those of you who have a mixed-client environment; windows binaries are available. And for those of you that are not CLI inclined, there is a full blown java graphical utility out there as well. You can get all of these versions from sourceforge. Now before we get started; not only will the window’s binary act as a client, but also as a server. For those of you who are in a mixed-OS environment, this can be beneficial in testing client access to various Window’s servers. The Windows Binary allows the network administrator to actually run the server as a running service (just be sure that you disable the service once you have completed testing). So without further adieu, let’s get started!
Before we jump into iperf with both feet, I’ll be holding your hand for the first part of the trip. First things first, running the iperf Server. The easiest way to get started with running Iperf is to execute the following: iperf -s as root, or any user that has privileges to open TCP connections over 1024. This will start the iperf server, listening on TCP Port 5001. During testing purposes it is advisable to either open up the port in iptables (iptables -A allowed -p tcp –dport 5001 -s xxx.xxx.xxx.xxx/xx -j ACCEPT *replace xxx with network information) or disable iptables on your internal server/clients during the test. Other Server options include the ability to run as a daemon, and allow a specific number of connections before closing the application (this is default to 0, accepting all connections indefinitely until the application is closed), and listen on a different TCP Port. There are other options as well, and different options between the Windows Server binary and the linux (such as the ability to write output to a file with the windows binary), see the iperf documentation that is included in the source .tar.gz for more server options than listed here.
Startup Options -s --Start as Server -D --Run as Daemon -p --Server Port to listen on -u --listen to UDP port 5001
Once you execute the iperf -s command, you should be presented with a screen as shown above. This little vital tidbit of information states the following: The Server is listening on TCP Port 5001, the TCP window size is set to 85.3 KBytes as default. As you can also see in the screenshot, any tests that are started by a client, is also echoed to the Server for statistics gathering. This is very useful when you have multiple clients running parallel tests; and you want to gather the information in a central location to further analyse at a later date.
So, now that the Server is running and listening for incoming connections, lets start up a client connection on the same machine. In a second terminal window on the same machine (for testing purposes, and ease of use; I opted to run the client/server localhost test in a screen session), run the following command: iperf -c 127.0.0.1. If everything is working well, you should get returned data with your bandwidth of your localhost. In this case, I also opted to use the -i flag to display a 3 second interval between the tests; instead of just 1 output after the 10 seconds of testing. Personally I find it nice to be able to check on any deviation in packet loss or bandwidth during testing.
From this screenshot we can see the following: The client connected to the Server running on the localhost via TCP Port 5001, TCP window size for the client was 49.5KBytes, [ID] of 3 allows you to view the status reports of individual connections on the Server if you are running multiple Parallel connections to the Server (as specified by the -P switch). The interval is just that, it specifies the interval of the reporting. Because I used the -i 3 flag, this reports intervals between 3 seconds, and then a final output of 0-10 seconds. Transfer is the amount of data that was transferred between the client and the Server, and finally bandwidth is the total bandwidth detected during the tests.
Now that we have proven that we can retrieve results from the Server, lets try this in a LAN environment.
LAN & Wireless Testing
There are many reasons why you might use the Iperf utility on a physical LAN environment; testing bandwidth between clients at the far end of a building to an intranet server between multiple switch hops, testing a Point-to-Point T1, and even testing a VPN Tunnel for available bandwidth between the client and the server. Not only can you check TCP utilisation, but you can also check UDP loss and delay. UDP is used in a lot of business applications, including but not limited to DNS services, Voice-over-IP, and Video Conference calls. If your having network bottleneck issues, this could seriously degrade performance on a VoIP, or other UDP traffic.
The procedures for running iperf on a client on the LAN is just like our examples earlier with running it against the localhost, but this time you can also run other switches to gather more in-depth data. Your basic: iperf -c ip.of.example.server -i 3 will result in a more accurate reading of your bandwidth between a server and client.
As you can see from these results, the amount of data transferred is smaller than in the localhost, along with receiving a more reasonable Bandwidth of 94.1 MBits/Sec. Now if for whatever reason, you would like to change the output from MBits/sec to some other format, the -f (–format) switch used during the execution of the client phase, will change the human readable output of your bandwidth.
'b' = bits/sec 'B' = Bytes/sec 'k' = Kbits/sec 'K' = KBytes/sec 'm' = Mbits/sec 'M' = MBytes/sec
Earlier in the article, I talked about the ability to run both a dual test, and a ‘trade off’ test. A dual test from the client to the server allows you to run a simultaneous test of both up and down traffic, testing the bandwidth connection of sending and receiving data from the client and the server. This allows you to see a ‘real world’ example of simultaneous TCP connections by using the -d (–dualtest) option in iperf: iperf -c ip.of.example.server -d.
From this example, you can see that my submission of data from the server to the client was significantly less (15.3Mbits/Sec) than data transferred from the client to the server. Now the other test that you can run, is the trade off mode, utilising the -r (–tradeoff) flag. This flag tells the server that once the client connection has closed, to open another connection from server to client and send data, running an alternating test for throughput. An example of this result is below. From these results you can see that the data itself didn’t change much from the original test..both ‘up’ and ‘down’ traffic stayed within the normal bandwidth of a 100MBit LAN connection.
If you want to test simultaneous connections to the server from a specific client, you can use the -P flag, with a number after the -P specifying the number of connections to make to the Server. In order for this to work, thread support needs to be compiled into the version of iperf that you are using. If you are using a pre-packaged binary odds are that thread support was compiled in (both ubuntu & fedora have thread support in their binaries); this can be checked with: ldd /usr/bin/iperf. If you installed from source, you need to specify –enable-threads. I see the -P flag used more as a benchmarking utility, being able to make multiple connections to the server at the same time is sure to display a much lower Bandwidth.
Now what about all that talk about UDP? Well, for these tests you need to change the server options on iperf to listen in for UDP connections. But before you do, as before; you should make sure that ipchains is allowing UDP traffic through port 5001. After this is accomplished, start up iperf server again; but this time with the -u flag (iperf -s -u). Upon executing, you should see a banner that states iperf is listening on udp port 5001.
Just as before, the output on your server shows all of the connections from client to server. But while testing UDP, you will come across extra rows. This is the Jitter (Variation of latency packets between hosts received in milliseconds), and the Lost/Total Datagram packets with a percentage of lost packets. As a default setting, iperf sets the datagram size to 1470 bytes; but for more accurate testing, you should set the receiving datagram size to that of the application that you are testing (-l datagramsize). If using the -i feature for intervals, as before, you will see intervals printed out to the screen.
And of course, last but not least; Wireless testing. Remember the day’s when wireless was the de-facto ‘must have’ for businesses? Everyone should go wireless, laptop users, power users, training environments. Why? Because it’s new, and fancy technology without wires. Although I never could understand why a company would chose to put wireless desktops in place for power users; but to each their own. With multiple wireless clients connecting to a single wireless router/access point, to a specific server; comes the possibility of a bottleneck. This is where iperf really shines, with the wireless N-Draft of 600Mbps, wireless just might replace the 100Mbps wired ethernet connection. But before you go putting power users (such as people who rely heavily on databases accessed through the intranet), onto a wireless connection; you should first do some bandwidth testing.
Now to setup a proper wireless environment for testing; it’s all about the placement of the server. If you are setting up an environment where wireless clients will be accessing a specific server or service on the intranet, your iperf server should be located closest to the server in question. If you are just testing the bandwidth utilization of multiple wireless clients simultaneously (or in parallel) accessing the internet (or intranet resources); then your iperf server should be on the network closest to the access point. If you just want to test wireless client-to-client communication, you can place the iperf server on 1 wireless client and access it with another wireless client running the iperf client switch. The best way to test wireless throughout is to run a client on the machines accessing the intranet service, and run trade off connections to the iperf server, simulating network traffic of multiple wireless clients accessing the intranet resource. Now don’t forget, if you plan on using your wireless client for VoIP, Video Conferencing, or any other application that relies heavily on UDP; to test your bandwidth via the UDP option.
So, as you can see; iperf can be a very valuable tool in the network administrator’s bag of tricks. I have barely scratched the surface of just what iperf can do. There are other options available to the network administrator once you have measured the TCP and UDP bandwidth of the individual clients, including Tuning the TCP connection on the individual client operating system, running multicast clients, and testing against IPv6 networks. In this article, I didn’t get into TCP buffer size tuning in linux due to the fact that with the 2.6.x kernel, there is a sender-side and client-side autotuning enabled in the kernel (2.4.x kernels have sender-side tuning enabled). But with other operating systems, there may be some fine-tuning of the TCP buffer sizes that you can read more about if you are so inclined to take a leap on the wild side. The index of URL’s at the end of this article will guide you through other Operating System changes for TCP window sizes. And as always, there is plenty of documentation that comes with the iperf source code (about 13 pages of the iperf Users Doc) that explains even futher in depth about topics that were not covered in this article.
http://sourceforge.net/projects/iperf/ – Download Iperf
http://fasterdata.es.net/TCP-tuning/background.html – TCP Tuning Guide for different operating systems