Bandwidth Delay Product

TCP is one of those protocols that we usually don’t think about too much. As network engineers we are busy working with network devices like routers or switches. TCP is one of those protocols that is used most between hosts or servers and it works without giving it much thought. It establishes connections, transmits data, sends acknowledgments and when something goes wrong…it retransmits it.

TCP uses a sliding window size that indicates how much the receiver is willing to receive from the sender. Depending on the receive buffer and network conditions, this window size will increase or decrease as needed. The larger the window size, the higher the throughput will be. With a window size of 1, the receiver would send an acknowledgment for each segment that it receives which results in a lot of overhead.

Host to server satellite connection

This “stop and go” mechanism of TCP works very well “out of the box” but on certain links, TCP might require some tuning. This is especially true on so called long fat networks (LFN).

A LFN is a network that offers a high bandwidth but also a very high delay. An example could be a satellite connection. These connections offer a high bandwidth but the delay is also quite high since you have to send your signal 22000 miles up to the satellite and another 22000 miles down to reach the receiver. You can expect a round trip time anywhere between about 500-1000 ms.

The problem here is that when the sender sends some data, it has to be wait a very long time for an acknowledgment of the receiver before it can send the next data. During the time we are waiting, nothing happens so we don’t utilize the full bandwidth of our link.

The throughput of TCP is limited by the round trip time of the link and the window size. We can’t change the round trip time but we can play with the window size. Take a look at the image below:

TCP Long Fat Network Low Window Size

Imagine we send some data from the host to the server, when this piece of data is on its way we have to wait a long time before it reaches the server and for the acknowledgment to come back. A lot of bandwidth is wasted. This is what happens with a large window size:

TCP long fat network high window size

With a large window size, we can fill the entire “pipeline” with data. We don’t waste anything.

When you are using a 5 Mbit satellite link and you have a transmission rate of 1 or 2 Mbit of TCP traffic, you probably have some TCP tuning to do.

The most optimal window size depends on the bandwidth and delay of the link, we call this the bandwidth delay product. We can calculate it with the following formula:

Bandwidth Delay Product = bandwidth (bits per sec) * round trip time (in seconds)

So for example, let’s calculate the bandwidth delay product of a satellite link that has a round trip time of 500 ms:

5000000 bits * 0.5 seconds = bandwidth delay product 2500000

So our bandwidth delay product is 2500000 bits. The window size is typically configured in bytes so 2500000 / 8 would be 312500 bytes.

Here are some other examples:

ADSL 2 Mbit with 50 ms round trip time:

2000000 bits * 0.05 seconds = bandwidth delay product 100000 bits (or 12500 bytes)
ADSL2 20 Mbit with 50 ms round trip time:

20000000 bits * 0.05 seconds = bandwidth delay product 1000000 bits (or 125000 bytes)
FastEthernet LAN Interface with 1 ms round trip time:

100000000 bits * 0.001 seconds = bandwidth delay product 100000 bits (or 12500 bytes)
Gigabit LAN Interface with 1 ms round trip time:

1000000000 bits * 0.001 seconds = bandwidth delay product 1000000 bits (or 125000 bytes)

Are there any downsides to increasing the TCP window size? One thing to consider is that by increasing the window size, you also need a large receive buffer but this
shouldn’t be much of a problem on any modern hardware. Also with a larger window size you will have a lot of data “in transit” so if you have any errors on the link,
there’s a lot of data to retransmit.

iPerf Demonstration

Once you have calculated the bandwidth delay product, you should test if it works. A nice way to test this is by using iPerf. This application allows you to generate TCP traffic with different window sizes. To demonstrate this, I’ll use two hosts:

Iperf Client Server Switch

These two hosts are connected through a gigabit link so this is a high bandwidth low delay link. Even though the round trip time is low, we still have to use a decent window size to get some decent performance.

TCP is one of those protocols that we usually don't think about too much. As network engineers we are busy working with network devices like routers or switches. TCP is one of those protocols that is used most between hosts or servers and it works without giving it much thought. It establishes conne

A quick ping tells us the round trip time:

We're Sorry, Full Content Access is for Members Only...

If you like to keep on reading, Become a Member Now! Here is why:

  • Learn any CCNA, CCNP and CCIE R&S Topic. Explained As Simple As Possible.
  • Try for Just $1. The Best Dollar You've Ever Spent on Your Cisco Career!
  • Full Access to our 662 Lessons. More Lessons Added Every Week!
  • Content created by Rene Molenaar (CCIE #41726)

519 Sign Ups in the last 30 days

satisfaction-guaranteed
100% Satisfaction Guaranteed!
You may cancel your monthly membership at any time.
No Questions Asked!

Tags:


Forum Replies

  1. Hi Zaman,

    Originally the window size is a 16 bit value so the largest window size would be 65535. We couldn’t add more bits to the TCP header but it was possible to reassign the purpose of those 16 bits.

    What we do nowadays is that we use a scaling factor so that we can use higher window sizes.

    For example, the window size value is 400 and the scaling factor is 64.

    400 x 64 = 25600

    In my lesson one of the screenshots also showed a windows size of 132480.

    Window size value = 2070
    Window size scaling factor = 64

    2070 x 64 = 132480

    Here’s a short explanation of th

    ... Continue reading in our forum

  2. Hello Hussein!

    First lets take a look and see what is meant by the window size: The window size indicates the size of a device’s receive buffer for the particular connection. In other words, window size represents how much data a device can handle from its peer at one time before it is passed to the application layer. This buffer size can change based on the hardware being used (physical memory available on the NIC for buffering for example) as well as by the total number of TCP sessions the device is taking part in at any given time. Of course this window s

    ... Continue reading in our forum

  3. Hello Azm

    Yes, these numbers can be confusing. Here is an attempt to clarify these parameters:

    The first thing to keep in mind is that in any TCP communication, there are actually TWO sequence numbers and TWO acknowledgement numbers: those of each party in the exchange of data. For the sake of this example, and for the diagram below, let’s call these SNL and SNR for Sequence Number Left and Sequence Number Right for the left and right hosts. Similarly, the acknowledgement numbers will be called ANL and ANR.

    Note, these abbreviations are my own and are not ge

    ... Continue reading in our forum

  4. Hello AZM

    I can understand the confusion. Keep in mind that the window size, the sequence number and the number of segments sent are somewhat independent from each other. What do I mean?

    Well, let’s say we have a window size of 21000 bytes. It is very unlikely that this will all be sent in one segment. It will definitely be split into several segments. The window size is “the number of bytes sent before an acknowledgement is required from the receiver.” These bytes can be sent in one or more segments. So, let’s take the following example:

    Host A is sendi

    ... Continue reading in our forum

  5. Hello Juan

    Just like any other protocol communicating on the network, BGP requires the appropriate MTU sizes to be set in order for communication to occur successfully. If you have to tune the MTU value to get BGP to work then it seems that BGP is sending IP packets larger than the interface MTU that have the DF set to 1, which means to not fragment. By adjusting the (IP or interface MTU) of the subinterface, you are essentially adjusting the allowable MTUs such that the IP MTU will be small enough to fit into the interface MTU. It also depends on what other

    ... Continue reading in our forum

54 more replies! Ask a question or join the discussion by visiting our Community Forum