TCP Window Size Scaling

TCP (Transmission Control Protocol) is a connection oriented protocol which means that we keep track of how much data has been transmitted. The sender will transmit some data and the receiver has to acknowledge it. When we don’t receive the acknowledgment in time then the sender will re-transmit the data.

TCP uses “windowing” which means that a sender will send one or more data segments and the receiver will acknowledge one or all segments. When we start a TCP connection, the hosts will use a receive buffer where we temporarily store data before the application can process it.

When the receiver sends an acknowledgment, it will tell the sender how much data it can transmit before the receiver will send an acknowledgment. We call this the window size. Basically, the window size indicates the size of the receive buffer.

Typically the TCP connection will start with a small window size and every time when there is a successful acknowledgement, the window size will increase. Here’s an example:

TCP Window size 1

Above we have two hosts, the host on the left side will send one segment and the host on the right side will send an acknowledgment in return. Since the acknowledgement was successful, the windows size will increase:

tcp window size 2

The host on the left side is now sending two segments and the host on the right side will return a single acknowledgment. Everything is working fine so the window size will increase even further:

TCP Window size 4

The host is now sending four segments and the host on the right side responds with a single acknowledgment.

In the example above the window size keeps increasing as long as the receiver sends acknowledgments for all our segments or when the window size hits a certain maximum limit. When the receiver doesn’t send an acknowledgment within a certain time period (called the round-trip time) then the window size will be reduced.

When an interface has congestion then it’s possible that TCP packets are dropped. To deal with this, TCP has a number of algorithms that deal with congestion control. One of them is called slow start.

Congestion occurs when the interface has to transmit more data than it can handle. It’s queue(s) will hit a limit and packets will be dropped.

With TCP slow start, the window size will initially grow exponentially (window size doubles) but once a packet is dropped, the window size will be reduced to one segment. It will then grow exponentially again until the window size is half of what it was when the congestion occurred. At that moment, the window size will grow linearly instead of exponentially.

When an interface gets congested, it’s possible that all your TCP connections will experience TCP slow start. Packets will be dropped and then all TCP connections will have a small window size. This is called TCP global synchronization. Here’s what it looks like:

TCP Global Synchronization

The orange, blue and green lines are three different TCP connections. These TCP connections start at different times and after awhile, the interface gets congested and packets of all TCP connections are dropped. What happens is that the window size of all these TCP connections will drop to one and once the interface congestion is gone, all their window sizes will increase again.

The interface then gets congested again, the window size drops back to one and the story repeats itself. The result of this is that we don’t use all the available bandwidth that our interface has to offer. If you look at the dashed line you can see that we the average interface utilization isn’t very high.

To prevent global synchronization we can use RED (Random Early Detection). this is a feature that drops “random” packets from TCP flows based on the number of packets in a queue and the TOS (Type of Service) marking of the packets. When packets are dropped before a queue is full, we can avoid the global synchronization.

The end result will look similar to this:

TCP Global Synchronization Red

When we use RED, our average interface utilization will improve.

Now you have an idea what the TCP window size is about, let’s take a look at a real example of how the window size is used. We can use wireshark for this.

Wireshark Captures

To examine the TCP window size I will use two devices:

Host Raspberry Pi

The device on the left side is a modern computer with a gigabit interface. On the right side, we have a small raspberry pi which has a FastEthernet interface. The raspberry pi is a great little device but it’s cpu / memory / ethernet interface are limited. To get an interesting output, I will copy a large file through SSH from my computer to the raspberry pi which will be easily overburdened.

Here’s what happened, take a look at this picture:

Wireshark IO graphs window size drop

In the graph above you can see the window size that was used during this connection. The file transfer started after about 6 seconds and you can see that the window size increased fast. It went up and down a bit but at around 30 seconds, it totally collapsed. After a few seconds it increased again and I was able to complete the file transfer. Let’s take a closer look at this file transfer, which starts with the three way handshake:

Wireshark capture TCP SYN ACK Window Size

My fast computer uses 10.56.100.1 and the raspberry pi uses 10.56.100.164. Above you can see that in the SYN,ACK message that the raspberry pi wants to use a window size of 29200. My computer wants to use a window size of 8388480 (win=65535 * ws=128) which is irrelevant now since we are sending data to the raspberry pi.

After a few packets, the window size of the raspberry pi looks like this:

Wireshark Capture TCP SYN ACK Window Size Large

Above you can see that the window size has increased to 132480. Originally the window size is a 16 bit value so the largest window size would be 65535. Nowadays we use a scaling factor so that we can use larger window sizes.

At around the 10 second mark the window size decreased. Here’s what happened:

Wireshark Capture TCP Window Full

The raspberry pi seems to have trouble keeping up and its receive buffer is probably full. It tells the computer to use a window size of 26752 from now on. The computer sends 18 segments with 1460 bytes and one segment of 472 bytes (26752 bytes in total). The last packet shows us “TCP Window Full” message. This is something that wireshark reports to us, our computer has completely filled the receive buffer of the raspberry pi.

Once the raspberry pi has caught up a bit and around the 30 second mark, something bad happens. Take a look at the wireshark capture below:

Wireshark Capture TCP Window Zero

Above you can see that the raspberry pi sends an ACK to the computer with a window size of 0. This means that the window size will remain at 0 for a specified amount of time, the raspberry pi is unable to receive any more data at this moment and the TCP transmission will be paused for awhile while the receive buffer is processed.

Here’s the actual packet:

Wireshark Capture TCP Window Zero Packet

Above you can see that the window size is now 0. Once the receive buffer has been processed, the raspberry pi will send an ACK with a new window size:

Wireshark capture window size after zero

The window size is now only 25600 bytes but will grow again. The rest of the transmission went without any hiccups and the file transfer completed.

Conclusion

You have now seen how TCP uses the window size to tell the sender how much data to transmit before it will receive an acknowledgment. I also showed you an example of how the window size is used when the receiver is unable to process its receive buffer in time.

UDP, unlike TCP is a connectionless protocol and will just keep sending traffic. There is no window size, for this reason you might want to limit your UDP traffic or you might see starvation of your TCP traffic when there is congestion.

I hope you have enjoyed this lesson, if you have any more questions feel free to leave a comment in our forum.

Forum Replies

  1. Hi Zaman,

    Originally the window size is a 16 bit value so the largest window size would be 65535. We couldn’t add more bits to the TCP header but it was possible to reassign the purpose of those 16 bits.

    What we do nowadays is that we use a scaling factor so that we can use higher window sizes.

    For example, the window size value is 400 and the scaling factor is 64.

    400 x 64 = 25600

    In my lesson one of the screenshots also showed a windows size of 132480.

    Window size value = 2070
    Window size scaling factor = 64

    2070 x 64 = 132480

    Here’s a short explanation of th

    ... Continue reading in our forum

  2. Hello Hussein!

    First lets take a look and see what is meant by the window size: The window size indicates the size of a device’s receive buffer for the particular connection. In other words, window size represents how much data a device can handle from its peer at one time before it is passed to the application layer. This buffer size can change based on the hardware being used (physical memory available on the NIC for buffering for example) as well as by the total number of TCP sessions the device is taking part in at any given time. Of course this window s

    ... Continue reading in our forum

  3. Hello Azm

    Yes, these numbers can be confusing. Here is an attempt to clarify these parameters:

    The first thing to keep in mind is that in any TCP communication, there are actually TWO sequence numbers and TWO acknowledgement numbers: those of each party in the exchange of data. For the sake of this example, and for the diagram below, let’s call these SNL and SNR for Sequence Number Left and Sequence Number Right for the left and right hosts. Similarly, the acknowledgement numbers will be called ANL and ANR.

    Note, these abbreviations are my own and are not ge

    ... Continue reading in our forum

  4. Hello AZM

    I can understand the confusion. Keep in mind that the window size, the sequence number and the number of segments sent are somewhat independent from each other. What do I mean?

    Well, let’s say we have a window size of 21000 bytes. It is very unlikely that this will all be sent in one segment. It will definitely be split into several segments. The window size is “the number of bytes sent before an acknowledgement is required from the receiver.” These bytes can be sent in one or more segments. So, let’s take the following example:

    Host A is sendi

    ... Continue reading in our forum

  5. Hello Juan

    Just like any other protocol communicating on the network, BGP requires the appropriate MTU sizes to be set in order for communication to occur successfully. If you have to tune the MTU value to get BGP to work then it seems that BGP is sending IP packets larger than the interface MTU that have the DF set to 1, which means to not fragment. By adjusting the (IP or interface MTU) of the subinterface, you are essentially adjusting the allowable MTUs such that the IP MTU will be small enough to fit into the interface MTU. It also depends on what other

    ... Continue reading in our forum

53 more replies! Ask a question or join the discussion by visiting our Community Forum