Why your maximum throughput is less than your bandwidth.

This posting is more technical than most on this blog.  It was motivated by a recent customer that has a 6Mbps MPLS network connecting Boston to China.  The network was been performing very well.  But then they recently started using FTP to transfer files and they could not attain anywhere close to 6Mbps of throughput.  This entry goes into the details.

TCP provides reliable transport of data between devices. The sender sends data in a stream in which bytes are identified by sequence numbers. The receiver acknowledges that it has received the data by sending the sequence number (acknowledgment number) of the next byte of data that it expects to receive. The receiver also advertises its TCP Window Size to the sender to advertise the amount of data that it can accept.

This “TCP Window” is the maximum number of bytes that can be transmitted by the sender without being acknowledged as having been successfully received by the other end. This means the receiver can safely expect that no more than window-size bytes of data need to be buffered for that connection, and that any additional data beyond that in sequence order can legitimately be discarded (because it is so defined by the TCP protocol).

When you have a high latency connection, such as satellite or a very long distance, the latency has a big effect on thru-put since TCP specifies that acknowledgements must be sent and received as the data is streaming.

How can you determine your maximum transfer rate?  The formula below will do the trick.  Just remember that this formula does not account for packet loss, so reduce the number to account for this:

Maximum Possible Transfer Rate = TCP Window Size/RTT

Where RTT is Ping Response in Millseconds/1000.

An example:

1.Ping time from NY to Los Angeles: 70ms. Therefore, RTT= 70/1000=.07

2.Maximum Possible Transfer Rate = 32/.07 = 457 KiloBytes/sec or 3.657 Mbps (Remember: 1 byte = 8 bits) This tells you that for a single data stream, such as FTP, a circuit larger than 3.657Mbps will not allow you to transfer a file any faster than 3.657Mbps.

3.It doesn’t matter how big your circuit is.

Here is another example:

1.Ping time from NY to Shanghai is 280ms.Therefore, RTT = 280/1000=.28

2.Maximum Possible Transfer Rate = 32/.28 = 114 KiloBytes/sec or 914Kbps.

3.If you have an E1/T1, you will have full access to 1540 Kbps, but with a single data flow, such as FTP, you will be limited to 914Kbps.

Now what if you make configuration changes to your TCP Window size to 64K:

For the NY to LA example, 64/.07=914 KiloBytes/sec

 For NY to Shanghai: 64/.28=228 KiloBytes/sec

So what do we learn from this?

There are only two things you can do to affect your data thru-put on a wide area network:

1.Increase your TCP window size, or

2.Reduce latency.

So a low latency connection makes a big difference on your thru-put. If latency is 1 second round-trip, the peak data rate can never exceed 65KB/second, which is 524Kbps, using a TCP Window of 65,535 bytes.

What size TCP window is needed to fill a 10Mbps circuit that has a 70ms latency between two locations? Here is the calculation:

 .07 seconds x 10Mbps x 1byte/8bits = 87,500 bytes required window size to use entire bandwidth with one data stream.

Standard TCP allows a maximum window size of 64,000 bytes. Cisco’s WINScale TCP option allows you to configure a larger window size.  You should experiment with this, since larger window sizes also mean larger retransmits when packets are lost, so a point of diminshing returns will be approached.

If you need more thru-put for FTP, run multiple sessions so that you can take advantage of the limitation that apply to a single data stream.

Here are some numbers to save you the calculations.  If you configure a 64KB TCP Window Size, your maximum thoughtput, without packet loss, is the following:

RTT Max Throughput Max Throughput Max Throughput 

Latency 64K Window 64K Window Windows XP 

 In Kbps In KBps In Kbps 

0.1 mS 803,755 Kbps 100,469.4 565,965 Kbps 

50 mS 10,371 Kbps 1,296.4 2,795 Kbps 

60 mS 8,658 Kbps 1,082.3 2,330 Kbps 

70 mS 7,431 Kbps 928.9 1,998 Kbps 

80 mS 6,509 Kbps 813.6 1,749 Kbps 

90 mS 5,790 Kbps 723.8 1,555 Kbps 

100 mS 5,214 Kbps 651.8 1,400 Kbps 

110 mS 4,742 Kbps 592.8 1,272 Kbps 

120 mS 4,349 Kbps 543.6 1,167 Kbps 

130 mS 4,016 Kbps 502.0 1,077 Kbps 

140 mS 3,730 Kbps 466.3 1,000 Kbps 

150 mS 3,482 Kbps 435.3 933 Kbps 

200 mS 2,614 Kbps 326.8 700 Kbps 

250 mS 2,093 Kbps 261.6 560 Kbps 

260 mS 2,012 Kbps 251.5 539 Kbps 

270 mS 1,938 Kbps 247.9 519 Kbps 

280 mS 1,869 Kbps 233.6 500 Kbps 

290 mS 1,804 Kbps 225.5 483 Kbps 

300 mS 1,744 Kbps 218.0 467 Kbps 

350 mS 1,496 Kbps 187.0 400 Kbps 

400 mS 1,309 Kbps 163.6 350 Kbps 

450 mS 1,164 Kbps 145.5 311 Kbps 

500 mS 1,047 Kbps 130.9 280 Kbps 

 

I hope that this post adds some clarity to an often misunderstood concept.  This entire concept changes when you include Network Optimization or Network Acceleration in your wide area network.  This technology can increase your throughput on by several 100% for many types of files or applications, improving performance and reducing the need for expensive WAN bandwidth.  If you want faster WAN performance, you truly need to look at WAN Optimization or WAN Acceleration.  The long term payoff in productivity and lower bandwidth cost over time is clear.  MPLS-Experts can help you make the right decision and obtain the best prices

source link: http://www.mpls-experts.com/why-your-maximum-thru-put-is-less-than-your-bandwidth/