Skip to content. | Skip to navigation

Personal tools

Navigation

You are here: Home / Wiki / Kb97

Kb97

Emulab FAQ: Hardware Setup: What are the limits for link shaping?

Emulab FAQ: Hardware Setup: What are the limits for link shaping?

There are four characteristics of an emulated link that can be shaped: bandwidth, delay (latency), packet loss rate, and queue sizes. Shaping is performed either by dedicated shaping nodes or on the nodes themselves when using end node shaping. The fidelity (precision and accuracy) of shaping depends on the type of shaping used. The lowest fidelity is achieved when using multiplexed virtual nodes (which use virtual links and end node shaping), the highest with dedicated shaping nodes (the default). Mileage varies with end node shaping, depending on the applications in use, as shaping shares node resources (CPU cycles, memory) with the applications. The limits below apply only to dedicated shaping nodes.

Linktest can be used to verify characteristics of all links in an experiment, with possible limitations as explained in the list below.

  • Bandwidth. The maximum possible bandwidth of a link is 1000Mb/sec (aka "gigabit") and is available on more than half of our nodes (the pc3000s and d710s). The maximum value for which we have validated shaping is 100Mb/sec. Values between 100Mb and 1000Mb may be possible, but we have not attempted any characterization. Note that the bandwidths of 100Mb and 1000Mb are special in cases where no other shaping parameters are specified (i.e., 0ms delay, no loss). In these cases, no shaping is enabled (no shaping nodes allocated).
  • Delay. The minimum possible non-zero delay is 2ms. This is due to the scheduling granularity of the shaping nodes. The maximum possible delay depends on the available buffering on the shaping node and is proportional to the delay-bandwidth product of the links being shaped by that node. In the default configuration, a delay node has buffering for ~65,000 full-sized packets (the NMBCLUSTERS parameter of the FreeBSD kernel). With a delay node shaping only a single, unidirectional 100Mb/sec TCP stream (~8100 packets/sec) this would be adequate for about 8 seconds. Since packet buffers are a fixed size, extremely high packet rates can affect this dramatically. So a node shaping two bidirectional UDP streams (four "pipes") with an aggregate packet rate of 60,000 pps could handle about 0.25 second of delay per pipe. In general, delays of less than one second are recommended.
  • Packet Loss Rate. The packet loss rates can range from 0 to 1, with a minimum non-zero value of 0.000005. Recall that this is a probability, not a percentage; i.e., don't expect great things from a link with loss rate of 1.
  • Queue Size. Shaping "pipe" queue sizes cannot exceed 100 "slots" or one megabyte. For more on queues, see the advanced tutorial.