Skip to content. | Skip to navigation

Personal tools


You are here: Home / Wiki / Multiplexedlinks


From Tue Apr  1 11:11:13 2003
Date: Tue, 1 Apr 2003 11:03:10 -0700 (MST)
From: Mac Newbold <>
To: Testbed Project <>
Subject: routing/shaping w/multiplexed links

Here's notes from yesterday's discussion, and the proposed solution.

What works now:

A node A has 12 shaped virtual links (duplex links, not lans) multiplexed
over 4 physical links. This works today, because all shaping in the
non-LAN case happens on sending node. So it doesn't matter which virtual
link the packet arrives on, since it doesn't affect routing or shaping.

What doesn't work yet, but would be fixed by the proposed solution:

1) shaped virtual multiplexed LANs
 - For LANs, we have to do half of the shaping on the outgoing packet, and
   the other half on the incoming packet. In order to do that, we need to
   know which vlink it is coming in on to do the right shaping.

2) multiple multiplexed links between the same two nodes
 - You can't figure out which pipe it needs to be shaped by if there are
   multiple vlinks/LANs between the same two nodes multiplexed on the same

3) multiplexed links into a node with jails
 - The phys. node needs to know which jail it belongs to, and which
   routing table to use. If it is a shaped LAN, it also needs to figure
   out which pipe to shape with.

The proposed solution:

The kernel can be modified to use information from the ethernet packet
(specifically, the source MAC hardware address) to demultiplex. The
receiving node can figure out what "virtual IP" the packet came to, and
thus, which pipe and/or jail it came to, by looking up the source MAC
ethernet hardware address in a table (ARP table, via RARP, or via a table
generated and distributed apriori by emulab). The ARP and RARP approaches
are most desirable, since they are not emulab specific, and simplify
operation inside emulab.

This works without changing any MAC addresses except in the case where
there may be more than one virtual link on the source node that connects
to the recieving node (probably an Emulab-only case, not something found
in the real world). In this case, each virtual link would need a
distict MAC address. We can do that by inventing MAC addresses for every
virtual link endpoint. This set of endpoints is the same set of endpoints
that we have to make up IP addresses for. So a one-to-one mapping of IP
addresses and fake MAC addresses seems suitable.

Chad and Rob suggested a good way to make these MAC addresses, that
eliminates the need for a table. The top two bytes of the six byte MAC
addr can be used for a tag that identifies a unique experiment in emulab.
The lower four bytes can be the IP address assigned to that interface.
Both nodes involved would have the new MAC addrs, and basically perform a
proxy arp game for all of the MACs assigned to that interface. Then when a
virtual link wanted to send a packet, the ethernet src MAC would have the
sending IP in it, and the destination MAC would have the IP of the
destination in it, and demultiplexing becomes trivial. This also works
when there are multiple IP addresses in the same subnet assigned to the
same interface, since we know which of the IPs it was intended for.

The proposed meeting:

We've got a 1:30pm round table, and I've got a 4pm commitment, so the
proposed time (discussed w/Leigh and Mike already) is 2:30pm today. As Jay
pointed out, we probably will not be able to use the CSL lib. but there
are only about 5-6 people that need to come to this meeting anyway. So I
nominate the south end ("snack room") table as the alternate location.


Mac Newbold		Univ. of Utah School of Computing