Skip to content. | Skip to navigation

Personal tools


You are here: Home / Wiki / Netfpga


Using NetFPGA programmable network devices in Emulab

Using NetFPGA programmable network devices in Emulab

The NetFPGA is now integrated into Emulab, and is available to external researchers for remote experimentation. You can allocate a NetFPGA in an ns file in a similar manner to how you allocate other Emulab resources. NetFPGAs can be configured as part of an arbitrary topology with end nodes, links and LANs, traffic generators, and other standard Emulab options.

Currently, we've installed 6 NetFPGAs into pc3000s (although we hope to install more to enable larger routing experiments). You can check if the NetFPGAs are available for your experiment by looking at the Node Control Center.

Getting Started with NetFPGAs in Emulab

To start using NetFPGA nodes in Emulab, you first need an account in Emulab. See Emulab Documentation for more details; especially if you are using Emulab for the first time. You will also want to familiarize yourself with how to program and interact with NetFPGAs; this guide is a good starting point.

Creating a NetFPGA experiment

Here's a sample ns file, which allocates a NetFPGA node and two end-nodes connected in a simple linear topology.

# Boilerplate
source tb_compat.tcl
set ns [new Simulator]

# Create a NetFPGA node
set nfrouter [$ns node]

# Set its hardware type to NetFPGA
tb-set-hardware $nfrouter netfpga2

# Name the NetFPGA host, bind it to the NetFPGA node, and
# set its OS.
set nfhost [$ns node]
tb-bind-parent $nfrouter $nfhost
tb-set-node-os $nfhost FC6-NETFPGA

# Create end nodes
set n0 [$ns node]
set n1 [$ns node]

# Create links between the end nodes and the NetFPGA
set link0 [$ns duplex-link $n0 $nfrouter 1Gb 0ms DropTail]
set link1 [$ns duplex-link $n1 $nfrouter 1Gb 0ms DropTail]

# Set up static routing on all nodes,
$ns rtproto Static

# Boilerplate
$ns run

When the above ns file is submitted, Emulab creates an experiment with the requested nodes, creates the topology specified above, and configures the NetFPGA nodes as described below.

  • We automatically reprogram the CPCI and upload the IPv4 reference router bitfile (created by researchers at Stanford University) to the NetFPGA board.
  • If you use static routing in your ns file, routes are automatically generated between all nodes; interface configuration information like IP addresses is generated for the NetFPGA interfaces as well as other nodes. Emulab uses this to configure the NetFPGA interfaces and initialize the routing and ARP tables.
  • Also, if you use static routing, we start up the SCONE userspace router, which handles exception packets from the router hardware, including ARPs, and maintains a copy of the routing table in userspace. SCONE also supports a subset of OSPF, named PW-OSPF, and automatically sends OSPF hello messages on the NetFPGA interfaces.

More complicated topologies with LANs, emulated links, routers, and more, can be similarly created to test your NetFPGA-based application.

Running your custom bitfile on the NetFPGA

To customize the router's functionality inside the hardware, you'll need to customize the existing gateware source, or write your own, and compile it into a bitfile as described in the NetFPGA guide. You can upload the bitfile to your NetFPGA via the command:

  /usr/local/bin/nf2_download <path-to-your-bitfile>

The FC6-NetFPGA image does not have the tools necessary to synthesize bitfiles installed by default, so you'll need to install them in your own image if you plan to use Emulab nodes to compile your source.

Node Usage Policies

Emulab has certain Node Usage Policies to facilitate sharing of resources among experimenters. Since we have limited number of boards available, please use resources in accordance with these policies.

Setup Notes

  • The NetFPGA PHY chips support only Gigabit ethernet. Consequently, you must connect them only to 1Gb links in your NS file. Since we don't support shaping on links with speeds faster than 100Mb, you also can't insert shape the link coming into your NetFPGA. If you really want to experiment with shaped links of 100Mb or below flowing into your NetFPGA, you can just add a "dummy" node between the real end node and the NetFPGA, and customize the link between the "dummy" node and the real end node.
  • NetFPGAs are not real Emulab PCs. Rather, they are "subnodes" of a host PC. This means that to customize the NetFPGA, you should login to its host PC.
  • Our default disk image for NetFPGA host PCs, FC6-STD, is based on Fedora Core 6, not CentOS (as suggested in the NetFPGA guide). Furthermore, we used a much newer kernel than suggested in the guide ( to build the nf2 driver.
  • We automatically configure the NetFPGA device when its host PC boots via the /usr/local/etc/emulab/netfgpactl script. This script obtains configuration information from Emulab (such as IP addresses, routes, etc.) for the NetFPGA ports, sets up the routing table, starts up SCONE, etc. It invalidates all existing entries in the routing and ARP tables before setting the routes.
  • The nf2 driver presents 4 user-visible (i.e., with ifconfig) interfaces for each NetFPGA card. These interfaces all have MAC addresses like 00:4E:46:32:43:0(0-N). However, the actual MAC chips also need their own real MAC addresses. To ensure consistent naming for these MAC addresses, we set the first byte to 0x02 (a reserved bit to indicate "local" address assignment), and change the fourth byte to the numeric Emulab id of the NetFPGA node (i.e., for "netfpga5", the 4th byte would be set to 0x05).