Skip to content. | Skip to navigation

Personal tools


You are here: Home / Wiki / Secure


Running an Experiment in a

Running an Experiment in a Secure Environment


Originally, the goal of an Emulab experiment was to provide an isolated environment in which to run tests. Isolation here primarily meant resource isolation--preventing artifacts in an experiment due to other experiments or outside influences. While basic authentication and protection mechanisms were used, the threat model being addressed was accidental "attacks" on isolation; e.g., a misconfigured interface causing flooding of another experiment's network. We are now building up the Emulab infrastructure to allow experimentation with more potent threats, in particular malware, which attempts to actively exploit weaknesses on nodes and in the network.

Since Emulab is intended for use by researchers, we did not unnecessarily want to restrict access from the Internet to experimental nodes and vice-versa. Thus the central Emulab firewall is fairly permissive. For high-risk experiments, this is not acceptable. To address this, we have added per-experiment control net firewalls.

Another decision made early on, for the convenience of users, was for extensive use of shared infrastructure such as a shared filesystem and a central login machine within Emulab allowing for efficient control of experiments. Such shared infrastructure provides an easy target for malware, so through the use of [elabinelab.php3 Emulab in Emulab] we provide per-experiment Emulab infrastructure.

By combining the two facilities, we enable containment of high-risk experiments without sacrificing the features that make Emulab so easy to use.


In your NS file you can specify a security level at which an experiment should be run. Security levels are specified as colors (oh, now that is real original!) with the tb-set-security-level command. Colors are a way to conveniently configure a firewall with a known, fixed ruleset. If you use tb-set-security-level then you cannot modify the implied firewall (e.g., by using "add-rule"), nor can you allocate your own firewall. The exact configuration of the firewalls implied by the security level, is still a work-in-progress, but the current meanings are:

  • Green. The default for all experiments. No firewall is allocated. Hence all nodes are still on the shared control network.
  • Blue. A basic style firewall is allocated. This configuration is intended for preventing bad stuff from getting in, not to prevent it from getting out. The only practical implication right now is that, at swapout time, nodes in a Blue experiment do not undergo the rigorous decontamination process that all higher security levels (and explicit firewalls) require. This security level is appropriate for running a Windows node in while you customize it, without needing to worry about it becoming infected.
  • Yellow. Currently allocates a basic style firewall. All nodes going through the swapout decontamination process.
  • Orange. A closed style firewall is allocated. All nodes going through the swapout decontamination process.
  • Red. Not currently implemented. This will eventually be an experiment for which the control network has been completely disabled. The only outside access allowed will be via the serial console line.

You can explicitly combine a per-experiment Emulab with an "Orange" experiment to get the highest level of protection we currently offer. It further restricts access from the experiment to the "real" Emulab infrastructure (e.g., no NFS allowed to the real "fs" node). Please note that this configuration currently takes over 20 minutes to setup, regardless of the size of the experiment!

You may specify the type of firewall to use with the tb-set-elabinelab-fw-type command. Currently two values are valid:

  • ipfw2-vlan: This is a FreeBSD-based firewall using ipfw2.
  • iptables-vlan: This is a Linux-based firewall using iptables.

If you do not use the tb-set-elabinelab-fw-type command, the ipfw2-vlan type will be used by default.


See the firewall Limitations section.

Known Bugs

See the firewall Known Bugs section.

A Couple of Examples


	source tb_compat.tcl
	set ns [new Simulator]

	tb-set-security-level Yellow

	set n1 [$ns node]
	tb-set-node-os $n1 FBSD-STD
	set n2 [$ns node]
	tb-set-node-os $n2 RHL-STD
	set link [$ns duplex-link $n1 $n2 100Mb 0ms DropTail]

	$ns run

is nearly equivalent to the firewall example except that there are no additional firewall rules to allow traceroute.

To setup a high-security prison for running a Windows XP experiment you could do:

	source tb_compat.tcl
	set ns [new Simulator]

	tb-elab-in-elab 1
	tb-set-security-level Orange

	tb-set-inner-elab-eid winxpnodes

	$ns run

This will setup a firewalled, experiment-private Emulab in which the pre-existing winxpnodes experiment will be instantiated. Here winxpnodes might look like:

	# Windows XP experiment.
	source tb_compat.tcl
	set ns [new Simulator]

	set win1 [$ns node]
	tb-set-node-os $win1 WINXP-02-11
	tb-set-hardware $win1 pc850

	set win2 [$ns node]
	tb-set-node-os $win2 WINXP-02-11
	tb-set-hardware $win2 pc850

	set lan [$ns make-lan "$win1 $win2" 100Mb 0ms]

	$ns run

See 'Emulab in Emulab' for more details.