Boss-Emusoft.Html
Emulab Installation Documentation
Installing the Emulab Core Server
Installing Emulab Software on boss
Note:
If you are going to use a virtual machine (Jail) running on boss as your ops server node, then first follow the instructions for Creating your ops server as a VM (Jail) on boss. Then return to this page when you are finished installing the Emulab software inside the ops virtual machine.
This procedure is similar to installing the testbed software on ops. Before beginning, make sure that ops is installed, running, and reachable through the network by your new boss machine.
Step 1 - Installing Packages
Unpack the Emulab Server Package someplace. Let's say you unpacked it to /var/tmp. You would then run:
# force install of pkg pkg info pkg add /var/tmp/packages/emulab-boss-6.0
Of course, if you unpacked it somewhere else, put in the correct path.
We provide all of the binary packages necessary to build boss in the packages tarball. If you would like to install additional software on boss, unpack the 'approved' ports collection to a place such as /usr/ports (rename or delete the old directory, if it exists). The package unpacks to approximately 900MB. Install whichever software you would like.
Step 2 - Unpacking and running 'configure'
Download the Emulab source tree from the emulab-stable git repository and then run its configure script, providing the path of your definitions file:
git clone git://git-public.flux.utah.edu/emulab-stable.git /usr/testbed/src/testbed mkdir -p /usr/testbed/obj/testbed cd /usr/testbed/obj/testbed /usr/testbed/src/testbed/configure --with-TBDEFS=/path/to/your/defs-file
Remember, you have to use the same defs-file that you used on ops.
Step 3 - Running the boss installation script
In the object tree you've configured (say, /usr/testbed/obj/testbed), there's an 'install' subdirectory with a script called 'emulab-install'. Run this script as root (note the same package directory argument as above):
cd install perl emulab-install -p /var/tmp/packages boss
This will set up passwordless sudo for anyone in the wheel group. If this script fails, send the output to Utah so we can fix it. Note that emulab-install requires internet access so it can grab some files.
The source installation is performed by the emulab-install script and should be completely automatic. However, you may check it or fix up problems by running the makes yourself, as follows:
gmake
gmake boss-install
Step 4 - Verify SSH configuration
The emulab-install script should have set up root ssh from boss to ops. To confirm this, make sure this works:
boss> sudo ssh ops ls /
If this fails, you will need to set this up by hand, typing a password:
scp /root/.ssh/identity.pub ops:/root/.ssh/authorized_keys
Step 5 - Setting up named
The testbed software manipulates DNS zone files for two reasons. First, it adds your nodes to them so that you don't have to. Second, it creates CNAMEs for all the nodes in every experiment (so that you can use, for example, 'nodeA.myexp.myproj.emulab.net' to refer to your node regardless of which physical node it got mapped to).
The named_setup script does this by generating zone files. In general, it concatenates a '.head' file, written by you, with its own generated entries. The main zone file is /etc/namedb/OURDOMAIN.db, where OURDOMAIN is from your defs file. (OURDOMAIN, unless explicitly specified, is taken to be the domain portion of BOSSNODE). We also generate reverse zone files (for inverse lookups, i.e. turning IP addresses back into names) in /etc/namedb/reverse/SUBNET.db, where SUBNET is the class-C subnet in which the addresses reside (i.e. 10.0.0.db). This value is defined in the defs file, as is TESTBED_NETWORK.
emulab-install makes a reasonable attempt to create a set of named configuration files for you, placing them in /etc/namedb. If your testbed consists of a single class-C network, then these files will most likely be correct, although you'll want to look at them to make sure. Look at these files to make sure:
/etc/namedb/OURDOMAIN.db.head /etc/namedb/reverse/SUBNET.db.head /etc/namedb/named.conf
If you have more than one class-C subnet for your testbed, you'll need a copy of the reverse zone file for each one. You want to put boss, ops, and any 'infrastructure' equipment such as routers and switches into the zone files. These zone files do not need to include the nodes - they will be added automatically. Be sure to edit /etc/namedb/namedb.conf if you add any reverse map files (follow the format for the existing entry).
Once you think you've got things set up, run make sure the following command doesn't give you any error messages:
/usr/testbed/sbin/named_setup
This command will generate the following files:
/etc/namedb/OURDOMAIN.db /etc/namedb/reverse/SUBNET.db
Step 5a - Named for an unroutable control network.
If you are using routable IP addresses for all of your testbed, then you can skip this step. However, if you are using unroutable private IP addresses for all or part of the testbed, you will need to use a feature of bind called 'views' so that the private addresses are only exposed to clients within the testbed. See the bind documentation for more details on this feature. You will want to make sure that loopback queries from boss itself see the internal view; i.e., you want boss to resolve its own hostname to its private address, not its public one.
In order to use multiple views, we generate multiple zone files. In addition to OURDOMAIN.db, which will be used for the 'external' view, we create OURDOMAIN.internal.db for use with the 'internal' view. So, you'll also need to create OURDOMAIN.internal.db.head. When we generate the zone files, only publically-routable addresses are put into OURDOMAIN.db. OURDOMAIN.internal.db contains all addresses, both public and private. So basically, you'll want to put the public addresses for boss, ops, etc. into OURDOMAIN.db.head, and their private addresses into OURDOMAIN.internal.db.head.
Step 5b - Named recursive queries.
Current versions of Bind default to not allowing recursive lookups. This is a useful security feature, but means that nodes cannot lookup outside hosts! One simple fix to this is to explicitly allow recursion. In your named.conf file, find the options section and add the following line:
allow-recursion { any; };
You can also allow limited recursion by listing hosts and networks that can make recursive queries. For example, if your control network was 10.1.0.0/24, then you could:
allow-recursion { 127.0.0.1; 10.1.0.0/24; };
Step 6 - Miscellaneous configuration
There are a few things that we haven't been able to completely automate just yet, though we hope to soon.
hosts file - You want to put boss/ops name/IP addresses in /etc/hosts on both boss and ops to avoid boottime circular dependencies (because of NFS cross mounts). This is done for you in emulab-install, but you should confirm that it was done properly.
SSL Certificates - Our apache config file expects to find SSL certificates in:
/usr/local/etc/apache/ssl.crt/www.<sitename>.crt /usr/local/etc/apache/ssl.key/www.<sitename>.key
Where <sitename> is OURDOMAIN from the configure defs file, which defaults to boss' domain name.
emulab-install already generated a temporary no-passphrase certificate for you and placed it in the locations specified above. However, we recommend that you get a "real" certificate from Verisign (or one of the others).
DHCPD - emulab-install generated a dhcpd.conf.template file and installed it in /usr/local/etc (the template is derived from information you provided in the defs file). It then generated an actual dhcpd.conf file and started up dhcpd for you. Do not edit the dhcpd.conf file directly! If you need to make changes, change the template instead and then run:
/usr/testbed/sbin/dhcpd_makeconf -i -r
locate database - It can be useful to update the 'locate' database to help you find files as you're learning the system. This normally happens nightly, but you can force it to happen now by running the following as root:
/usr/libexec/locate.updatedb &
This will run in the background for several minutes. You can then find 'foo.conf' by running 'locate foo.conf'.
dhclient - If you don't control the public network to which boss and ops (and perhaps tipserv) are attached, your network admins may want you to run dhclient on those interfaces (and you may want to as well so that your boss /etc/resolv.conf contains the correct nameserver returned by dhclient, to avoid configuring it statically). However, boss runs its own nameserver that ops and the experimental nodes must contact to resolve internal names, so you must get dhclient to correctly write /etc/resolv.conf (i.e., boss must be the first nameserver). The easiest way to solve this problem is to add the following line to boss' /etc/dhclient.conf file:
prepend domain-name-servers 127.0.0.1;
and add the following line to ops' (and any other non-experimental nodes, such as tipservs) /etc/dhclient.conf:
prepend domain-name-servers <bossip>;
where <bossip> is replaced by a valid IP address for your boss. Make sure to restart dhclient on these machines since dhclient will continue to overwrite /etc/resolv.conf incorrectly.
Step 7 - Quotas (Optional)
Note:
Skip this step if you are running ops as a virtual machine on boss; quotas are not supported in this configuration.
If you are planning to run quotas for Emulab users on the ops (or fs) server, you will need to establish a default quota value for all users. Ideally, the quota would be configurable per-user, but for now all quotas are initialized from a "prototypical" user. Note that you can change individual user quotas later by running edquota(8) on ops (or fs). First, log into your ops node (or your fs node if you are using a separate file server node).
Now set the quota for elabman on each quota-enabled filesystem. For example:
edquota -e /proj:2000000:2000000 -e /users:1000000:1000000 elabman
would set a 1GB quota on /proj and 512MB on /users for the elabman (prototypical) user. Once the prototype user quotas are established, you should do:
edquota -p elabman 10000-15000
which would automatically apply the elabman quotas to any user created with uids between 10000 and 15000, assuming that you wanted to allow up to 5000 users.
Step 8 - Reboot boss
Okay, go ahead and reboot boss now, and make sure it comes up okay.