Opsvm-Install

Creating your ops server as a VM (Jail) on boss

To save on hardware, it is now possible to run your ops server node as virtual machine (VM) on your boss server. In FreeBSD, virtual machines are also known as jails. To learn more about jails, you can do a man jail on your boss node or any other FreeBSD machine. While not necessary, briefly scanning the man page is probably a good idea so that you know the basics.

Before we begin, note that you will need another set of file systems for the jail since each jail needs its own. In other words, you will want to reserve enough space on one of your disk drives to create a FreeBSD slice, and then partition that slice with the usual file systems (/, /usr, and /var). Remember to leave enough room to make /usr at least 10GB (and plenty of inodes, at least a million). You will want /var to be at least 1GB. Oh, you will want to mount these file systems at /ops, and you will want to load them with the same distribution set you used to load boss with.

For this example, lets say that your root disk is /dev/da0 and that you have used slice two for the aforementioned file systems. If you run mount on boss, you would see something like:

	/dev/da0s2a on /ops (ufs, local)
	/dev/da0s2e on /ops/var (ufs, local)
	/dev/da0s2f on /ops/usr (ufs, local)

At this point there are a few little things to do before you can start the jail running:

1. Neuter /etc/fstab inside the jail:

   	boss> cp /dev/null /ops/etc/fstab
   

2. Add magic sshd directive for X11 forwarding:

   	boss> echo 'X11UseLocalhost no' >> /ops/etc/ssh/sshd_config
   

3. Copy over password/group files.

   	boss> cp -p /etc/master.passwd /etc/group /ops/etc
   	boss> pwd_mkdb -p -d /ops/etc /ops/etc/master.passwd
   

4. Initial resolv.conf; it will eventually be replaced.

   	boss> cp -p /etc/resolv.conf /ops/etc
   

5. Create mount points for Emulab specific file systems.

   	boss> mkdir /ops/users /ops/proj /ops/groups /ops/share
   	boss> chmod 777 /ops/users /ops/proj /ops/groups /ops/share
   

The Emulab specific file systems need to be permanently mounted into the jail, but rather then use NFS we use the more efficient nullfs type. Refer to your defs file for the value of the four shell variables below, and add these to the end of /etc/fstab:

	boss> cat >> /etc/fstab
	$FSDIR_GROUPS /mnt/groups nullfs rw 0 0
	$FSDIR_USERS  /mnt/users  nullfs rw 0 0
	$FSDIR_PROJ   /mnt/proj   nullfs rw 0 0
	$FSDIR_SHARE  /mnt/share  nullfs rw 0 0
	^D

Then mount them:

	boss> mount -a -t nullfs

Now add the jail definition to /etc/rc.conf so that the jail starts automatically when boss reboots. Just tack these lines onto the end of /etc/rc.conf:

	boss> cat >> /etc/rc.conf
	jail_enable="YES"
	jail_list="ops"
	jail_ops_flags="-n ops"
	jail_ops_hostname="ChangeMe"
	jail_ops_ip="ChangeMe"
	jail_ops_interface="ChangeMe"
	jail_ops_rootdir="/ops"
	jail_procfs_enable="YES"
	jail_devfs_enable="YES"
	^D

About the lines above that are marked with ChangeMe:

jail_ops_hostname

This is the fully qualified hostname for your ops, and should be the same as the USERNODE variable in your defs file.

jail_ops_ip

The IP address that corresponds to jail_op_hostname above, also from your defs file. If you are using a public address on your boss, this will typically be a public address as well.

jail_ops_interface

The name of the network interface (ie: fxp0. bce1, etc.) to listen on. Typically, this will be the same interface as your primary boss interface. An IP alias will be added to this interface.

Create a basic rc.conf inside the jail.

	boss> cat > /ops/etc/rc.conf
	hostname="ChangeMe"
	sendmail_enable="NO"
	sshd_enable="YES"
	nfs_client_enable="YES"
	nfs_client_flags="-n 8"
	rpcbind_enable="NO"
	mountd_enable="NO"
	nfs_server_enable="NO"
	ntpd_enable="NO"
	background_fsck="NO"
	linux_enable="YES"
	accounting_enable="YES"
	^D

Be sure to change the hostname line above to match what you set jail_ops_hostname to in the previous section.

Now you are ready to start the jail:

	boss> /etc/rc.d/jail start ops

If all goes well, you can now enter the jail:

	boss> jexec -n ops '' /bin/csh

Be sure to add the following line to your defs file. Make sure you use the same defs file for both boss and ops:

	OPSVM_ENABLE=1

At this point, you should return to the section on Installing Emulab on Ops, since this has to be done before you can continue setting up your boss node. Be sure to enter the jail to do this, as described above. When you are told to reboot your ops node, you should do this on boss (outside the jail):

	/etc/rc.d/jail restart ops

When you have finished, proceed with the next section, Installing Emulab Software on boss.