Skip to content. | Skip to navigation

Personal tools

Navigation

You are here: Home / Wiki / Install / Tipserv-Install

Tipserv-Install

Setting up the Console Server "tipserv"

Setting up the Emulab Console Server

IMPORTANT NOTE: The Comtrol Rocketport driver does not work for any FreeBSD newer than 7.3! This means that for new Emulab installs, you would have to use a separate tipserv machine running an older FreeBSD. The following instructions still assume FreeBSD 6.3. We will fix this as we get the chance.

For those with "standalone" serial concentrators, we have done some basic support for for those with Telnet-based interaction. See this page for details.

This guide assumes that your tipserv is a separate machine from your ops server. It is also written with steps for Comtrol Rocketport devices, other devices may have different set up procedures.

Please see the Console Lines page for a overview of Emulab's console support.

Step 1 - Installing FreeBSD on tipserv

Ensure that your tipserv machine is properly connected to your network and that all of the required software is available through the network or some other media. Boot the FreeBSD installation CD on the machine you will use as your tipserv server. When it asks for the desired installation mode, select Standard. Use the standard bootloader unless you have special requirements.

As with ops and boss, you need to create partitions will the following minimum sizes:

/usr/

Needs space for the ports tree and a system object tree, at least 10GB. Be sure to build with plenty of inodes (the ports tree itself uses about 200000, so be safe and build with at least a million).

/usr/testbed/

Needs space for testbed software and logs. At least 10GB. You may simply make sure this space is available in /usr, and not make this a separate partition.

/home/

Needs enough space to hold user accounts for this machine as you will not use your ops login. However, regular users will not be logging into this machine.

/var/

At least a few hundred MB.

When prompted, continue by selecting the 'Developer' distribution set - this will include full sources. When it asks you if you want to install the ports collection, answer no. Do not install any packages as this time. You will get a chance to do so later.

Near the end of the installation, you will be asked if you wish to install or configure several optional items. Apart from setting the hostname, configuring your network devices, enabling SSH logins (if desired), and setting your time zone and root password, you should skip additional configuration.

You can make yourself a user account at this point since accounts are not shared with boss/ops.

Step 1B - Configure root ssh login

Be sure to copy ops:/root/.ssh/authorized_keys to your new tip machine. Be sure that all directory and file permissions, owner and group, match ops:/root/.ssh and the authorized_keys file. If this is done properly, you should be able to "sudo ssh" from boss to the tip machine without having to type the root password. This must work in order for the tipserver to be functional.

Step 2 - Installing packages

To make sure that you're running with versions of software from the ports collection that are known to work with our software, and to save you hours of compile time, we provide pre-built binary packages of the ports required by Emulab.

If you haven't already, download the Emulab Server Package (you can use the FreeBSD 'fetch' command to download the file.)

Unpack this someplace (you need about 550MB of space, so don't use /tmp unless it is large enough). Let's say you unpacked it to /usr/tmp. You would then run:

ln -sf /usr/local/bin/perl5.8.8 /usr/bin/perl
env PKG_PATH=/usr/tmp/packages pkg_add emulab-ops-2.1

For consistency and to avoid clashes between versions of FreeBSD ports, we will install based on the same Emulab approved ports tree as ops and boss. If you haven't already, obtain a copy of our 'approved' ports collection. The package is about 40MB; it unpacks to 175MB. Unpack it to a place such as /usr/ports (if it exists, rename the old directory to ports.old, or just remove it), and install whichever ports you want to make ops feel like 'home' (such as emacs, jove, or whatever).

Now drag over a copy of your emulab source tree, and configure it just like you did on ops. Then run the "tipserv-install" target to install the necessary software. You will also need to copy over capture.pem,client.pem, and emulab.pem from boss:/usr/testbed/etc to /usr/testbed/etc on your tipserv machine.

Step 3 - Installing the RocketPort Driver

At this point you should have the PCI host adapter card installed and connections to the rackmount interface.

Download the FreeBSD rp driver from the Comtrol website and extract it somewhere. Replace the source located in /sys/dev/rp with that from the archive.

Edit /sys/conf/files to delete the lines containing dev/rp/rp_isa.c and dev/rp/rp_pci.c. Modify the line containing dev/rp/rp.c to the following:

dev/rp/rp.c     optional rp isa pci

Edit /sys/i386/conf/GENERIC to include the line:

device   rp

Edit /sys/modules/Makefile to remove the line:

rp  \

Rebuild the kernel to include the rp driver:

cd /sys/i386/conf
/usr/sbin/config GENERIC
cd /sys/i386/compile/GENERIC
make depend
make
make install
Reboot the machine, and confirm that the devices exist. These should be something like /dev/cuaC* or /dev/ttyC*.

Step 4 - Install the Console Software

Set up the logging directories. Capture is hardware to log into /var/log/tiplogs, so a symlink is required.

mkdir -p /usr/testbed/log/tiplogs
ln -s /usr/testbed/log/tiplogs /var/log/tiplogs
cp /dev/null /usr/testbed/log/tiplogs/capture.log

Set up logging for the capture processes by adding the following to /etc/syslog.conf

# testbed stuff
!capture
*.*           /usr/testbed/log/tiplogs/capture.log

Set up capture log rollover by adding this to /etc/newsyslog.conf

/var/log/tiplogs/capture.log            664  7     *    168   Z

Create a capture startup script in /usr/local/etc/rc.d/capture.sh. It should look something like the below. Add new lines for each console line you wish to connect, filling in the nodename, baudrate, and ttyport. nodename is the name of the node as recorded in the database tiplines table (e.g., "pc1"), baudrate is the speed of the serial port, typically 115200, and ttyport is the device file associated with the given device (e.g., "/dev/ttyC14").

#!/bin/sh

if ! PREFIX=$(expr $0 : "\(/.*\)/etc/rc\.d/$(basename $0)\$"); then
    echo "$0: Cannot determine the PREFIX" >&2
    exit 1
fi

case "$1" in
start)
        ;;
restart)
        killall capture
        ;;
stop)
        killall capture && echo -n ' capture'
        exit 0
        ;;
*)
        echo "Usage: `basename $0` {start|stop|restart}" >&2
        exit 0;
        ;;
esac

# list of captures to start up
bindir=/usr/local/sbin

# Add one or more lines for devices here
$bindir/capture -T 15 -r -s [baudrate] [device] [ttyport] >/dev/null 2>&1 &
...
      
A device line might look like this:
$bindir/capture -T 15 -r -s 115200 pc1 cuaC14 >/dev/null 2>&1 &

After creating the script, make sure you make it executable by doing chmod 0775 /usr/local/etc/rc.d/capture.sh. You will also need to manually run the script the first time (see Step 7 below), but after that, it will be run automatically whenever the machine is rebooted.

Add devices, servers, and nodes to /etc/remote to allow access by name. Make sure that your port mappings and device baud rate are correct. It makes adding devices later on easier to keep track of if you enter all the device filenames now, and just add names to them later. Here is an example with a 16 port Rocketport device:

#Emulab device access
pc1|rp0-1:/dev/cuaC0:br#115200:pa=none:
pc2|rp0-2:/dev/cuaC1:br#115200:pa=none:
rp0-3:/dev/cuaC2:br#115200:pa=none:
rp0-4:/dev/cuaC3:br#115200:pa=none:

rp0-5:/dev/cuaC4:br#115200:pa=none:
rp0-6:/dev/cuaC5:br#115200:pa=none:
rp0-7:/dev/cuaC6:br#115200:pa=none:
rp0-8:/dev/cuaC7:br#115200:pa=none:

rp0-9:/dev/cuaC8:br#115200:pa=none:
rp0-10:/dev/cuaC9:br#115200:pa=none:
rp0-11:/dev/cuaC10:br#115200:pa=none:
rp0-12:/dev/cuaC11:br#115200:pa=none:

boss|rp0-13:/dev/cuaC12:br#115200:pa=none:
ops|rp0-14:/dev/cuaC13:br#115200:pa=none:
cisco1|rp0-15:/dev/cuaC14:br#9600:pa=none:
cisco2|rp0-16:/dev/cuaC15:br#9600:pa=none:

Step 5 - Device Console Configuration

Each device type may have different configuration requirements for console forwarding. Generally nodes have a bios option to set forwarding after boot, as well as the baud rate. The default images provided by Utah will have FreeBSD/Fedora configured already to forward the console. For your boss and ops server you may need one or both of the following lines in /boot/loader.conf:

console="comconsole"
comconsole_speed="115200"
If console login is desired, you will also have to edit /etc/ttys to allow it:
console "/usr/libexec/getty std.115200"         unknown on secure

Step 6 - Boss Configuration

For each independant tipserv machine, add to the nodes and tipservers table in the database.

insert into tipservers values ("mytipserv.emulab.mydomain.net");
insert into nodes (node_id, type, phys_nodeid, role) values ("mytipserv", "pc850", "mytipserv", "ctrlnode");

For each node or device connected to the serial lines, add an entry to the tiplines table:

insert into tiplines (tipname, node_id, server) values ("pcXXX","pcXXX","mytipserv.emulab.mydomain.net")

As with ops, boss must be able to SSH to your tipservers without password. If you haven't set this up already, copy over your root public key from boss:

scp /root/.ssh/id_rsa.pub tipserv:/root/.ssh/authorized_keys
This command should now work from boss (without a password prompt):
sudo ssh tipserv ls /

Step 7 - Test It!

After you have added the database state for the tiplines, you can start up the capture processes on the tipserv machine. You can either login to the tipserv machine and run the capture.sh script you created in Step 4 manually:

sudo /usr/local/etc/rc.d/capture.sh start
or just reboot the tipserv machine. Afterward, make sure that there are "capture" processes running. Otherwise you will need to look at /var/log/tiplogs/capture.log to see what happened.

You should now be able to access the console for any node that was added. From the users machine, you can run console pcXXX and hit enter a few times, or reboot the machine and make sure something comes up.