Skip to content. | Skip to navigation

Personal tools

Navigation

You are here: Home / Wiki / Appendix / Upgrade-410-63.Html

Upgrade-410-63.Html

Upgrading from FreeBSD 4.x or later to 6.3

Upgrading from FreeBSD 4.x or later to 6.3

This is the procedure we used to upgrade FreeBSD-4.10-based boss and ops nodes to FreeBSD 6.3 with the Emulab-approved 2008-08-25 ports collection. It should work for upgrading later versions of FreeBSD as well.

Note that all the following commands assume you are running as root and using csh as your shell (not sh or bash).

Before You Start

Backup!

First and foremost, you should backup your boss and ops before you start. A complete backup to tape or other offline storage is always best, but if nothing else you should backup the assorted configuration files, logs and databases. Copying /etc, /usr/local/etc, and /var should capture most of that.

Shutdown Emulab

You will want to following the instructions in "Step 0" of the Testbed Update instructions. The various software packages will be in an inconsistent state during the upgrade process, so it is best to not have any of the Emulab software (or users!) running.

Do you need Elvin?

You will need to decide whether you need "Elvin Compatibility." A couple of years ago, we reimplemented the publish/subscribe system underlying our event system to avoid licensing issues. The new mechanism ("pubsub") is incompatible with the old ("Elvin") mechanism. This can be an issue after you update the server-side of the event system to use pubsub but still have event clients (node disk images) that use Elvin. The preferred solution is to update the Emulab client-side on your disk images to use the pubsub mechanism. However, if you have custom disk images that you cannot re-install the Emulab client-side software on, then you can enable the so-called "elvincompat". This primarily comes in to play below when deciding which packages to install.

Know your Partitions

When you start the FreeBSD upgrade process, you will be prompted to identify which special devices belong to which filesystems. You best figure that out now by doing:

  df -t ufs

and noting which special files (e.g. "/dev/da0s1a") belong to which filesystems (e.g. "/"). You will only need to mount system directories (e.g., /, /usr and /var) and not the Emulab directories (e.g., /proj, /users, /usr/testbed). Write this information down as you will need it shortly.

Know your installed Packages

Create a list of currently installed packages:

   pkg_info > /var/tmp/installed_packages

as you may need to reinstall some packages after the upgrade is done.

The port-update script you will run later, is not elegant or efficient at what it does; it just gets the job done. As a result, it is also not shy about removing existing ports to "get the job done." In particular, it will remove python 2.3 (py23-*), python 2.4 (py24-*), Mysql (mysql-*), and PHP4 (php4-*) packages.

Before you Reboot

Finally, before rebooting your machine to start the upgrade there are two things you should fix up in the filesystem. First, rename the existing /usr/src directory so that the new 6.3 source tree will be installed:

  mv /usr/src /usr/src.old

and second, if you are using a serial console at a speed other than 9600 baud, you can fix the baud rate by editing /boot/loader.conf and adding:

  comconsole_speed=115200

or whatever your desired speed is. This file is preserved across the upgrade and will take effect when the new kernel boots. You will still need the VGA console for the upgrade itself however.

The Upgrade Procedure

Update FreeBSD

1) Boot the first CD of the FreeBSD 6.3 release. Once in the sysinstall menu, select Upgrade. You may be prompted to mount your FreeBSD partitions before proceeding. When it asks you for a distribution set, select Developer. Do not install the ports collection at this time.

After performing the binary upgrade, sysinstall should inform you that the operation was successful.

2) Reboot your machine and make sure the kernel can load. There will almost certainly be issues with your configuration files in /etc.

If your kernel will not boot, you may need to reenter sysinstall and reinstall your master boot record and/or the boot blocks with the bsdlabel utility.

3) We have heard that when upgrading from 4.x, various include files do not get updated. To be safe, on both boss and ops do the following:

  cd /usr/src/include
  make install

4) Apply security patches. We have combined all the current FreeBSD security advisories (as of 8/25/2008) into a single patch that you can download:

  cd /usr
  fetch -o /var/tmp/FreeBSD-6.3-src.patch http://www.emulab.net/downloads/FreeBSD-6.3-src.patch
  patch -p1 < /var/tmp/FreeBSD-6.3-src.patch

Now you can rebuild and reinstall the necessary applications:

  cd /usr/src/secure/lib/libssh
  make obj && make depend && make && make install
  cd /usr/src/secure/usr.sbin/sshd
  make obj && make depend && make && make install
  cd /usr/src/lib/bind
  make obj && make depend && make && make install
  cd /usr/src/usr.sbin/named
  make obj && make depend && make && make install

and rebuild, install and reboot a new kernel. The kernel configuration should be either "SMP" or "GENERIC". Do "uname -i" and if it says "SMP-GENERIC" use the SMP config, otherwise use the GENERIC config.

	set CONF=SMP    # or "GENERIC"
        cd /usr/src
        make buildkernel KERNCONF=$CONF
        make installkernel KERNCONF=$CONF
        shutdown -r now

5) After the machine reboots (some startup daemons may fail, don't worry about it yet), login as root again and save off your current /etc directory:

  cp -rp /etc /Oetc

6) There are some new "standard" users that you should add now to avoid problems later. Use "vipw" and add these lines if they are not already present:

proxy:*:62:62::0:0:Packet Filter pseudo-user:/nonexistent:/usr/sbin/nologin
_pflogd:*:64:64::0:0:pflogd privsep user:/var/empty:/usr/sbin/nologin
_dhcp:*:65:65::0:0:dhcp programs:/var/empty:/usr/sbin/nologin
nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/usr/sbin/nologin

Then add these groups to /etc/group (use your standard editor) if they do not already exist:

proxy:*:62:
authpf:*:63:
_pflogd:*:64:
_dhcp:*:65:
audit:*:77:

Finally, add these to /etc/mail/aliases:

_dhcp:  root
_pflogd: root
proxy:  root

7) You now need to update your startup scripts. This is one of the more painful parts of the upgrade. You will use "mergemaster":

  mergemaster -i

which will present you with diffs of any old and new /etc files that differ, and let you choose whether to install the new version or not. You should choose to install (answer 'i') new versions for all except:

  mail/aliases
  etc/disktab
  etc/group
  etc/hosts
  etc/motd
  etc/shells
  etc/ttys
  etc/master.passwd

For these, you should keep the existing version (answer 'd' to delete the new version). Note that some of the new versions will clobber some testbed specific changes, but we will add those back in the next step. Add the end, it will ask if you want to run cap_mkdb, choose yes.

8) Add back some Emulab changes to some files. For these it was deemed easier to just add old state to new files rather than adding new state to old files like we did for the passwd and group files above. Which files and how they are modified depends on whether you are in the process of updating boss or ops.

8a) For the "ops" node, append these back to /etc/ssh/sshd_config:

# The follwing lines were added by the testbed installation process
PermitRootLogin yes
Protocol 2,1
# End of testbed-added configuration

and to /etc/newsyslog.conf:

# The follwing lines were added by the testbed installation process
/var/log/logins                         640  7     200  *     Z
/var/log/mountd.log                     640  5     200  *     Z
/usr/testbed/log/pubsubd.log            640  5     1000 *     Z
/usr/testbed/log/elvin_gateway.log      640  5     1000 *     Z
/var/log/tiplogs/capture.log            644  7     *    168   Z
# End of testbed-added configuration

and /etc/syslog.conf:

# The follwing lines were added by the testbed installation process
!capture
*.*                                        /var/log/tiplogs/capture.log
!mountd
*.*                                        /var/log/mountd.log
!pubsubd
*.*                                        /usr/testbed/log/pubsubd.log
!elvin_gateway
*.*                                        /usr/testbed/log/elvin_gateway.log
# End of testbed-added configuration

8b) On the "boss" node, you will need to add these. /etc/ssh/sshd_config:

# The follwing lines were added by the testbed installation process
PermitRootLogin yes
Protocol 2,1
# End of testbed-added configuration

For /etc/crontab:

# The follwing lines were added by the testbed installation process
0      6       *       *       *       root    /usr/testbed/sbin/audit
0      1       *       *       *       root    /usr/testbed/sbin/dbcheck
45     1       *       *       *       root    /usr/testbed/sbin/backup
*/5    *       *       *       *       root    /usr/testbed/sbin/node_status
*/5    *       *       *       *       root    /usr/testbed/sbin/idlemail
# End of testbed-added configuration

For /etc/newsyslog.conf:

# The follwing lines were added by the testbed installation process
/usr/testbed/log/tmcd.log               640  9     1000 *     Z
/usr/testbed/log/stated.log             640  9     300  *     Z
/usr/testbed/log/osselect.log           640  9     300  *     Z
/usr/testbed/log/power.log              640  7     300  *     Z
/usr/testbed/log/frisbeed.log           640  7     300  *     Z
/usr/testbed/log/tftpd.log              640  7     200  *     Z
/usr/testbed/log/dhcpd.log              640  7     200  *     Z
/usr/testbed/log/bootinfo.log           640  7     200  *     Z
/usr/testbed/log/capserver.log          640  5     200  *     Z
/usr/testbed/log/pubsubd.log            640  5     1000 *     Z
/usr/testbed/log/suexec.log             640  3     200  *     Z
/usr/testbed/log/genlastlog.log         640  3     200  *     Z
/usr/testbed/log/genlastlog             640  3     200  *     Z /var/run/lastlog_daemon.pid
/usr/testbed/log/xmlrpcbag.log          640  7     300  *     Z
/usr/testbed/log/sshxmlrpc.log          640  7     300  *     Z
/usr/testbed/log/sslxmlrpc.log          640  7     300  *     Z
# End of testbed-added configuration

and for /etc/syslog.conf:

# The follwing lines were added by the testbed installation process
!bootinfo
*.*                                            /usr/testbed/log/bootinfo.log
!tmcd
*.*                                            /usr/testbed/log/tmcd.log
!capture
*.*                                            /usr/testbed/log/capture.log
!dhcpd
*.*                                            /usr/testbed/log/dhcpd.log
!proxydhcpd
*.*                                            /usr/testbed/log/proxydhcpd.log
!tftpd
*.*                                            /usr/testbed/log/tftpd.log
!capserver
*.*                                            /usr/testbed/log/capserver.log
!frisbeed
*.*                                            /usr/testbed/log/frisbeed.log
!pubsubd
*.*                                            /usr/testbed/log/pubsubd.log
!stated
*.*                                            /usr/testbed/log/stated.log
!osselect
*.*                                            /usr/testbed/log/osselect.log
!genlastlog
*.*                                            /usr/testbed/log/genlastlog.log
!sdcollectd
*.*                                            /usr/testbed/log/sdcollectd.log
!xmlrpcbag
*.*                                            /usr/testbed/log/xmlrpcbag.log
!sshxmlrpc
*.*                                            /usr/testbed/log/sshxmlrpc.log
!sslxmlrpc
*.*                                            /usr/testbed/log/sslxmlrpc.log
# End of testbed-added configuration

Update ports

1) Remove/rename the old ports collection in /usr/ports if it exists. Unpack the Emulab-approved ports collection and install it:

  cd /usr
  fetch -o /var/tmp/ports-20080901.tar.gz http://www.emulab.net/downloads/ports-20080901.tar.gz
  mv ports ports.old
  tar xzf /var/tmp/ports-20080901.tar.gz

2) Download the pre-built package tarball and untar it. These packages will allow you to install and upgrade the needed ports much faster. The tarball is about 125MB and requires another 125MB to unpack, so make sure your target filesystem has enough space.

  set DIR=/var/tmp   # or somewhere with ~250MB of free space

You need to fetch a different tarball depending on whether you need Elvin compatibility or not. (You did read the Before You Start section before you started, right?) If you do not need Elvin compat then:

  set TARBALL=FreeBSD-6.3-20080901.tar.gz
  setenv ELVIN_COMPAT 0

If you do need Elvin compat then:

  set TARBALL=FreeBSD-6.3-20080901-elvincompat.tar.gz
  setenv ELVIN_COMPAT 1

In either case, you then need to fetch the packages tarball, unpack and ready for the install.

  cd $DIR
  fetch http://www.emulab.net/downloads/$TARBALL
  tar xzf $TARBALL
  setenv PKG_PATH $DIR/packages

3) Since upgrading your ports is less a feat of engineering and more an instance of performance art, we have encapsulated the process in a script which you can run with either the argument boss or ops depending on which you are upgrading.

  set NODE=boss             # or "ops"
  setenv UTMPDIR /var/tmp   # or some other tmp directory
  fetch -o $UTMPDIR/update-ports http://www.emulab.net/downloads/update-ports-20080901
  fetch -o $UTMPDIR/update-patches.tar.gz http://www.emulab.net/downloads/update-patches-20080901.tar.gz
  tar -C $UTMPDIR -xvf $UTMPDIR/update-patches.tar.gz
  chmod +x $UTMPDIR/update-ports
  $UTMPDIR/update-ports $NODE |& tee $UTMPDIR/update.out

This process will take tens of minutes and generate a lot of scary output. In general, don't worry about messages from pkg_delete concerning things that could not be completely removed. Only pkg_add errors are important.

4) One final update to the startup files. You will need to update /etc/rc.conf to ensure that named and apache are correctly started as well as the compatibility packages just installed. Append the following to /etc/rc.conf on boss:

apache_enable="YES"
named_enable="YES"
named_chrootdir=""
named_flags=""
named_uid="root"
named_pidfile="/var/run/named.pid"

compat4x_enable="YES"
compat5x_enable="YES"

and for /etc/rc.conf on ops:

smbd_enable="YES"
apache_enable="YES"

compat4x_enable="YES"
compat5x_enable="YES"

Reboot your machines and verify that everything starts. Watch out for packages that update-ports may have removed. You can reinstall any such "collateral damage" by looking at the list of (previously) installed ports you created before you started.

Update Emulab Software

Now that you are running 6.3 with the latest ports, you can update the Emulab software starting with Step 1 of Testbed Update instructions.