Skip to content. | Skip to navigation

Personal tools


You are here: Home / Wiki / Makinpackages


Making boss/ops/fs/etc. packages:

[ Updated 11/05/2013 for 9.2 and 9.2 64-bit packages. ]

0a. Make sure the ports tree in /share/freebsd/X.X/ports has the current
   Emulab ports. They are all in the "misc" subdir. If they are out of
   date, update them and remake the ports.tar.gz tarball.

0b. Allocate a node on which to build everything and load the appropriate
   OS on it. Pick a d710 if possible, both for the speed and the second disk.
   I try to use pc599 if available because it has more processors and RAM than
   the other d710s.

1. Create a couple of extra filesystems for src/ports and the build tree.
   I put these on different disks for better performance:

       <login to the node>

       sudo mkextrafs /repos
       sudo mkextrafs -r /dev/da1 -s 0 /usr/obj
       sudo rmdir /usr/src /usr/ports
       sudo ln -s ../repos/src /usr/src
       sudo ln -s ../repos/ports /usr/ports
       sudo mkdir /repos/src /repos/ports /repos/tmp
       sudo chmod 1777 /repos/tmp

       # make sure soft updates are enabled
       sudo umount /repos
       tunefs -n enable /dev/da0s4e
       sudo mount /repos
       sudo umount /usr/obj
       tunefs -n enable /dev/da1s1e
       sudo mount /usr/obj

2. Install the src and ports trees. From the node do:

       scp fs:/share/freebsd/9.2/src.tar.gz /repos/tmp/
       scp fs:/share/freebsd/9.2/ports.tar.gz /repos/tmp/
       cd /repos
       sudo tar xzf tmp/src.tar.gz
       sudo tar xzf tmp/ports.tar.gz

   Note that we avoid using NFS do to the race condition. We scp instead.

   Note also that the FreeBSD source tree (/usr/src) is really only needed
   to build the "lsof" package in emulab-extras.

3. Deinstall all existing ports. You will want to make sure you are root first,
   since this will get rid of "sudo".

4. Be paranoid, clear out old state.

   Look in /var/db/pkg. The only thing in that directory after doing the
   "pkg_delete -a" should be the pkgdb.db file (and maybe pkgdb.fixme?)
   If there is anything else, get rid of it.

   Remove everything in /var/db/ports.

   I have never been bold enough to remove everything in /usr/local, but
   in the latest build I did remove some stuff:

       cd /usr/local/lib
       rm -rf python* compat/pkg/* perl5* ruby*
       cd /usr/local/libexec
       rm -rf cups
       cd /usr/local
       rm -rf share/* man/* info/* libdata/ldconfig32

   If you are building ports on 8.3 and above, edit /etc/make.conf and add:

   # for 8.3, or...
   # ...for 9.2

5. Now begin the tedious, iterative process. First, do:

       setenv PACKAGES /usr/obj/ports/packages
       setenv TMPDIR /repos/tmp

   The former ensures that packages wind up in a single directory. The latter
   will force the compiler (and other utilities) to use /repos/tmp for
   temporary files.

   Now for each emulab-* port you install, you will need to perform the ritual
   steps below. But before doing these for any port, first check for any
   port-specific notes below (5a-5h). The ritual:

       rm -f /usr/obj/ports/packages/All/*

       # This step appears to not be strictly necessary.
       # If you are really paranoid (and have lots of time) you can do this.
       pkg_delete -a

       cd /usr/ports/misc/<emulab-port>
       make clean
       make install
       make clean
       make deinstall
       make package-recursive
   From ops do:
       mkdir /share/freebsd/x.x/packages/<emulab-port>
       cd /share/freebsd/x.x/packages/<emulab-port>
       scp -p pcXXX:/usr/obj/ports/packages/All/\* .

   Note that the deinstall doesn't undo the effect of the install, it just
   deinstalls (really, unregisters) the meta-port so that the package-recursive
   will work. The "make clean" avoids complaints about work directories

5a. Do emulab-boss. Notes:
    * Boost. We use the boost-all package which includes docs. I generally
      prefer not to drag around the docs, so I go into devel/boost-all, do
      a "make config" and deselect the docs box. Now when you do the make
      in emulab-boss, it should use that config when it gets to boost.
    * Git. Go into the devel/git directory, do a "make config" and deselect
      the GUI box. Otherwise it will drag in possibly conflicting versions of
      TCL and TK.
    * Pubsub. If the pubsub libraries are installed but were installed by hand
      and not through the port, then they will cause the port to not be
      installed as part of the emulab-boss port (since the dependency in the
      boss Makefile will be satisfied). This will cause an error when creating
      packages. The easiest fix is to just remove /usr/local/lib/libpubsub*,
      which is what the Makefile checks for.

    Additional 9.2 issues:

    * Curl. Defaults to building with GSSAPI which does not exist in the
      base system. Should it? I don't know, but I just go into ftp/curl
      and do "make config" to remove that option.
    * When doing "make install", graphviz wanted pango which wanted cairo.
      The pango build blew up because it claims to need cairo with glib
      enabled. But the default config does have glib enabled and retrying
      the make worked fine. I think this is a case of stale state where
      something that was not true when the pango build started was true
      later on. I fixed the Makefiles for this and another port with a
      related problem, see /share/freebsd/9.2/README.ports for details.

5b. Do emulab-ops.
    * Assuming you built emulab-boss already, everything should be fine.
      Otherwise you will need to deal with git and pubsub as above.

5c. Do emulab-fs.

5d. Do emulab-protogeni.

5e. Do emulab-tipserv.

5f. Do emulab-mailman.

5g. Do emulab-extras. Notes:
    * We do this one last so that portaudit doesn't get installed til the
      end. Otherwise, it would cause builds of vulnerable ports to fail.
      See Step 6 below if installing vulnerable ports bothers you.
    * Emacs. I used to build the -nox11 varient so we didn't drag in a ton
      of crap, but Leigh prefers the real deal and came up with a reasonable
      set of options. These are now baked into the emulab-extras Makefile.
      If you get an error from the Makefile about "m17n support requires Xft"
      then you probably have an old options file. You should remove
    * Lsof. Remember to install the FreeBSD source in /usr/src as talked about
      back on step #1. Otherwise, the build of lsof is going to fail.
    * Vim. Yes, you are not imagining that, it really is downloading 400+
      patches. Swearing will make you feel better and pass the time...
    * If you wind up with the ruby+nopthreads port, then again you probably
      have some left over /var/db/ports/*/option files. This isn't a show
      stopper, so probably not worth fixing after the fact.

5h. Painful post-port processing.
    * I don't always do it, but after all the ports are installed
      run portaudit and see what security vulnerabilities there are. Manually
      update the ports and make notes in /share/freebsd/x.x/README.ports.
      This is fraught with peril however. Some ports versions are assumed in
      various other ports and things can degenerate rapidly when you try to
      do this! See one of the existing README.ports files for examples.
    * Install the bsdadmintools port (sysutils/bsdadmintools) and run
      "pkg_libchk" to ensure that all the library dependencies of the ports
      are correctly resolved.

6. After you have collected all the ports, you should have seven directories
   full of packages on ops: boss, ops, fs, protogeni, tipserv, mailman, extras;
   which we will now aggregate into one set.

   Most of the subdirs will include packages like perl which should in theory
   be the same, and for certain will be if you don't do "pkg_delete -a" between
   making each one. However, just in case there are differences, I use the
   strategy I have always used, which is to favor the version built as part of
   the "more important" package. So to create the aggregate package I do
   (csh syntax):

       cd /share/freebsd/x.x/packages
       foreach d (extras mailman tipserv protogeni fs ops boss)
         mv $d/* .
         rmdir $d

7. Now you should have a fully functional set of packages. The last thing to
   do is make sure they install. ON YOUR BUILD NODE (NOT on boss or ops!) do
   the following and watch for any errors (there will be plenty of
   informational messages):

       sudo tcsh
       cd /share/freebsd/x.x/packages

       # make sure you are not on boss or ops!
       hostname -s | grep -Eq '^(boss|ops)$' && \
           (echo 'STOP! you are on boss or ops, ^C now!'; sleep 100000; exit)

       # all possible boss-install packages
       pkg_delete -a
       pkg_add emulab-boss*
       pkg_add emulab-protogeni*
       pkg_add emulab-fs*
       pkg_add emulab-extras*

       # all possible ops-install packages
       pkg_delete -a
       pkg_add emulab-ops*
       pkg_add emulab-fs*
       pkg_add mailman*
       pkg_add cvsd*	# may already be installed
       pkg_add adodb*	# may already be installed
       pkg_add emulab-extras*

       # all possible fs-install packages
       pkg_delete -a
       pkg_add emulab-fs*
       pkg_add emulab-extras*

       # all (someday) possible tipserv-install packages
       pkg_delete -a
       pkg_add emulab-tipserv*
       pkg_add emulab-extras*