Skip to content. | Skip to navigation

Personal tools

Navigation

You are here: Home / Wiki / Linuxkernel

Linuxkernel

Installing and Building Linux Kernels

In the past it was fairly trivial to build and install a Linux kernel on
a Utah Emulab machine.  You'd grab our latest configuration that had all
the necessary configuration options for our machines' hardware, and
follow something like the steps listed below in the "Manual"
instructions section, fix up your bootloader config to boot the
newly-installed kernel, and you'd be done.

You can still use a manual approach, but it's become preferable to
rebuild your distribution's kernel package using additional patches
and/or configuration options.  This is a desireable approach because the
resulting installed, custom kernel packages are tracked in your distro's
package database and can thus provide and satisfy dependencies for other
packages on the system---for instance, allowing you to install packages
that depend on newer kernels and their features.

We cannot maintain instructions for every possible kernel and kernel
module build or install procedure, and they're easily findable online
anyway.  So, what we do here is outline several possible solutions, and
provide helpful links.

But first, some background --- this page has to try to help people build
kernels on multiple generations of our images, so some history is in
order, sadly.


Modern Emulab Images vs Older Images

Prior to the UBUNTU12 and CENTOS63 images, we at Utah had long built
custom kernels each time we created a standard image.  We did this at
various times to either ensure that all the config options we wanted to
provide were provided; or because the distro's default kernel wouldn't
run on all our machines; because we had custom kernel patches of our
own; or simply because it became tradition!

We no longer need to do this.  Distro kernels for our supported
distributions (Ubuntu, CentOS) tend to have all the necessary hardware
options in a stable, well-tested configuration; we no longer have custom
kernel patches (only an extra, optional module); and we have broken with
tradition!  Thus, we now rely on distro kernels, and compile our extra
modules against the distro kernel.  This makes it significantly easier
for our users to extend or replace the kernel on one of our standard
disk images.


Building Modules for an Existing Kernel

When you compile a single module to add to an already-built kernel, you
need the kernel headers and a few files generated when the kernel you
are building the module(s) against was originally built.  Distributions
package these files up so you can build your extra modules against the
pre-packaged kernel.

For Ubuntu, find out which kernel you want to install headers for:

    $ dpkg-query -l | grep linux-image

Make sure the headers are available:

    $ apt-cache search linux-headers

Then install the headers:

    $ apt-get install linux-headers-<VERSION>


For CentOS/Fedora:

    $ yum install kernel-devel kernel-headers

Note: it's always possible that the headers package for the specific
kernel you want have been superseded by a newer version.  In this case,
if you insist on having the missing version, you'll need to search the
distro's package archive manually, and download and install the
necessary packages.  Alternatively, you can update the kernel to a newer
version for which headers are available.

Then, using the module's included Makefile, proceed to build your
module, with something like

    $ make && sudo make modules_install


Building a Customized Distro Kernel

In this case, it's best to follow your distro's instructions.  Here's a
collection of useful links that will help you do this.

Ubuntu:

https://help.ubuntu.com/community/Kernel/Compile
https://wiki.ubuntu.com/Kernel/BuildYourOwnKernel
https://wiki.ubuntu.com/KernelCustomBuild
https://help.ubuntu.com/community/Kernel
https://wiki.ubuntu.com/Kernel/MainlineBuilds

CentOS/Fedora:

https://fedoraproject.org/wiki/Building_a_custom_kernel
http://fedoraproject.org/wiki/Kernel


Manually Building a Kernel From Source

It's pretty easy to build a kernel from scratch.  You should read
https://www.kernel.org/doc/readme/README to make sure you know what
you're doing, but here's a quick overview too.

All standard Emulab disk images have the tools you need to build the
kernel already (GNU Make, gcc, etc).  So extract your kernel source
tarball into a directory on a filesystem containing 6-8 GB of space.
Then you need to find a kernel config that will work on all the machines
you care about.  You can do this manually by looking at hardware in each
machine type you care about, and select the drivers you need --- or you
can start with an existing config and customize it.  For older Emulab
images, we used to store configs and compiled kernel binaries on
users.emulab.net:/share/<distro>/<release>/... (i.e.,
/share/fedora/F15/config-2.6.40-4.emulab1.fc15.x86_64).  However, kernel
config files are nearly always stored on the image itself.  Either
they'll be in /boot/config-<version> (or similar), or if /proc/config.gz
is available, it provides the config of the running kernel.  If you
can't find an existing config in any of these places, you'll need to
grab a config from somewhere else.

Once you've got your source tree, cd to that dir and run

    $ make mrproper

Then, copy your config file into your source tree into .config (i.e.,
'cp -pv /path/to/my/config/file .config').  Then, make sure you don't
need to toggle any new options via

    $ make oldconfig

which will prompt you to make choices for new options.  Then do the
standard dance:

    $ make && make modules
    $ make bzImage
    $ sudo make modules_install

At this point, if you have built a kernel that requires an initramfs to
boot, you'll need to run

    $ sudo make install

to install the kernel and build an initramfs for it based on your
distro's /sbin/installkernel .  Or, if you've built a kernel that has
all the necessary hardware and filesystem drivers compiled in (instead
of built as modules), you can just copy the kernel binary and metadata
files into place, like:

    $ export KVERSION=<version>
    $ sudo cp System.map /boot/System.map-${KVERSION}
    $ sudo cp vmlinux /boot/vmlinux-${KVERSION}
    $ sudo cp arch/<arch>/boot/bzImage /boot/vmlinuz-${KVERSION}

where <arch> is probably x86_64 (maybe i386 for old stuff).  The vmlinux
file is mostly useful for debugging your kernel if necessary.

Configure the Bootloader

Finally, you need to make sure your bootloader is ready to boot the new
kernel!  Fortunately, the LILO bootloader has been replaced everywhere
with GRUB or GRUBv2, so unless you're working with a really ancient
image, you'll get to configure GRUB.  The main difference between LILO
and GRUB is that GRUB was designed to be "installed" once in your boot
sector, and then read a config file in /boot to get the list of kernels
and boot options; this file could be changed each time you installed a
new kernel to add a new boot entry corresponding to the new kernel.
With LILO, you had to re-run /sbin/lilo each time you changed
/boot/lilo.conf, because the boot entries were baked into the boot
sector along with the LILO boot program.  GRUB is much more flexible.
You should not need to reinstall the GRUB bootloader program when
customizing an Emulab image!

(If you ran 'make install' above, your distro's /sbin/installkernel
script should have created a new GRUB bootloader entry for you already,
but it's good to check that the format and entry is correct and points
to the files you just installed --- you don't want to end up with a
machine that can't boot!)

(You can determine which version of GRUB is installed by running
grub-probe --version or grub2-probe --version or grub --version)

 

GRUBv2

Most distros have switched over to GRUBv2, which has a different config
file format than GRUB 0.97.x.  The config file should be found at
/boot/grub/grub.cfg, although some distros have a /boot/grub/grub.conf
symlink to that file.  You'll want to either add an entry if you
manually installed your kernel, or check the distro-added entry if you
used 'make install' during the manual kernel install above.  The entry
should look like this, although it is very distro-dependent.  The best
way for you to add a new entry is to copy an existing entry, and tweak
the kernel filename and initramfs to point to your new files --- or else
use whatever procedure your distro recommends (surely not the manual
method I just outlined!).

    menuentry 'Ubuntu, with Linux 3.8.4' --class ubuntu --class gnu-linux --class gnu --class os {
        recordfail
        gfxmode $linux_gfx_mode
        insmod gzio
        insmod part_msdos
        insmod ext2
        set root='(hd0,msdos2)'
        search --no-floppy --fs-uuid --set=root d91991ed-cfb6-4384-a9d5-e1f57e56cb00
        linux   /boot/vmlinuz-3.8.4 root=UUID=d91991ed-cfb6-4384-a9d5-e1f57e56cb00 ro console=tty0 console=ttyS0,115200  crashkernel=384M-2G:64M,2G-:128M
        initrd  /boot/initrd.img-3.8.4
    }

The GRUBv2 manual is found at
<http://www.gnu.org/software/grub/manual/grub.html>.  An Ubuntu-specific
guide can be found at <https://help.ubuntu.com/community/Grub2>.  A RHEL
7 (which CentOS 7 is based off of) GRUBv2 guide can be found at
<https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/System_Administrators_Guide/ch-Working_with_the_GRUB_2_Boot_Loader.html>.
A Fedora-specific guide to GRUBv2 can be found at
<https://fedoraproject.org/wiki/GRUB_2>.


GRUB 0.97

The config file will be in /boot/grub/grub.conf, or /boot/grub/menu.lst
.  Here's an example entry, although as described in the GRUBv2
information, you should really use the procedure described by your
distro to create new entries (or at least copy an existing entry to
create your own, rather than use this one).

title CentOS (2.6.18-194.8.1.el5.emulab1)
        root (hd0,1)
        kernel /boot/vmlinuz-2.6.18-194.8.1.el5.emulab1 ro root=UUID=c25415f0-1eff-4b3e-a03b-99c3fecbbe57  console=ttyS0,115200 selinux=0
        initrd /boot/initrd-2.6.18-194.8.1.el5.emulab1.img

The GRUB 0.97 documentation is at
http://www.gnu.org/software/grub/manual/legacy/grub.html.  A
CentOS-specific guide (for GRUB 0.97) can be found at
http://wiki.centos.org/HowTos/GrubInstallation.


LILO

This entry shouldn't apply to anyone, since we have (or will!)  retired
all our ancient images (i.e., RHL90 --- Red Hat Linux 9.0), but just in
case.  Working with LILO is easy; just add a new entry into
/etc/lilo.conf and run 'sudo /sbin/lilo -v', and it'll show you what it
has done.


Building Emulab Extra Modules

No module is "required" by Emulab, but our software can make use of
them, or they are simply broadly useful.

apod

We developed an "authenticated-ping-of-death" module to provide
LAN-secure, soft-power reboot of machines in our testbed (better method
to restart an unreachable machine whose kernel and network stack are
still running, than a hard power cycle).  To build and install, follow
these instructions:

    $ cd /tmp
    $ cp -rp /share/emulab/emulab-stable/ipod/linux_mod .
    $ cd linux_mod
    $ sudo make
    $ sudo make install
    $ sudo depmod -a -v `uname -r`

openvswitch

Lots of people like having openvswitch!  For version 2.1.0, do something
like

    $ cd /tmp
    $ tar zxf /share/openvswitch/openvswitch-2.1.0.tar.gz
    $ cd openvswitch-2.1.0
    $ ./configure --with-linux=/lib/modules/`uname -r`/build
    $ make
    $ sudo make install
    $ sudo make modules_install
    $ sudo depmod -a -v `uname -r`

Then get it set up, if you haven't already:

    $ sudo ovsdb-tool create /usr/local/etc/openvswitch/conf.db vswitchd/vswitch.ovsschema
    $ sudo /usr/local/share/openvswitch/scripts/ovs-ctl start
    $ sudo ovs-vsctl --no-wait init
    $ sudo /usr/local/share/openvswitch/scripts/ovs-ctl stop