Ubuntu Server: Kernel Configuration Considerations

The latest version of Ubuntu server edition comes with both server and desktop kernels. How do they differ, and what options are available with each?

Last week we looked at Ubuntu Server’s documentation, discussed hardware requirements, tried to figure out what sets Ubuntu Server apart from Ubuntu Desktop, and what’s included in the current release, 7.10 (Gutsy Gibbon). Ubuntu itself is not very helpful with these things, and I don’t like to write reviews that complain without also offering some answers, so today we’re going to learn how to dig into Ubuntu without installing it and find out these things for ourselves. We still need the installation .iso image, but at least we won’t have to install it or even burn it to a CD to learn some useful information.

What Is a “Server” Kernel?

We’re going to compare the /boot/config-2.6.22-14-server and /boot/config-2.6.22-14-generic files and find out exactly what Ubuntu considers a server kernel vs. a desktop kernel is. We’ll do this by mounting the installation .iso in a temporary directory, and then extract and compare the two files:

# mkdir temp
# mount -o loop ubuntu-7.10-server-i386.iso
# cd temp
# find temp -name linux-image*
temp/pool/main/l/linux-meta/linux-image-generic_
2.6.22.14.21_i386.deb
temp/pool/main/l/linux-meta/linux-image-server_
2.6.22.14.21_i386.deb

temp/pool/main/l/linux-source-2.6.22/linux-image-
2.6.22-14-generic_2.6.22-14.46_i386.deb

temp/pool/main/l/linux-source-2.6.22/linux-image-
2.6.22-14-server_2.6.22-14.46_i386.deb

Check it out, we get both desktop and server kernels in one location, so this is going to be easy. We want the last two, as those are the real kernel packages, which the linux-meta packages are not. Copy them into separate directories for unpacking. .deb packages are special archives that contain ordinary tar archives, and these two packages contain files with the same names: control.tar.gz, data.tar.bz2, and debian-binary. So you can’t unpack them in the same directory.

Use the ar and tar commands to unpack the .debs, and then to unpack their data.tar.gz files:

# ar -x linux-image-2.6.22-14-server_2.6.22-14.46_i386.deb

# tar jxvf data.tar.bz2

Now you can pluck out the boot/config-2.6.22-14-server and boot/config-2.6.22-14-generic files, copy them into the same directory for convenience, and diff them:

# diff --suppress-common-lines  -y config-2.6.22-14-server
  config-2.6.22-14-generic

This gives us some nice manageable output — around 50 lines, rather than about 3,100 lines per file. So let’s take a look at diff’s output to see what’s different.

There are four different types of I/O scheduling: CFQ (Completely Fair Queuing), Deadline, NOOP, and Anticipatory. Ubuntu makes CFQ the default for desktop kernels, and Deadline for server kernels. The goal is the same for all of them: to optimize hard disk bandwidth for different classes of workloads. In your configuration file, this is the CONFIG_DEFAULT_IOSCHED option, plus the CONFIG_IOSCHED_CFQ, _DEADLINE, _AS, and _NOOP options.

• CFQ tries to balance all read/write requests equally.
• 
  Deadline gives a higher priority to read requests, and will re-order read/write requests aggressively to meet the goal of completing read requests within a specified time, without “starving” write requests, which are not given deadlines.

• Anticipatory aims to reduce latency by giving priority to already-running applications. It is supposed to be suitable for smaller systems with one or two hard disks, and single or dual-core CPUs.

• NOOP is a minimal scheduler for systems with hardware that handles I/O scheduling, like large SCSI RAID arrays.

The question of which one is appropriate depends on your systems and how you use them: how many CPUs, how many hard disks and controllers, what types of applications, and the loads your systems have to handle. You can run benchmarks, and then tune your systems accordingly. You can pass scheduler options in as boot-time options, or you can even enable different schedulers per block device and change them on-the-fly (see Resources). The Ubuntu defaults are good starting points, and if you must tweak the settings, they’re just as tweakable as on any Linux.

The server kernel has kernel preemption turned off (CONFIG_PREEMPT_NONE=y), while the desktop kernel has it enabled (CONFIG_PREEMPT_BKL=y, CONFIG_PREEMPT_VOLUNTARY=y). Preemption works along with scheduling to fine-tune performance, efficiency and responsiveness. In non-preemptive kernels, kernel code runs until completion; the scheduler can’t touch it until it’s finished. But the Linux kernel allows tasks to be interrupted at nearly any point (but not when it is unsafe, which is a whole huge fascinating topic all by itself), so that tasks of lesser-priority can jump to the head of the line.

This is appropriate for desktop systems because users typically have several things going at once: writing documents, playing music, Web surfing, downloading and so on. Users don’t care how responsive background applications are; they care only about the ones they’re actively using. So if loading a Web page takes a little longer while the user is writing an e-mail, it’s an acceptable trade-off. Overall efficiency and performance are actually reduced but not in a way that annoys the user.

On servers you want to minimize any and all performance hits, so turning off preemption is usually the best practice.

The 32-bit server kernel supports up to 64 GB of memory; the desktop kernel, a mere 4 GB (CONFIG_HIGHMEM64G=y, CONFIG_HIGHMEM4G=y). You’ll only see these options in 32-bit kernels because the 32-bit address space is big enough to support only 4 GB without trickery. Or by using the Intel Physical Address Extension (PAE) mode, if you want to get technical. Linux supports PAE, and you also need PAE support in your CPU. Anything newer than a Pentium Pro or AMD K6-3 should be fine. On a 64-bit system you won’t see any memory options because it doesn’t need hacks to overcome a lack of memory addressing space; you should be fine until your needs exceed 16 exabytes of RAM.

Both kernels support on-demand interrupt timers (CONFIG_NO_HZ=y), or the so-called “tickless” option. This means that during periods of no activity, the system goes into a truly idle state, which is supposed to save on power and cooling.

The server kernel is set to a timer interrupt rate of 100 Hz (CONFIG_HZ=100, CONFIG_HZ_100=y), which means it accepts 100 interrupts per second. Another way to think of this is the kernel looks up and peers around 100 times per second for something to do. The desktop kernel is set to 250 Hz — lower numbers equal lower overhead and higher latency; higher numbers equal higher overhead and lower latency. Higher numbers generally mean the system feels more responsive, at the price of higher CPU usage. Some processes require more interrupts; for example, video processing and VoIP servers need 1000 Hz. If you need to change the Hz value it requires a kernel re-compile.