Virtualize Linux to test your apps

With a steady flow of improvements and bug fixes to existing software, the open source world is in constant flux. Staying at the bleeding edge of software upgrades can be a full-time job. One of the trickiest parts about upgrading your software is that you never really know if your applications are going to work after the upgrade is complete. Most software packaging systems offer a rollback feature, but that's often not enough; ideally, you want to play with these new updates to test and try them in an environment where they can't hurt anything.

Like a kid on the playground, you want your own sandbox to play in, where you can make a mess and not worry about picking up.

Virtualization is often used to separate (or "sandbox") applications and systems from the others on the same hardware. Linux supports many different forms of virtualization ranging from hardware emulators to pure hardware virtualization. One recent standout in the growing list of virtualization technologies is Xen, developed at the University of Cambridge. Xen is noteworthy because of its unprecedented performance and security.

Along came Xen

Developed as a research project at the University of Cambridge, Xen has gained a lot of momentum in the open source community. Xen is a paravirtualizing VMM (Virtual Machine Monitor), meaning that the operating system is modified in select areas to make calls into the hypervisor, whereas the applications that run on that operating system are unmodified. While other virtualization systems like VMWare demonstrate true virtualization (in which they don't have to modify the running operating systems), they still need to do on-the-fly translation of the machine code, which can affect performance.

Because Xen requires a modified operating system, you cannot just take a current Linux kernel and run it under the Xen hypervisor unless it is ported to the Xen architecture. However, if the current system can use a new Linux kernel that has been ported to the Xen architecture, you can run your existing system without change.

Finding Xen

The source code for Xen is available from the project page (see the Resources section for the link), but if you're already running a Linux distribution, you may be able to get Xen from a package update or installation. These major distributions are packaging Xen:

Fedora Core.
SUSE Linux.
Debian.
Experimental e-builds at Gentoo's bugzilla are available.
Check with your distribution -- if it is not in this list, you may have to use the source code directly. Beyond distribution-specific install procedures, everything works the same in Xen on any distribution.

Using Fedora Core

I have a working installation of Fedora Core 3, and to keep this article brief, I'm going to talk about getting the system up and running on a Fedora Core.

Fedora sports a great community of Xen users already, even though the packages have only recently landed in the development channels. Because of the number of people trying out Xen on Fedora, you're probably going to have lots of success finding online help.

Installing Xen

To help you get up and running quickly, this section highlights the major points of the Quick Start Guide from the Resources section . As Xen updates are pushed into Fedora, the Quick Start Guide will most likely change over time and hopefully the wiki will remain updated to those changes.

Start with a minimal server installation for the base machine. This base machine is your hypervisor and won't be running any of the server applications. You'll have a chance to install the packages you want on the Xen servers that run atop this machine, so there's no need to install the applications you want to run just yet.

Once you have a system running, you'll need to update it to the latest development version of Fedora, called rawhide. You can do this by going through the elements in the /etc/yum.repos.d/ directory and changing all of them except the fedora-devel.repo to be enabled=0; change fedora-devel.repo to enabled=1.

Now that your yum repository is set up, you need to update the machine to rawhide so you can start using the latest Xen release.

Listing 1. Update FC3 to rawhide and install the Xen packages

yum update

yum install xen kernel-xen0 kernel-xenU
 

Next, create a dummy filesystem to work with.

Listing 2. Format and set up Xen server filesystems

mkdir -p /xen/base
dd if=/dev/zero of=/root/base.img bs=1M count=1 seek=1024
mkfs.ext3 /root/base.img
mount -o loop /root/base.img /xen/base
 

With your Xen server filesystems formatted and mounted, the next step is installing the packages necessary to run something on them. To run a Web server on your Xen servers, you'll need to install the Web-server set of packages. (Warning: This will take a while! Run yum, then go for a walk.)

Listing 3. Installing Xen server packages

yum --installroot=/xen/base -y groupinstall web-server \
  --enablerepo=base --disablerepo=development
 

(You should be walking now.)

You now have all the packages you need installed, but you need to drop in a simple fstab file, which tells the Xen servers that they are going to be getting a root device sda1, which they will use as root. This root device is actually a virtualized device coming from the hypervisor, but your servers won't know that.

Listing 4. Xen server fstab file

/dev/sda1  /  ext3 defaults 1 1
none   /dev/pts devpts  gid=5,mode=620 0 0
none                    /dev/shm        tmpfs   defaults 0 0
none                    /proc           proc    defaults 0 0
none                    /sys            sysfs   defaults 0 0
 

Write this file into /xen/base/etc/fstab. Without a normal Fedora install, no fstab was created for you. Future versions of Fedora are expected to include a tool for handling this issue.

Finally, you need to do some hacks to get the system running properly. Xen is currently having issues with initrd, so you need to create a few device codes for yourself if you want the system to boot. Then, in order to keep things running smoothly, it's beneficial to move the TLS libraries; at the moment, they interact badly with Xen. Once you've done that, you can unmount this filesystem.

Listing 5. Final hacks to get running

for i in console null zero ; do MAKEDEV -d /xen/base/dev -x $i ; done

mv /lib/tls /lib/tls.disabled
mv /xen/base/lib/tls /xeb/base/lib/tls.disabled

umount /xen/base/
 
Running Xen

Now that you've painfully set up your guests by hand, you can actually reboot the machine and start using the Xen kernel. You'll probably see a lot more printouts than normal, and you'll also get scary message about the TLS library (the one you moved out of the way in the previous section).

Xen requires configuration files for each server you run. Although you can create a single dynamic configuration file for all of your servers, I recommend creating two static configuration files, one for each. As you can see, the disk label tells the servers that your file image is a sda1 device. The xenU kernel you are using and the memory that the server guest will receive are also specified.

Listing 6. Xen server configuration files

/etc/xen/base
kernel ="/boot/vmlinuz-2.6.10-1.1141_FC4xenU"
memory = 64
name = "BaseServer"
nics = 1
disk = ['file:/root/base.img,sda1,w']
root = "/dev/sda1 ro"

/etc/xen/test
kernel ="/boot/vmlinuz-2.6.10-1.1141_FC4xenU"
memory = 64
name = "TestServer"
nics = 1
disk = ['file:/root/test.img,sda1,w']
root = "/dev/sda1 ro"
 

To get the servers up and running, start up the Xen service with the command xend start and then create your BaseServer from your base config file with the command xm create base. With those actions, you'll be taken into your Xen guest and you can watch it boot up. If at anytime you want to escape the guest console, press Ctrl-] and go right back to the hypervisor console.
Using Xen

You've now spent some time editing configurations and getting your system set up to run a couple of Xen servers. After all that hard work, what is your reward?

Now you have two copies of a system running the same kernel, virtualized on the same machine. If your BaseServer represents the stable environment that you'd like to run your Web service on normally, then your TestServer can represent the latest software updates that Fedora rawhide provides. You can keep upgrading your TestServer system at will, trying all the new versions of software updates -- when you feel it's stable enough, you can make the TestServer run as the BaseServer.

Setting up the BaseServer

When the BaseServer is operating, it's a good time to change the configuration for the apache httpd service you'll be running. You might want to turn off ssl by moving the ssl.conf file out of the /etc/httpd/conf.d/ directory; otherwise, you'll need to generate a certificate for the servers. Also you might need to add an apache user to the system.

The Xen hypervisor has automatically given you a virtual network device to work with. If you can run dhcp on this device, just run dhclient eth0; you should get an IP address for your BaseServer.

Setting up the TestServer

Once you have your configuration setup for the BaseServer, run poweroff from the BaseServer console to shut down the instance. If you've already logged out of the BaseServer, regain access to it using xm console BaseServer.

Now copy the base image file by using cp /root/base.img /root/test.img so you can have a duplicate copy of the same filesystem. When you run xm create -c test, it runs the same server as the BaseServer, but it's called TestServer. Log into the TestServer and enable rawhide support like you did in the Installing Xen section, and run yum update.

Your TestServer instance has all the same configurations as your BaseServer did, but you've upgraded to the latest packages that Fedora rawhide has to offer. Here's where you get to play around to see if your Web site is still working.