FreeBSD jails with Qjail

Jails & Me

I always wanted to use jails on my FreeBSD server, but I never really dove into it. I did set up a jail using the method described in the FreeBSD Handbook – Chapter 16 Jails successfully, but found that particular method a bit unwieldy.

During my searches I had stumbled upon ezjail, which makes the administration easier: a lot of the manual labor is done for you, it’s easy to keep the jail’s Ports tree updated, and it’s clever enough to save space by linking to a basejail from which jail deployments are “cloned” (saves space!). But still it did not work as easy as I desired… I still had to edit rc.conf when deploying a new jail, it’s best to compile FreeBSD from source when you use ezjail. I have no problems with those two tidbits, as it’s certainly easier than the manual method, but the tool didn’t really grow on me you see?

Another part is: I use ZFS, and ezjail has optional support for ZFS. Great! It can automatically create file-systems for your jails when you deploy them. But after a while you discover it makes you feel a bit out of control. For me a utility that helps with jail administration should focus on that, and not interfere with my file-system.

Enter Qjail

A while ago I stumbled upon Qjail. I did some searching about it, read through the article about in in BSD Mag 2012-06 and decided I had to give it a try myself. So what is it?

Qjail is a utility that can be used to manage FreeBSD jails. It’s designed to make administration of jails easy, and it succeeds very well at it. It’s designed to cope with large environments and keep it all easy to use, even when you deploy dozens of jails. But it also works great if you just have a couple, like me. So a bit like ezjail, but designed with scaling in mind, while also keeping it simple enough so you could manage a lot of jails with it, without it going over your head.

Another advantage of Qjail is that it works better when you like using binary upgrades for FreeBSD. Upgrading by compiling from source can be time consuming. It’s really great when you want to customize FreeBSD to your tastes or require specific hardware support, but I have no need for that. My machine runs great using FreeBSD as distributed by the foundation. I use freebsd-update to keep my OS up-to-date.

I had trouble keeping my jails up to date the binary way when using ezjail. No more with Qjail, it can copy the binaries from the host OS and use those for the basejail, from which all jails are formed. Qjail also performs required networking set-up for your jails on-the-fly. So I no longer have to create aliases for my network interface within /etc/rc.conf. And I can manage the IP address of a jail by telling Qjail what I want it to be.

One thing Qjail does not do: managing your file-system. And after using it, I decided that is a good thing. This way I’m in control of my files and I don’t have to worry about a utility screwing with it. Creating a file-system by hand when using ZFS is easy anyway, and now I can set options I want on there directly before writing any data to it.

Getting started with Qjail

So let’s get started! First we need to install Qjail, which can be done using pkg_add or using Ports, and possibly by using the new pkgng (which I haven’t used yet myself). Personally I prefer using Ports and do it this way:

# cd /usr/ports/sysutils/qjail
# make install clean

Now, you’ve got Qjail installed and ready for use. Well almost. Qjail places it’s jails by default in /usr/jails, so in any case make sure you have enough space available there. If you use ZFS you can opt to create a separate file-system for the Qjail installation. I did this on my server like so:

# zfs create -o compression=lzjb zroot/freebsd9/usr/jails

Adapt this command to your needs, as your system will differ from mine. I made use of the mountpoint inheritance of my existing zroot/freebsd/usr file-system, mounted at /usr. Also note I used compression. This file-system will hold a lot of configuration files and the Ports tree in the basejail, both tend to compress well. Currently this file-system has a compression ratio of 1.71x over here, so it saves some space.

Next you want to install qjail to /usr/jails. It will determine your FreeBSD version, download the appropriate files and create a basejail with them.

# qjail install

After this process completes you will see a couple of new directories in /usr/jails: archive, basejail, flavours and newjail. archive is used to keep archived jails, which I will not go into here. The basejail and newjail directories are used for creating jails, the first holding a FreeBSD system and the second symlinks into the first for use when creating a new jail. flavours is a directory in which you can pre-set configuration files to your liking and later create jails with those presets. To learn more about these things I suggest you take a look at the Qjail manual.

Installing the Ports tree

If you wish to make use of FreeBSD Ports in your jails you can ask Qjail to install a Ports tree into the basejail. If you do not intend to use Ports inside your jails, feel free to skip this step.

# qjail update -p

This installs a fresh Ports tree into basejail using the portsnap utility. You can use the same command to update the Ports tree of the basejail.

Creating a new jail

Time to create a new jail using the just installed qjail utility. But first, you might want to create a new file-system for your jail to live in when you are using ZFS. In my case I will use a quota option, because I’m creating this jail just to host this website. I will need very little space and I when something goes wrong inside it will not use up all my disk space. Feel free to leave out or adapt the quota to your needs.

Remember that the name you choose for the file-system should be the same as the name you are going to give to your jail in the next step. Because you probably want Qjail to install the jail to the file-system you just created. So replace satyr with the name of the jail you wish to create.

# zfs create -o quota=4G zroot/freebsd9/usr/jails/satyr

We can now tell Qjail to create a jail for us. Figure out on which networking interface and with what IP address the jail should work. I did the following:

# qjail create -n re0 satyr

Replace ‘re0’ with your networking interface.

If you wish to build jail of a specific flavour, you can do so by adding -f <flavour_name>

Starting the new jail

We’ve build a new jail, but it’s not running yet. Qjail provides control for starting and stopping jails.

Start the new jail:

# qjail start satyr

And the jail will be started, along with the required networking alias.

To verify the jail is running you can use FreeBSD’s jls, of which the output is easy to understand. Qjail also provides means for listing jails by typing: qjail list It output all jails known to Qjail, including inactive ones. You can tell the state of a jail by looking at the flags Qjail presents. Have a look at the Qjail manual for more in depth information.

Entering the jail

Access to your jail can now be done from the host system:

# qjail console satyr

You will be logged in as root and can start installing software you want to have available. A good one to start with is OpenSSH, so you can log into your jail without having to go through your host system. Remember to secure it just as you would any other host running an SSH daemon.

Updating the base system of your jail

Over time, updates to FreeBSD will become available and you probably want to install the new goodness or even upgrade to another RELEASE. Because FreeBSD Jails need to be the same version as the host OS, you will need to upgrade your jails when you upgrade FreeBSD on the host.

Upgrading your host system can be done using freebsd-update. How this is done can be read in the FreeBSD Handbook – Chapter 25 Updating and Upgrading FreeBSD

When you have performed the upgrade on the host’s FreeBSD, you can upgrade your jails. First you have to stop currently running jails

# qjail stop

And then you can tell Qjail to copy the binaries from the host to the basejail, which forms the base for all the jails you created using Qjail.

# qjail update -b

Remember that what goes for the FreeBSD host also counts for jails. For example if you upgrade to a new major release (e.g. from FreeBSD-8.3 → 9.1), you will have to reinstall/rebuild all installed packages/ports. Upgrades to new minor releases or installing security updates to not require reinstalling/rebuilding unless stated otherwise.