Creating a Minimal Kernel Development Setup using KVM/Qemu

I wanted to setup a Linux kernel development system and the best choice I found was using KVM/Qemu. Both KVM and Qemu use the same virtual hardware specification, but KVM could give near native performance using special processor instructions. Since both of them use the same virtual hardware, the same virtual disk image could be used by either KVM or Qemu. So it is possible to replace ‘kvm’ command everywhere instead of ‘qemu’ at the penalty of performance.

I am using a Debian Unstable amd64 as both host and guest. It is possible to use other distributions, but package names etc. would be different.

First of all install the required packages on host, as root
apt-get install build-essential kernel-package fakeroot \
libncurses5-dev qemu kvm

Compiling the kernel

  1. Get the kernel source code. As of now I am using v3.0-rc5
    Untar/Clone it to say /path/to/linux_src
  2. Kernel Configuration.
    I found the following website for a minimal Linux Kernel Configuration
    http://gruba.blogspot.com/2009/08/qemu-minimal-linux-kernel-config.html
    But this configuration does not support libata, udev and ext4 which are used by current Debian systems. So, I created a configuration myself which also supports es1370 soundcard and 8139too network card, but there is no USB support. It could be downloaded from here. config-3.0.0.txt. Save it as /path/to/linux_src/.config
  3. Build the kernel. I used make-kpkg because of its simplicity, use
    a CONCURRENCY_LEVEL according to the number of processors in the build machine. It is used to set the “-jn” option for make.
    export CONCURRENCY_LEVEL=2
    make-kpkg --initrd --revision=1custorm kernel_image

    Time Taken: With this kernel configuration and concurrency option the build takes around 4 minutes on my AMD Athlon X2 3600 machine.

Setting up a Debian virtual machine

  1. Check the host hardware could support KVM
    egrep 'svm|vmx' /proc/cpuinfo
  2. Create a filesystem image, I am using 3GB raw image format instead of qcow2, because the image could be mounted using a loopback device.
    qemu-img create -f raw debian.img 3G
  3. Get the latest x86_64 Debian netinstall CD image from,
    http://www.debian.org/devel/debian-installer/ Instead, you could install the i386 image, but the custom kernel also should be compiled for x86 architecture.
  4. Boot kvm(or qemu) and install the image. Use something like 512MB of RAM if possible
    kvm -m 512 -boot d -hda /path/to/debian.img \
    -cdrom /path/to/debian-testing-amd64-netinst.iso
  5. To transfer files to the virtual machine, there are two methods, a) using SSH to a running virtual machine, or b) mount the virtual machine image using a loopback device.
    To mount loopback, first find the sector offset
    fdisk -u -l /path/to/debian.img
    offset = sector size * start offset
    Normally that would be 32256 or 1048576
    mount -o loop,offset=1048576 /path/to/debian.img \ /path/to/mountpoint
    To ssh to a running guest, first install ssh server in the guest,
    restart guest with the “hostfwd” qemu option as given in the next section.
    Then do
    ssh root@127.0.0.1 -p 10022
  6. To boot into the newly installed virtual machine using the stock Debian kernel
    kvm -smp 2 -m 512 -boot c -vga std -soundhw es1370 \
    -net nic -net user,hostfwd=tcp::10022-:22 \
    -hda /path/to/debian.img

    Here,
    * SMP: use SMP if host supports it.
    * Networking: Realtek 8139too with “hostfwd” port forwarding so that it is possibe to SSH to the guest.
    * Sound: ES1370 card
  7. To boot using the custom kernel
    kvm -smp 2 -m 512 -boot c -vga std -soundhw es1370 \
    -net nic -net user,hostfwd=tcp::10022-:22 \
    -kernel /path/to/linux_src/arch/x86/boot/bzImage \
    -append root=/dev/sda1 \
    -hda /path/to/debian.img

Published by Anand Sivaram (आनन्दः )

twitter.com/anand_sivaram https://www.linkedin.com/in/anandsivaram/

6 thoughts on “Creating a Minimal Kernel Development Setup using KVM/Qemu

  1. Can you tell me how to develop the Linux kernel for PC. I have experience developing Linux kernel for embedded boxes based on ARM and MIPS but i have no idea how to do it for PC. I find it too clumsy to do always the kernel based development using the embedded Linux boxes.

    I want to do some thing like this… DO the development in a virtual or Linux machine and run the new kernel on another virtual machine. Means i transfer the module to the VM using some technique. For example in the current case we use scp command.

    Have you tried something like that please let me know?

  2. @Prajosh.Premdas

    Your requirement is the same as what is described here. You have both options of 1) working on your native/another real machine or 2) using a virtual machine for development. Choise of 1 or 2 entirely depends upon what you want to do. If the goal is some important components like Scheduler, VM, file system etc. it may be better to start with a virtual machine, otherwise your main machine may get locked up. Or if it is something very simple, either options would be fine.

    If you are planning to run the kernel in a real machine, copy the current .config of that machine, that is /boot/config-$(uname -r), remove unwanted drivers and configurations to save space and time and build the kernel. Or if you want to try KVM/Qemu, create an instance first, see that it is booting up before getting into kernel compilation.

  3. Great post! Two things anyway:
    1. the minimal config is downloadable in openoffice format, what about to put a raw text file there?
    2. I think the fastest way to build a custom kernel to be used as starting kernel for KVM is: make -j{2*num_of_cpus} bzImage, no need to create a debian package right?

    Cheers.

    1. 1. It is actually a text file only, just copy it to your kernel tree and rename it as “.config”. WordPress did not allow me to upload text file
      2. You could use bzImage, but then you have to copy it many manually, update grub etc. Debian package is a convenient method on Debian/Ubuntu which would do this work.

      Thanks for your comments.

  4. There are two kernels involved for 1) Host machine and 2)Guest machine. We do not need to modify the host machine in any way provided the required kvm, kvm-intel or kvm-amd modules are present. This article is about compiling the guest kernel for debug purpose only.

Leave a Reply to prajosh.premdas Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: