Archive for the ‘linux’ Category

I wanted to setup a Linux kernel development system and the best choice I found was using KVM/Qemu. Both KVM and Qemu use the same virtual hardware specification, but KVM could give near native performance using special processor instructions. Since both of them use the same virtual hardware, the same virtual disk image could be used by either KVM or Qemu. So it is possible to replace ‘kvm’ command everywhere instead of ‘qemu’ at the penalty of performance.

I am using a Debian Unstable amd64 as both host and guest. It is possible to use other distributions, but package names etc. would be different.

First of all install the required packages on host, as root
apt-get install build-essential kernel-package fakeroot \
libncurses5-dev qemu kvm

Compiling the kernel

  1. Get the kernel source code. As of now I am using v3.0-rc5
    Untar/Clone it to say /path/to/linux_src
  2. Kernel Configuration.
    I found the following website for a minimal Linux Kernel Configuration
    But this configuration does not support libata, udev and ext4 which are used by current Debian systems. So, I created a configuration myself which also supports es1370 soundcard and 8139too network card, but there is no USB support. It could be downloaded from here. config-3.0.0.txt. Save it as /path/to/linux_src/.config
  3. Build the kernel. I used make-kpkg because of its simplicity, use
    a CONCURRENCY_LEVEL according to the number of processors in the build machine. It is used to set the “-jn” option for make.
    make-kpkg --initrd --revision=1custorm kernel_image

    Time Taken: With this kernel configuration and concurrency option the build takes around 4 minutes on my AMD Athlon X2 3600 machine.

Setting up a Debian virtual machine

  1. Check the host hardware could support KVM
    egrep 'svm|vmx' /proc/cpuinfo
  2. Create a filesystem image, I am using 3GB raw image format instead of qcow2, because the image could be mounted using a loopback device.
    qemu-img create -f raw debian.img 3G
  3. Get the latest x86_64 Debian netinstall CD image from,
    http://www.debian.org/devel/debian-installer/ Instead, you could install the i386 image, but the custom kernel also should be compiled for x86 architecture.
  4. Boot kvm(or qemu) and install the image. Use something like 512MB of RAM if possible
    kvm -m 512 -boot d -hda /path/to/debian.img \
    -cdrom /path/to/debian-testing-amd64-netinst.iso
  5. To transfer files to the virtual machine, there are two methods, a) using SSH to a running virtual machine, or b) mount the virtual machine image using a loopback device.
    To mount loopback, first find the sector offset
    fdisk -u -l /path/to/debian.img
    offset = sector size * start offset
    Normally that would be 32256 or 1048576
    mount -o loop,offset=1048576 /path/to/debian.img \ /path/to/mountpoint
    To ssh to a running guest, first install ssh server in the guest,
    restart guest with the “hostfwd” qemu option as given in the next section.
    Then do
    ssh root@ -p 10022
  6. To boot into the newly installed virtual machine using the stock Debian kernel
    kvm -smp 2 -m 512 -boot c -vga std -soundhw es1370 \
    -net nic -net user,hostfwd=tcp::10022-:22 \
    -hda /path/to/debian.img

    * SMP: use SMP if host supports it.
    * Networking: Realtek 8139too with “hostfwd” port forwarding so that it is possibe to SSH to the guest.
    * Sound: ES1370 card
  7. To boot using the custom kernel
    kvm -smp 2 -m 512 -boot c -vga std -soundhw es1370 \
    -net nic -net user,hostfwd=tcp::10022-:22 \
    -kernel /path/to/linux_src/arch/x86/boot/bzImage \
    -append root=/dev/sda1 \
    -hda /path/to/debian.img


Read Full Post »

It is possible to do a lot of calculations from command shell itself without using any spreadsheet applications or scripting languages like perl or python. What we require is simple un*x applications like, bc, xargs, awk etc.

To sum a list of numbers

  • There are many different ways to do that, but the best way I like is using xargs, tr and bc
  • $ command | xargs | tr ' ' '+' | bc -l
    where command could be either ‘cat filename’ or the output of any other command. The beauty of this command is that, it works regardless of the number of columns or rows in the file. Even, blank lines would not create any problem. The following file has got extra white spaces and blank lines and xargs arranges them in a single line with one space between them beautifully.

    $cat somefile.txt
     2 3
       4  5 6

    bc is arbitrary precision, so that, it could handle any big number.

  • If we want to sum a column of numbers, it is possible to use awk also, for example to find the sum of the size of all files in a directory, do the following
  • $ ls -l | awk '{s+=$5} END {print s}'

    This command is also not susceptible to blank lines.

  • If there are no blank lines, even paste along with bc could be used
  • $ command | paste -sd+ | bc -l

    To multiply a list of numbers

  • For example, to find factorial of a number, say 10
  • $ for ((k=1;k<=10;k++)) do echo $k; done | xargs | tr ' ' '*' | bc -l


    $ for ((k=1;k<=10;k++)) do echo $k; done | paste -sd* | bc -l

    To sum a list of hexadecimal numbers and to display their sum in hexadecimal

    The following command line runs a ‘for’ loop to generate numbers from 0x00 to 0xff (255), compute the sum and prints the output in hexadecimal itself.

    $ for ((k=0;k<256;k++)) do printf "0x%x\n" $k; done \
    | sed -e 's/^0x//g' | tr '[a-f]' '[A-F]' \
    | xargs echo 'obase=16;ibase=16;0' | tr ' ' '+' | bc -l

    Actually bc does not accept the hexadecimal prefix ‘0x’, that is the reason of using sed to remove the 0x prefix.  Also bc expects uppercase of 0xa to 0xf, so ‘tr’ is used translate them.  The ‘0’ inside the echo statement is used so that there is a number before the first ‘+’, if we want to remove the hack of using a ‘0’, the above command could be modified to the following.

    $ for ((k=0;k<256;k++)) do printf "0x%x\n" $k; done \
    | sed -e 's/^0x//g' | tr '[a-f]' '[A-F]' \
    | xargs | tr ' ' '+' | xargs echo 'obase=16;ibase=16;' | bc -l

    Or with a sub shell and a single xargs

    $ for ((k=0;k<256;k++)) do printf "0x%x\n" $k; done \
    | sed -e 's/^0x//g' | tr '[a-f]' '[A-F]' \
    | (echo -ne 'obase=16;ibase=16;' && xargs) | tr ' ' '+' | bc -l

    The above commands could be used for binary or octal also by replacing the ibase and obase to 2 or 8 respectively. Note that, it is better to write obase first. If ibase is written first, then obase has to be specified in terms of ibase
    That means
    and ibase=16;obase=10 are the same

    Read Full Post »

    « Newer Posts

    %d bloggers like this: