Feeds:
Posts
Comments

There are many network throughtput and performance measurement tools available. One of the most widely used one is “iperf”. But, I found “netcat” to be a very versatile and fantastic tool to do the same.

The only “extra” requirement is to have shell access to both sides, so that netcat could be used.

First install the required tools
apt-get install netcat-traditional bwm-ng pv

Considering “machine1” and “machine2” are the two systems on both side of the link. Do the following

machine1:$ nc -l -p 5000 | pv > /dev/null
machine2:$ dd if=/dev/zero | pv | nc ip_address_or_hostname_of_machine1 5000

Logic:
Netcat of machine1 opens a TCP listening socket and dumps the data to /dev/null. machine2 reads from /dev/zero and forwards that data to machine1 through netcat. Here, flow control is done by TCP stack and keeps the throughput just below the network link speed. Normally “pv” prints the pipe throughput in bytes, so multiply the real time value of “pv” by 8 to get the throughput in bits.

Tools like “bwm-ng” could be used to monitor the througput of the the interface instead of or along with “pv” also.

Some version of of netcat does not require “-p” in listening mode, so the comment could be “nc -l 5000” instead. In the sending side “cat” or “pv” could be used instead of “dd” also. But “dd” prints a nice summary when it is interrupted.

Of course, this is not completely one directional because TCP acknowledgements would be forwarded from machine1 to machine2, but that is much smaller compared to the data flowing in the forward direction.

If we have the liberty of having more shells, one more pair of netcat sessions could be opened in the reverse direction so that simultaneous/full duplex throughput could be measured.

I used this method consistently with Gigabit Ethernet and WiFi many times.

“Every block of stone has a statue inside it and it is the task of the sculptor to discover it” – Michelangelo

As of now, there are around 1.5 Billion people in the world without access to electricity grid, out ot which the share of India would come to around 400 Million people or around around 100 Million families. Majority of them live in the 80000 or so non grid connected villages in India.

They all depend mainly on Kerosene lanterns as the source of light. That makes Kerosene a very sensitive commodity in India. Kerosene is sold as a subsidized fuel in Government run ration shops for the poor people. Currently it is sold for around Rs. 12.50 per litre, but government gives around Rs.19.60 as subsidy on top of that to meet the actual open market price of around Rs. 32.00 Poor families are eligible to get around 6 lites per month at this subsidized rate.

Around 12 Billion litres of Kerosene are supplied to Ration shops annually and as much as 40% of that which is around 5 Billion litres are getting stolen to black market. The money lost by Indian government is around Rs. 10000 Crore (US$ 2.2 Billion) in subsidy itself. That gives around 7 Billion litres for the actual use.

There is another dimension for the “lost” kerosene. It is mainly used to adulterate petrol and diesel fuel. Adulterated petrol and diesel causes less mileage, higher maintenace cost, higher smoke and particulate pollution and emission.

Kerosene Usage Statistics

Summarizing Kerosene usage, Carbon Dioxide Emission and actual costs data.

Table1: Kerosene Statistics

Usage 7 Bn Litres Black 5 Bn Litres Total 12 Bn Litres
Energy Content KWh 70 Bn 50 Bn 120 Bn
CO2 Emission 17.5 MMT 12.5 MMT 36 MMT
Total Cost
(Rs 32.00/Litre)
INR 22400 Cr
USD 5000 Mn
INR 16000 Cr
USD 3560 Mn
INR 38400 Cr
USD 8540 Mn
Subsidized Cost
(Rs 12.50/Litre)
INR 8750 Cr
USD 1950 Mn
INR 6250 Cr
USD 1390 Mn
INR 15000 Cr
USD 3340 Mn
Govt. Subsidy
(Rs 19.60/Litre)
INR 13720 Cr
USD 3050 Mn
INR 9800 Cr
USD 2180 Mn
INR 23520 Cr
USD 5230 Mn

* Assuming: kerosne has an energy density of 10KWh/36MJ per Litre and it produces 2.5Kg of CO2 per Litre. USD 1.00 is around INR 45.00

* CO2 emission of India is around 1800 MMT, so this figure is around 2% of that.

Kerosene Lamp Efficiency

As per a study conducted by Lawrence Berkey National Labaratories, kerosene lamps energy consumption and light output vary a lot. In fact, a lot of kerosene is evaporated through the wick without getting burnt. Typical kerosene lanterns use around 5mL to 42mL of kerosene per hour, whereas light outputs vary from around 8 Lumens to 67 Lumens. This corresponds to light efficiency of 935 Lumen.Hour/Litre to 1914 Lumen.Hour/Litre. This leads to an energy efficiency of just 0.1 to 0.2 Lumen/Watts.

As a comparison, even an average incandescent lamp which many countries want to ban is more than 50 times better than these kerosene lamps. !!!!! To put it better, kerosene lamps are the costiest and dirtiest way to generate the same light output.

Apart from wastage of fuel, other problems like smoke, safety, burning hazard, pollution etc. associated with kerosene lamps are not discussed here. Also cost of kerosene lamps, running cost to buy wicks etc. are not mentioned.

Assuming average of around 1428 Lumen.Hour/Litre of Kerosene and 7 Billion Litres of usage, the total light produces is around 10000 Billion Lumen.Hours

Table 2: Electricity requirement from a baseload plant to generate the same light

Light Source Incandescent
10 lumens/Watt
Fluroscent
60 lumens/Watt
LED
100 lumens/Watt
Energy Usage
KWh/Year
1000 Mn 166 Mn 100 Mn
Energy Usage
KWh/Day
2750 x 1000 455 x 1000 275 x 1000
Constant Baseload
Equivalent
114 MW 18.9 MW 11.4 MW

This tables summarizes the power requirement of Incandescent, Fluorescent and Light Emitting Diode to produce the same amount of light. Light is normally required only in the evening time for around 5 hours. So 5 times peak power requirement could be assumed during evenings, where as no power is required for the rest of the time.

That means with a power plant of 1000 MW operating in the evening for 5 hours and using standard CFL lighting, 10 times more light could be delivered to the same people. (18.9 MW x 5 x 10 times light) This is not even 1% of the electricity production capacity of India. That is around 600 lumens of light per family compared to the meager 60 lumens which a kerosene lamp could provide.

Table 3: Power produced if the same kerosene is diverted to baseload power plants

7 Bn Litres
Usage
12 Bn Litres
Usage
Energy Content 70 Bn KWh 120 Bn KWh
Energy Content
in MW.Year
7985 MW.Year 13689 MW.Year
Standard DG Output
at 20% efficiency
1600 MW.Year 2700 MW.Year
Combined Cycle Gas Turbine Output
at 50% efficiency
4000 MW.Year 6800 MW.Year

This shows that the same kerosene could drive a 6800 MW Combined Cycle Gas Turbine based Power plant continuously for one year.

Using Clean Technologies for the same lighting scenario

As seen from above, to produce 10 times more light output using CFL, we require 4.55 Million KWh of electricity per day (189 MW baseload power plant equivalent).

Using Solar Photovoltaic
Being a tropcial country, India gets very good solar insolation of 5.5 KWh/m2/day or 2000KWh/m2/year average. Generating 4.55 Million KWh requires 830 MW Solar Panel. Currently solar modules are available below $2 per Watt. Maximum Power Point Trackers and inverters are available at around $0.60 per Watt and $0.75 per Watt respectively. Considering around $4 per Watt for all of these, it would lead to $3320 Million

Using Small Wind Power
With a capacity factor of 25%, the name plate capacity required to generate 189 MW of baseload power is 756 MW. Normally 2MW turbines are available at around $2 Million per MW. But small ones which generate a few KW are more expensive at around $4 per Watt. That comes to $3024 Million

Storage using Batteries
Standard Lead Acid Batteries cost around $200 per KWh. Assuming a depth of dischare of 50% it is required to have double the name plate capacity, so assuming $400 per KWh for the name plate capacity of 4.55 Million KWh, the requirement would be around $1820 Million.

Other Costs
There are many other expenditures associated like installations etc. In this case, many of them could be shared among different users. Assuming a cost of $40 associated with these, the total cost for 100 Million users would be $4000 Million.

Table 4: Summary of Clean Technology Costs

Energy Requirement
per Day
4.55 Mn KWh
Using Solar Systems 830 MW
at $4.00 per Watt
$3320 Mn
Using Windpower Systems 756 MW
at $4.00 per Watt
$3024 Mn
Storage Batteries 4.55 Mn KWh
at $200 per KWh
$1820 Mn
Other Miscellaneous 100 Million users
at $40 per user
$4000 Mn
Total Using Solar $9140 Mn
Total using Wind $8844 Mn

Conclusion:

To rephrase what Michelangelo said: “Money is there with the people, it is the task of the Government to reprioritize it”

References

http://www.interfaceflor.eu/internet/web.nsf/webpages/536_IN.html?OpenDocument&

http://articles.timesofindia.indiatimes.com/2011-01-30/special-report/28373372_1_kerosene-prices-ration-shops-kilolitre

http://articles.timesofindia.indiatimes.com/2011-01-27/india/28363076_1_kerosene-ration-shops-fuel-mafia

http://evanmills.lbl.gov/pubs/pdf/offgrid-lighting.pdf Table 1/Page 4

http://solarbuzz.com/facts-and-figures/retail-price-environment/module-prices

http://www.popularmechanics.com/home/improvement/energy-efficient/4321836


Edited on 26/07/2011:
I got a couple of comments about using biomass. Actually biomass gasifiers or digesters could be used to produce electricity also. One advantage of using this methods is that electricity could be generated as and when it is required. In many cases, this could be cost effective compared to solar or wind coupled with energy storage. Around 200 Million tonnes of biomass is used in India for cooking purpose itself.

There have been a lot of proposals about having Solar Power Stations on Moon. The plan is to create a belt of solar modules around 11000km equator. The power will be taken to Earth using Microwave beams. After going through some of their ideas, I thought it was not an idea at all. Some of them were very basic so that I do not know how they over come them. Consider the following points.

  1. Earth receives 6000 times solar energy than we require, so there is no need to look outside.
  2. http://en.wikipedia.org/wiki/Space-based_solar_power#On_the_Moon These ideas are not new. They came back in around 1970.
  3. Current launch system costs are exorbitantly high, somewhere around $5000 per kg. Until systems like space elevators etc. become commercially available, it would be really expensive to launch these systems. Also note that, Moon does not have an atmosphere but it has a gravity of around 1.62 m/s2 which is 16% of Earth. So reverse thrust rockets would have to be used to land anything on Moon. Also these rockets and their fuel also have to be taken from Earth at the same cost of $5000 per kg.
  4. Actually only one side of Moon is visible, and Moon has a day of around 29 days. So for 14 days when the Sun is on the other side of the Moon, that means around what we call as New Moon times, no power or very little power could be sent to Earth.
  5. As per Wikipedia article, microwave beam power is around 23mW/cm2 which 25% of Solar radiation. So large area rectennas have to be used.
  6. Remember that Apollo program cost was $25.4 Billion actually (1973 report) but estimated cost in 2005 was around $170 Billion. (from Wikipedia)

Currently solar modules prices are going down very fast and very soon they would reach the landmark $1 per Watt level. There have been a lot of work going on in that area. Even Printed Solar Cells work is progressing . If these solar panel could be depoyed on everywhere on our own Earth, why do we need to taken them to Moon at all?

OpenEmbedded uses SRC_URI to determine where the source code is taken from. For many projects it would be convenient to use source code from local repository due to many reasons, in order to support that, it is possible to use file URI as ‘file://’

For example,
SRC_URI=”file:///path/to/local_repo/whatever_project”
This can create problem because it is not sharable between different people due to absolute path. It may be possible to update FILESPATH varible, but I preferred to use the following method.

  • In $HOME/.bashrc, export a new environment variable, say LOCAL_REPO
    export LOCAL_REPO='/path/to/local_repo'
  • In /path/to/oe/source-me.txt, add the following lines
    if [ -z ${LOCAL_REPO} ]; then
      echo No LOCAL_REPO environment variable set
      return 2
    fi
    export BB_ENV_EXTRAWHITE="${BB_ENV_EXTRAWHITE} LOCAL_REPO"
    
  • In related bb files, use SRC_URI as
    SRC_URI = "file://${LOCAL_REPO}/whatever_project"

I wanted to setup a Linux kernel development system and the best choice I found was using KVM/Qemu. Both KVM and Qemu use the same virtual hardware specification, but KVM could give near native performance using special processor instructions. Since both of them use the same virtual hardware, the same virtual disk image could be used by either KVM or Qemu. So it is possible to replace ‘kvm’ command everywhere instead of ‘qemu’ at the penalty of performance.

I am using a Debian Unstable amd64 as both host and guest. It is possible to use other distributions, but package names etc. would be different.

First of all install the required packages on host, as root
apt-get install build-essential kernel-package fakeroot \
libncurses5-dev qemu kvm

Compiling the kernel

  1. Get the kernel source code. As of now I am using v3.0-rc5
    Untar/Clone it to say /path/to/linux_src
  2. Kernel Configuration.
    I found the following website for a minimal Linux Kernel Configuration
    http://gruba.blogspot.com/2009/08/qemu-minimal-linux-kernel-config.html
    But this configuration does not support libata, udev and ext4 which are used by current Debian systems. So, I created a configuration myself which also supports es1370 soundcard and 8139too network card, but there is no USB support. It could be downloaded from here. config-3.0.0.txt. Save it as /path/to/linux_src/.config
  3. Build the kernel. I used make-kpkg because of its simplicity, use
    a CONCURRENCY_LEVEL according to the number of processors in the build machine. It is used to set the “-jn” option for make.
    export CONCURRENCY_LEVEL=2
    make-kpkg --initrd --revision=1custorm kernel_image

    Time Taken: With this kernel configuration and concurrency option the build takes around 4 minutes on my AMD Athlon X2 3600 machine.

Setting up a Debian virtual machine

  1. Check the host hardware could support KVM
    egrep 'svm|vmx' /proc/cpuinfo
  2. Create a filesystem image, I am using 3GB raw image format instead of qcow2, because the image could be mounted using a loopback device.
    qemu-img create -f raw debian.img 3G
  3. Get the latest x86_64 Debian netinstall CD image from,
    http://www.debian.org/devel/debian-installer/ Instead, you could install the i386 image, but the custom kernel also should be compiled for x86 architecture.
  4. Boot kvm(or qemu) and install the image. Use something like 512MB of RAM if possible
    kvm -m 512 -boot d -hda /path/to/debian.img \
    -cdrom /path/to/debian-testing-amd64-netinst.iso
  5. To transfer files to the virtual machine, there are two methods, a) using SSH to a running virtual machine, or b) mount the virtual machine image using a loopback device.
    To mount loopback, first find the sector offset
    fdisk -u -l /path/to/debian.img
    offset = sector size * start offset
    Normally that would be 32256 or 1048576
    mount -o loop,offset=1048576 /path/to/debian.img \ /path/to/mountpoint
    To ssh to a running guest, first install ssh server in the guest,
    restart guest with the “hostfwd” qemu option as given in the next section.
    Then do
    ssh root@127.0.0.1 -p 10022
  6. To boot into the newly installed virtual machine using the stock Debian kernel
    kvm -smp 2 -m 512 -boot c -vga std -soundhw es1370 \
    -net nic -net user,hostfwd=tcp::10022-:22 \
    -hda /path/to/debian.img

    Here,
    * SMP: use SMP if host supports it.
    * Networking: Realtek 8139too with “hostfwd” port forwarding so that it is possibe to SSH to the guest.
    * Sound: ES1370 card
  7. To boot using the custom kernel
    kvm -smp 2 -m 512 -boot c -vga std -soundhw es1370 \
    -net nic -net user,hostfwd=tcp::10022-:22 \
    -kernel /path/to/linux_src/arch/x86/boot/bzImage \
    -append root=/dev/sda1 \
    -hda /path/to/debian.img

It is possible to do a lot of calculations from command shell itself without using any spreadsheet applications or scripting languages like perl or python. What we require is simple un*x applications like, bc, xargs, awk etc.

To sum a list of numbers

  • There are many different ways to do that, but the best way I like is using xargs, tr and bc
  • $ command | xargs | tr ' ' '+' | bc -l
    where command could be either ‘cat filename’ or the output of any other command. The beauty of this command is that, it works regardless of the number of columns or rows in the file. Even, blank lines would not create any problem. The following file has got extra white spaces and blank lines and xargs arranges them in a single line with one space between them beautifully.

    $cat somefile.txt
         1
     2 3
    
       4  5 6
    
    


    bc is arbitrary precision, so that, it could handle any big number.

  • If we want to sum a column of numbers, it is possible to use awk also, for example to find the sum of the size of all files in a directory, do the following
  • $ ls -l | awk '{s+=$5} END {print s}'

    This command is also not susceptible to blank lines.

  • If there are no blank lines, even paste along with bc could be used
  • $ command | paste -sd+ | bc -l

    To multiply a list of numbers

  • For example, to find factorial of a number, say 10
  • $ for ((k=1;k<=10;k++)) do echo $k; done | xargs | tr ' ' '*' | bc -l

    or

    $ for ((k=1;k<=10;k++)) do echo $k; done | paste -sd* | bc -l

    To sum a list of hexadecimal numbers and to display their sum in hexadecimal

    The following command line runs a ‘for’ loop to generate numbers from 0x00 to 0xff (255), compute the sum and prints the output in hexadecimal itself.

    $ for ((k=0;k<256;k++)) do printf "0x%x\n" $k; done \
    | sed -e 's/^0x//g' | tr '[a-f]' '[A-F]' \
    | xargs echo 'obase=16;ibase=16;0' | tr ' ' '+' | bc -l

    Actually bc does not accept the hexadecimal prefix ‘0x’, that is the reason of using sed to remove the 0x prefix.  Also bc expects uppercase of 0xa to 0xf, so ‘tr’ is used translate them.  The ‘0’ inside the echo statement is used so that there is a number before the first ‘+’, if we want to remove the hack of using a ‘0’, the above command could be modified to the following.

    $ for ((k=0;k<256;k++)) do printf "0x%x\n" $k; done \
    | sed -e 's/^0x//g' | tr '[a-f]' '[A-F]' \
    | xargs | tr ' ' '+' | xargs echo 'obase=16;ibase=16;' | bc -l

    Or with a sub shell and a single xargs

    $ for ((k=0;k<256;k++)) do printf "0x%x\n" $k; done \
    | sed -e 's/^0x//g' | tr '[a-f]' '[A-F]' \
    | (echo -ne 'obase=16;ibase=16;' && xargs) | tr ' ' '+' | bc -l

    The above commands could be used for binary or octal also by replacing the ibase and obase to 2 or 8 respectively. Note that, it is better to write obase first. If ibase is written first, then obase has to be specified in terms of ibase
    That means
    obase=16;ibase=16
    and ibase=16;obase=10 are the same

    Welcome to WordPress.com. After you read this, you should delete and write your own post, with a new title above. Or hit Add New on the left (of the admin dashboard) to start a fresh post.

    Here are some suggestions for your first post.

    1. You can find new ideas for what to blog about by reading the Daily Post.
    2. Add PressThis to your browser. It creates a new blog post for you about any interesting  page you read on the web.
    3. Make some changes to this page, and then hit preview on the right. You can alway preview any post or edit you before you share it to the world.
    %d bloggers like this: