Using a Xen virtual machine image with Eucalyptus

By jansipke on 2009-04-28

eucalyptus
In the previous article we created a CentOS virtual machine image that was usable with Xen. In this short article we will use this image in Eucalyptus.

Bundle, upload and register the kernel with Eucalyptus:

ec2-bundle-image --image /boot/vmlinuz-2.6.18-128.1.6.el5xen --kernel true
ec2-upload-bundle --bucket centos-kernel-bucket --manifest /tmp/vmlinuz-2.6.18-128.1.6.el5xen.manifest.xml
ec2-register centos-kernel-bucket/vmlinuz-2.6.18-128.1.6.el5xen.manifest.xml

Bundle, upload and register the ramdisk with Eucalyptus:

ec2-bundle-image --image /root/centos-ramdisk.img --ramdisk true
ec2-upload-bundle --bucket centos-ramdisk-bucket --manifest /tmp/centos-ramdisk.img.manifest.xml
ec2-register centos-ramdisk-bucket/centos-ramdisk.img.manifest.xml

Bundle, upload and register the filesystem with Eucalyptus:

ec2-bundle-image --image /root/centos-root.img
ec2-upload-bundle --bucket centos-root-bucket --manifest /tmp/centos-root.img.manifest.xml
ec2-register centos-root-bucket/centos-root.img.manifest.xml

Start an instance, replacing the emi-, eki- and eri-identifiers with your own:

ec2-run-instances emi-F4CB118E --kernel eki-38AC43DC --ramdisk eri-98FE2101

Posted in Tagged , , 1 Comment

Creating a CentOS Xen virtual machine image

By jansipke on 2009-04-28

xen
In this article we will create a CentOS 5.2 virtual machine image that can be used by Xen. We need to bring three pieces together:

  1. The kernel (vmlinuz)
  2. The ramdisk (initrd)
  3. The filesystem

Kernel

The easiest way to create a suitable kernel, ramdisk and filesystem is to use a system with CentOS 5.2 on it with xen enabled:

yum install xen

Change the default kernel to the new kernel with xen support by editing /boot/grub/menu.lst:

default=0

and reboot

Ramdisk

Create the ramdisk by running mkinitrd:

mkinitrd --omit-scsi-modules --with=xennet --with=xenblk --preload=xenblk /root/xen-image/centos-ramdisk.img

Filesystem

The hardest job is getting a filesystem. Create a directory where we will hold the filesystem image:

mkdir /root/xen-image
cd /root/xen-image

Create an image file and make a filesystem on this file:

dd if=/dev/zero of=centos-root.img bs=1M count=1999
mkfs.ext3 centos-root.img

Create a directory where we can mount the newly created image:

mkdir rootdisk
mount -o loop centos-root.img /root/xen-image/rootdisk/
cd rootdisk

Create the /etc directory:

mkdir etc

and edit the file /etc/fstab:

/dev/sda1   /          ext3     defaults         1 1
none        /dev/pts   devpts   gid=5,mode=620   0 0
none        /dev/shm   tmpfs    defaults         0 0
none        /proc      proc     defaults         0 0
none        /sys       sysfs    defaults         0 0

Create some necessary device files:

mkdir dev
for i in console null zero; do /sbin/MAKEDEV -d /root/xen-image/rootdisk/dev -x $i; done

Copy the kernel modules:

mkdir -p lib/modules
cp -a /lib/modules/2.6.18-128.1.6.el5xen/ /root/xen-image/rootdisk/lib/modules/

Create the directory for network scripts:

mkdir -p etc/sysconfig/network-scripts

Edit the file that describes the first network interface, /etc/sysconfig/network-scripts/ifcfg-eth0:

DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes

Do the same for the second network interface, /etc/sysconfig/network-scripts/ifcfg-eth1:

DEVICE=eth1
BOOTPROTO=dhcp
ONBOOT=yes

And the last network script, etc/sysconfig/network:

NETWORKING=yes
HOSTNAME=centos52
GATEWAY=x.x.x.x

Create the RPM lock directory:

mkdir -p var/lock/rpm

We need to create a specific configuration file for yum, /root/xen-image/yum-xen.conf:

[main]
cachedir=/var/cache/yum
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
distroverpkg=redhat-release
tolerant=1
exactarch=1
obsoletes=1
gpgcheck=0
plugins=1
metadata_expire=1h

[base]
name=CentOS-5.2 - Base
baseurl=http://mirror.centos.org/centos/5.2/os/i386/
enabled=1

[extras]
name=CentOS-5.2 - Extras
baseurl=http://mirror.centos.org/centos/5.2/extras/i386/
enabled=1

[updates]
name=CentOS-5.2 - Updates
baseurl=http://mirror.centos.org/centos/5.2/updates/i386/
enabled=1

[addons]
name=CentOS-5.2 - Addons
baseurl=http://mirror.centos.org/centos/5.2/addons/i386/
enabled=1

Use the configuration file we just edited to install the base system and add the openssh server:

yum -c /root/xen-image/yum-xen.conf --installroot=/root/xen-image/rootdisk -y groupinstall base
yum -c /root/xen-image/yum-xen.conf --installroot=/root/xen-image/rootdisk -y install openssh openssh-server

To set an initial password for root, we chroot into the rootdisk we created:

chroot /root/xen-image/rootdisk

Edit the /etc/passwd file inside the chroot and change the ‘*’ on the first line with an ‘x’:

root:x:0:0:root:/root:/bin/bash

Run pwconv to enable shadow passwords and set the root password:

pwconv
passwd root

Disable TLS:

mv /lib/tls /lib/tls.disabled

Exit from the chroot:

exit

Unmount the rootdisk:

cd /root
umount /root/xen-image/rootdisk

Putting it all together

Create a new configuration file for Xen specific to this image, /etc/xen/centos52:

kernel = "/boot/vmlinuz-2.6.18-128.1.6.el5xen"
ramdisk = "/root/xen-image/centos-ramdisk.img"
name = "centos52"
memory = "256"
disk = [ 'file:/root/xen-image/centos-root.img,sda1,w' ]
root = '/dev/sda1 ro'
vif = [ 'bridge=xenbr0', '']
vcpus=1
on_reboot = 'destroy'
on_crash = 'destroy'

And finally, starting the Xen VM image:

/usr/sbin/xm create -c centos52

Posted in Tagged , 5 Comments

Fetching meta-data and user-data in Eucalyptus

By jansipke on 2009-04-13

eucalyptus
Eucalyptus is a cloud computing platform that is intended to be API compatible with Amazon’s EC2, but can be installed on your own machines. In the latest stable version available at the time of writing, version 1.4, there is a catch in fetching meta-data and user-data. According to Amazon’s API description, you can get the meta-data by doing an HTTP GET on the URL:

http://169.254.169.254/latest/meta-data

However, this doesn’t work in Eucalyptus 1.4. The correct host to fetch this data from is the gateway for the running instance and the port is 8773. This little script will print the correct host and port:

echo http://$(route -n | awk '$4 ~ ".*G.*" {print $2}'):8773/

Meta-data

You can fetch a list of meta-data keys by getting /latest/meta-data. Fetching the value for a key is as easy as getting /latest/meta-data/key. The following table shows an example of the keys and associated values:

local-ipv4 10.0.2.3
reservation-id r-412F078F
local-hostname 10.0.2.3
security-groups [default]
product-codes not yet supported.
ami-launch-index 0
public-hostname 192.168.1.3
hostname 192.168.1.3
public-ipv4 192.168.1.3
ramdisk-id emi-F4CB118E
kernel-id eki-90591388
ami-id emi-7C83135D
placement/ availability-zone
public-keys/ 0=admin_key
ancestor-ami-ids none
block-device-mapping not yet supported.
ami-manifest-path http://cloud1.example.org:8773/services/Walrus/ubuntu804-bucket/ubuntu.8-04.img.manifest.xml
instance-type m1.small
instance-id i-2ECA052C

User-data

You can supply user-data as an option to ec2-run-instances:

ec2-run-instances <emi> -k <key> -d <user-data>

This data is fetched by getting /latest/user-data.

Posted in Tagged Leave a Comment

Installing Eucalyptus on CentOS

By jansipke on 2009-04-06

eucalyptus
Setup

Eucalyptus is software that enables you to run your own cloud. It aims to be API compatible with Amazon’s EC2, which means you can use most of the tools that are written for that system with Eucalyptus too.

This articles describes my successful attempt at installing Eucalyptus 1.4 on CentOS 5.2 with managed network mode. The installation is performed on two different types of machines:

  • The front-end, which is the machine controlling the cloud
  • The compute nodes, which run the virtual machines

Front-end

eucalyptus-head-node

The front-end has two network interfaces. One (eth0) is connected to the compute nodes and has IP address 192.168.1.254. The other (eth1) is connected to the LAN and has IP address 172.16.0.254.

We start by disabling SELinux. This is accomplished by editing the file /etc/selinux/config:

SELINUX=disabled

Now reboot for this change to take effect.

Allow the machine to forward IP packets by editing /etc/sysctl.conf:

net.ipv4.ip_forward = 1

and change the value immediately without rebooting:

sysctl -p /etc/sysctl.conf

Start with a clean iptables firewall and allow NAT:

iptables --flush
iptables --table nat --flush
iptables --delete-chain
iptables --table nat --delete-chain
iptables --table nat --append POSTROUTING --out-interface eth1 -j MASQUERADE
iptables --append FORWARD --in-interface eth0 -j ACCEPT

Make the changes permanent by running:

/etc/init.d/iptables save

Download the Sun Java Development Kit version 6 and install it:

chmod +x jdk-6u13-linux-i586-rpm.bin
./jdk-6u13-linux-i586-rpm.bin

Download Apache ANT and install it:

cd /opt
tar -zxvf apache-ant-1.7.1-bin.tar.gz

Download the EC2 tools and install them:

cd /opt
unzip /root/ec2-ami-tools-1.3-26357.zip
unzip /root/ec2-api-tools-1.3-30349.zip

Some environment variables need to be present for Eucalyptus to work. Edit the file /etc/profile and add the following:

export JAVA_HOME=/usr/java/jdk1.6.0_13
export EC2_HOME=/opt/ec2-api-tools-1.3-30349
export EC2_AMITOOL_HOME=/opt/ec2-ami-tools-1.3-26357
export PATH=$PATH:/opt/apache-ant-1.7.1/bin:$EC2_HOME/bin:$EC2_AMITOOL_HOME/bin

Install some dependencies:

yum install dhcp xen-libs bridge-utils

Download the Eucalyptus files and extract the RPM dependencies file:

tar -zxvf eucalyptus-rpm-deps-i386.tar.gz

Install the Eucalyptus RPMs:

rpm -Uvh euca-axis2c-1.4-1.i386.rpm
         euca-httpd-1.4-1.i386.rpm
         euca-libvirt-1.4-1.i386.rpm
         eucalyptus-1.4-2.i386.rpm
         eucalyptus-cloud-1.4-2.i386.rpm
         eucalyptus-gl-1.4-2.i386.rpm
         eucalyptus-cc-1.4-2.i386.rpm

The configuration file /opt/eucalyptus/etc/eucalyptus/eucalyptus.conf contains the following for our setup:

EUCALYPTUS="/opt/eucalyptus"
START_CLOUD="Y"
START_CC="Y"
START_NC="N"
ENABLE_WS_SECURITY="Y"
LOGLEVEL="DEBUG"
CLOUD_PORT="8773"
CLOUD_SSL_PORT="8443"
CC_PORT="8774"
SCHEDPOLICY="GREEDY"
NODES="192.168.1.1 192.168.1.2 192.168.1.3"
NC_SERVICE="axis2/services/EucalyptusNC"
NC_PORT="8775"
VNET_INTERFACE="eth0"
VNET_DHCPDAEMON="/usr/sbin/dhcpd"
VNET_MODE="MANAGED"
VNET_SUBNET="10.0.0.0"
VNET_NETMASK="255.0.0.0"
VNET_DNS="172.16.0.1"
VNET_ADDRSPERNET="64"
VNET_PUBLICIPS="172.16.0.11 172.16.0.12 172.16.0.13 172.16.0.14"

Compute nodes

eucalyptus-compute-node

The compute nodes have one network interface (eth0) which is connected to the front-end and they have IP addresses ranging from 192.168.1.1 to 192.168.1.3 (for three compute nodes).

We start by disabling SELinux. This is accomplished by editing the file /etc/selinux/config:

SELINUX=disabled

Now reboot for this change to take effect.

Start with a clean iptables firewall:

iptables --flush
iptables --table nat --flush
iptables --delete-chain
iptables --table nat --delete-chain

Make the changes permanent by running:

/etc/init.d/iptables save

Install Xen:

yum install xen

Make sure the correct kernel with xen enabled is started at boot by editing the file /boot/grub/menu.lst:

default=0

And reboot.

Download the Eucalyptus files and extract the RPM dependencies file:

tar -zxvf eucalyptus-rpm-deps-i386.tar.gz

Install the Eucalyptus RPMs:

rpm -Uvh euca-axis2c-1.4-1.i386.rpm
euca-libvirt-1.4-1.i386.rpm
euca-httpd-1.4-1.i386.rpm
eucalyptus-1.4-2.i386.rpm
eucalyptus-gl-1.4-2.i386.rpm
eucalyptus-nc-1.4-2.i386.rpm

Make a directory for holding the running instances:

mkdir -p /usr/local/instances

The configuration file /opt/eucalyptus/etc/eucalyptus/eucalyptus.conf contains the following for our setup:

EUCALYPTUS="/opt/eucalyptus"
START_CLOUD="N"
START_CC="N"
START_NC="Y"
ENABLE_WS_SECURITY="Y"
LOGLEVEL="DEBUG"
NC_PORT="8775"
INSTANCE_PATH="/usr/local/instances"
VNET_INTERFACE="peth0"
VNET_BRIDGE="xenbr0"
VNET_MODE="MANAGED"

Start Eucalyptus by running:

/etc/init.d/eucalyptus start

Back to the front-end

Start Eucalyptus by running:

/etc/init.d/eucalyptus start

If all went according to plan, the following website should be viewable: https://172.16.0.254:8443. Login with username admin and password admin. Add a cluster with a name you like, e.g. my_cluster, and the IP address of the front-end, in our case 172.16.0.254.

Now download the x509 certificate to be able to connect to the Eucalyptus setup. Unzip the contents of this file:

mkdir /root/.euca
cd /root/.euca
unzip /root/euca2-admin-x509.zip

There is a file containing aliases for several commands you run that needs to be sourced every time you login. It is easier to do this automatically, so edit the file ~/.bash_profile and add this line to the end:

source /root/.euca/eucarc

Synchronise the keys between all the hosts:

/opt/eucalyptus/usr/sbin/euca_sync_key -c /opt/eucalyptus/etc/eucalyptus/eucalyptus.conf

Install ruby to be able to run the EC2 command line tools:

yum install ruby

You can now see if the EC2 command line tools and Eucalyptus are working:

ec2-describe-availability-zones verbose

This should give you an overview of the running system, with a listing of the compute nodes and the capacity of them in terms of number of running instances they can hold.

Extract the example VM image from Eucalyptus:

tar -zxvf euca-ttylinux.tgz

Bundle the image, upload it and register it:

ec2-bundle-image --image /root/ttylinux/vmlinuz-2.6.16.33-xen --kernel true
ec2-upload-bundle --bucket kernel-bucket --manifest /tmp/vmlinuz-2.6.16.33-xen.manifest.xml
ec2-register kernel-bucket/vmlinuz-2.6.16.33-xen.manifest.xml

ec2-bundle-image --image /root/ttylinux/ttylinux.img
ec2-upload-bundle --bucket image-bucket --manifest /tmp/ttylinux.img.manifest.xml
ec2-register image-bucket/ttylinux.img.manifest.xml

If all went well, you can list the images just uploaded:

ec2-describe-images

Add a keypair for the current user to Eucalyptus:

ec2-add-keypair admin_key > /root/admin_key.private
chmod 0600 /root/admin_key.private

Run a VM (change emi-F4CB118E with the emi-identifier returned by ec2-describe-images):

ec2-run-instances emi-F4CB118E -k admin_key

Get a list of instances:

ec2-describe-instances

To see the console output of the running instance (change i-3F170798 with the instance-identifier returned by ec2-describe-instances):

ec2-get-console-output i-3F170798

Posted in Tagged , 2 Comments