In a previous article we looked at LVM. This short article describes how to grow an existing volume by adding an extra disk to the system. Specifically, we use a XenServer guest VM with CentOS installed.

The first disk of a XenServer guest VM is called xvda, the second disk (the one we added) is called xvdb. We start with the creation of a Physical Volume (PV) on the extra disk.

pvcreate /dev/xvdb

Then we extend the existing Volume Group (VG) called VolGroup00.

vgextend VolGroup00 /dev/xvdb

Then we extend the Logical Volume (LV) by the size of the extra disk, in this case 8GB.

lvextend -L8G /dev/VolGroup00/LogVol00

Finally we resize the filesystem that uses this LV.

resize2fs /dev/VolGroup00/LogVol00

Bundle, upload and register the kernel with Eucalyptus:

ec2-bundle-image --image /boot/vmlinuz-2.6.18-128.1.6.el5xen --kernel true
ec2-upload-bundle --bucket centos-kernel-bucket --manifest /tmp/vmlinuz-2.6.18-128.1.6.el5xen.manifest.xml
ec2-register centos-kernel-bucket/vmlinuz-2.6.18-128.1.6.el5xen.manifest.xml

Bundle, upload and register the ramdisk with Eucalyptus:

ec2-bundle-image --image /root/centos-ramdisk.img --ramdisk true
ec2-upload-bundle --bucket centos-ramdisk-bucket --manifest /tmp/centos-ramdisk.img.manifest.xml
ec2-register centos-ramdisk-bucket/centos-ramdisk.img.manifest.xml

Bundle, upload and register the filesystem with Eucalyptus:

ec2-bundle-image --image /root/centos-root.img
ec2-upload-bundle --bucket centos-root-bucket --manifest /tmp/centos-root.img.manifest.xml
ec2-register centos-root-bucket/centos-root.img.manifest.xml

Start an instance, replacing the emi-, eki- and eri-identifiers with your own:

ec2-run-instances emi-F4CB118E --kernel eki-38AC43DC --ramdisk eri-98FE2101

1. The kernel (vmlinuz)
2. The ramdisk (initrd)
3. The filesystem

Kernel

The easiest way to create a suitable kernel, ramdisk and filesystem is to use a system with CentOS 5.2 on it with xen enabled:

yum install xen

Change the default kernel to the new kernel with xen support by editing /boot/grub/menu.lst:

default=0

and reboot

Ramdisk

Create the ramdisk by running mkinitrd:

mkinitrd --omit-scsi-modules --with=xennet --with=xenblk --preload=xenblk /root/xen-image/centos-ramdisk.img

Filesystem

The hardest job is getting a filesystem. Create a directory where we will hold the filesystem image:

mkdir /root/xen-image
cd /root/xen-image

Create an image file and make a filesystem on this file:

dd if=/dev/zero of=centos-root.img bs=1M count=1999
mkfs.ext3 centos-root.img

Create a directory where we can mount the newly created image:

mkdir rootdisk
mount -o loop centos-root.img /root/xen-image/rootdisk/
cd rootdisk

Create the /etc directory:

mkdir etc

and edit the file /etc/fstab:

/dev/sda1   /          ext3     defaults         1 1
none        /dev/pts   devpts   gid=5,mode=620   0 0
none        /dev/shm   tmpfs    defaults         0 0
none        /proc      proc     defaults         0 0
none        /sys       sysfs    defaults         0 0

Create some necessary device files:

mkdir dev
for i in console null zero; do /sbin/MAKEDEV -d /root/xen-image/rootdisk/dev -x $i; done

Copy the kernel modules:

mkdir -p lib/modules
cp -a /lib/modules/2.6.18-128.1.6.el5xen/ /root/xen-image/rootdisk/lib/modules/

Create the directory for network scripts:

mkdir -p etc/sysconfig/network-scripts

Edit the file that describes the first network interface, /etc/sysconfig/network-scripts/ifcfg-eth0:

DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes

Do the same for the second network interface, /etc/sysconfig/network-scripts/ifcfg-eth1:

DEVICE=eth1
BOOTPROTO=dhcp
ONBOOT=yes

And the last network script, etc/sysconfig/network:

NETWORKING=yes
HOSTNAME=centos52
GATEWAY=x.x.x.x

Create the RPM lock directory:

mkdir -p var/lock/rpm

We need to create a specific configuration file for yum, /root/xen-image/yum-xen.conf:

[main]
cachedir=/var/cache/yum
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
distroverpkg=redhat-release
tolerant=1
exactarch=1
obsoletes=1
gpgcheck=0
plugins=1
metadata_expire=1h

[base]
name=CentOS-5.2 - Base
baseurl=http://mirror.centos.org/centos/5.2/os/i386/
enabled=1

[extras]
name=CentOS-5.2 - Extras
baseurl=http://mirror.centos.org/centos/5.2/extras/i386/
enabled=1

[updates]
name=CentOS-5.2 - Updates
baseurl=http://mirror.centos.org/centos/5.2/updates/i386/
enabled=1

[addons]
name=CentOS-5.2 - Addons
baseurl=http://mirror.centos.org/centos/5.2/addons/i386/
enabled=1

Use the configuration file we just edited to install the base system and add the openssh server:

yum -c /root/xen-image/yum-xen.conf --installroot=/root/xen-image/rootdisk -y groupinstall base
yum -c /root/xen-image/yum-xen.conf --installroot=/root/xen-image/rootdisk -y install openssh openssh-server

To set an initial password for root, we chroot into the rootdisk we created:

chroot /root/xen-image/rootdisk

Edit the /etc/passwd file inside the chroot and change the ‘*’ on the first line with an ‘x’:

root:x:0:0:root:/root:/bin/bash

Run pwconv to enable shadow passwords and set the root password:

pwconv
passwd root

Disable TLS:

mv /lib/tls /lib/tls.disabled

Exit from the chroot:

exit

Unmount the rootdisk:

cd /root
umount /root/xen-image/rootdisk

Putting it all together

Create a new configuration file for Xen specific to this image, /etc/xen/centos52:

kernel = "/boot/vmlinuz-2.6.18-128.1.6.el5xen"
ramdisk = "/root/xen-image/centos-ramdisk.img"
name = "centos52"
memory = "256"
disk = [ 'file:/root/xen-image/centos-root.img,sda1,w' ]
root = '/dev/sda1 ro'
vif = [ 'bridge=xenbr0', '']
vcpus=1
on_reboot = 'destroy'
on_crash = 'destroy'

And finally, starting the Xen VM image:

/usr/sbin/xm create -c centos52

In this article we will install the Ganglia monitoring system on a set of machines running CentOS. There are two kinds of machines involved:

• The meta node: one machine that receives all measurements and presents it to a client through a website.
• The monitoring nodes: machines that run only the monitoring daemon and send the measurements to the meta node.

Meta node

For this example we assume the meta node has the IP address 192.168.1.253. We start by installing the necessary software:

rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm
yum install rrdtool ganglia ganglia-gmetad ganglia-gmond ganglia-web httpd php apr apr-util

If you want to monitor the meta node as well as the monitoring nodes, edit the gmond configuration file /etc/gmond.conf:

cluster {
  name = "cluster1"
  owner = "owner1"
  latlong = "unspecified"
  url = "unspecified"
}

udp_send_channel {
  host = 192.168.1.253
  port = 8649
  ttl = 1
}

udp_recv_channel {
  port = 8649
}

Start the gmond service and make sure it starts at boot:

chkconfig gmond on
service gmond start

Edit the gmetad configuration file /etc/gmetad.conf:

data_source "my cluster" 192.168.1.253:8649

Start the gmetad service and make sure it starts at boot:

chkconfig gmetad on
service gmetad start

Enable the http daemon, to be able to see the pretty monitoring pictures:

chkconfig httpd on
service httpd start

Monitoring nodes

On all the monitoring nodes start by installing the necessary software:

rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm
yum install ganglia-gmond

Edit the gmond configuration file /etc/gmond.conf. You can use an exact replica of the gmond configuration file shown for the meta node.
Start the gmond service and make sure it starts at boot:

chkconfig gmond on
service gmond start

If you would like to emit your own measurements (called metrics in Ganglia) and view them on the website, call the gmetric program:

gmetric --name mymetricname --value mymetricvalue --type string

To use the output of a program you wrote as a metric, simply call it like this, making sure to use backticks (`) instead of quotes (‘):

gmetric --name mymetricname --value `/home/user/mymetricprogram` --type string

Eucalyptus is software that enables you to run your own cloud. It aims to be API compatible with Amazon’s EC2, which means you can use most of the tools that are written for that system with Eucalyptus too.

This articles describes my successful attempt at installing Eucalyptus 1.4 on CentOS 5.2 with managed network mode. The installation is performed on two different types of machines:

• The front-end, which is the machine controlling the cloud
• The compute nodes, which run the virtual machines

Front-end

The front-end has two network interfaces. One (eth0) is connected to the compute nodes and has IP address 192.168.1.254. The other (eth1) is connected to the LAN and has IP address 172.16.0.254.

We start by disabling SELinux. This is accomplished by editing the file /etc/selinux/config:

SELINUX=disabled

Now reboot for this change to take effect.

Allow the machine to forward IP packets by editing /etc/sysctl.conf:

net.ipv4.ip_forward = 1

and change the value immediately without rebooting:

sysctl -p /etc/sysctl.conf

Start with a clean iptables firewall and allow NAT:

iptables --flush
iptables --table nat --flush
iptables --delete-chain
iptables --table nat --delete-chain
iptables --table nat --append POSTROUTING --out-interface eth1 -j MASQUERADE
iptables --append FORWARD --in-interface eth0 -j ACCEPT

Make the changes permanent by running:

/etc/init.d/iptables save

Download the Sun Java Development Kit version 6 and install it:

chmod +x jdk-6u13-linux-i586-rpm.bin
./jdk-6u13-linux-i586-rpm.bin

Download Apache ANT and install it:

cd /opt
tar -zxvf apache-ant-1.7.1-bin.tar.gz

Download the EC2 tools and install them:

cd /opt
unzip /root/ec2-ami-tools-1.3-26357.zip
unzip /root/ec2-api-tools-1.3-30349.zip

Some environment variables need to be present for Eucalyptus to work. Edit the file /etc/profile and add the following:

export JAVA_HOME=/usr/java/jdk1.6.0_13
export EC2_HOME=/opt/ec2-api-tools-1.3-30349
export EC2_AMITOOL_HOME=/opt/ec2-ami-tools-1.3-26357
export PATH=$PATH:/opt/apache-ant-1.7.1/bin:$EC2_HOME/bin:$EC2_AMITOOL_HOME/bin

Install some dependencies:

yum install dhcp xen-libs bridge-utils

Download the Eucalyptus files and extract the RPM dependencies file:

tar -zxvf eucalyptus-rpm-deps-i386.tar.gz

Install the Eucalyptus RPMs:

rpm -Uvh euca-axis2c-1.4-1.i386.rpm
         euca-httpd-1.4-1.i386.rpm
         euca-libvirt-1.4-1.i386.rpm
         eucalyptus-1.4-2.i386.rpm
         eucalyptus-cloud-1.4-2.i386.rpm
         eucalyptus-gl-1.4-2.i386.rpm
         eucalyptus-cc-1.4-2.i386.rpm

The configuration file /opt/eucalyptus/etc/eucalyptus/eucalyptus.conf contains the following for our setup:

EUCALYPTUS="/opt/eucalyptus"
START_CLOUD="Y"
START_CC="Y"
START_NC="N"
ENABLE_WS_SECURITY="Y"
LOGLEVEL="DEBUG"
CLOUD_PORT="8773"
CLOUD_SSL_PORT="8443"
CC_PORT="8774"
SCHEDPOLICY="GREEDY"
NODES="192.168.1.1 192.168.1.2 192.168.1.3"
NC_SERVICE="axis2/services/EucalyptusNC"
NC_PORT="8775"
VNET_INTERFACE="eth0"
VNET_DHCPDAEMON="/usr/sbin/dhcpd"
VNET_MODE="MANAGED"
VNET_SUBNET="10.0.0.0"
VNET_NETMASK="255.0.0.0"
VNET_DNS="172.16.0.1"
VNET_ADDRSPERNET="64"
VNET_PUBLICIPS="172.16.0.11 172.16.0.12 172.16.0.13 172.16.0.14"

Compute nodes

The compute nodes have one network interface (eth0) which is connected to the front-end and they have IP addresses ranging from 192.168.1.1 to 192.168.1.3 (for three compute nodes).

We start by disabling SELinux. This is accomplished by editing the file /etc/selinux/config:

SELINUX=disabled

Now reboot for this change to take effect.

Start with a clean iptables firewall:

iptables --flush
iptables --table nat --flush
iptables --delete-chain
iptables --table nat --delete-chain

Make the changes permanent by running:

/etc/init.d/iptables save

Install Xen:

yum install xen

Make sure the correct kernel with xen enabled is started at boot by editing the file /boot/grub/menu.lst:

default=0

And reboot.

Download the Eucalyptus files and extract the RPM dependencies file:

tar -zxvf eucalyptus-rpm-deps-i386.tar.gz

Install the Eucalyptus RPMs:

rpm -Uvh euca-axis2c-1.4-1.i386.rpm
euca-libvirt-1.4-1.i386.rpm
euca-httpd-1.4-1.i386.rpm
eucalyptus-1.4-2.i386.rpm
eucalyptus-gl-1.4-2.i386.rpm
eucalyptus-nc-1.4-2.i386.rpm

Make a directory for holding the running instances:

mkdir -p /usr/local/instances

The configuration file /opt/eucalyptus/etc/eucalyptus/eucalyptus.conf contains the following for our setup:

EUCALYPTUS="/opt/eucalyptus"
START_CLOUD="N"
START_CC="N"
START_NC="Y"
ENABLE_WS_SECURITY="Y"
LOGLEVEL="DEBUG"
NC_PORT="8775"
INSTANCE_PATH="/usr/local/instances"
VNET_INTERFACE="peth0"
VNET_BRIDGE="xenbr0"
VNET_MODE="MANAGED"

Start Eucalyptus by running:

/etc/init.d/eucalyptus start

Back to the front-end

Start Eucalyptus by running:

/etc/init.d/eucalyptus start

If all went according to plan, the following website should be viewable: https://172.16.0.254:8443. Login with username admin and password admin. Add a cluster with a name you like, e.g. my_cluster, and the IP address of the front-end, in our case 172.16.0.254.

Now download the x509 certificate to be able to connect to the Eucalyptus setup. Unzip the contents of this file:

mkdir /root/.euca
cd /root/.euca
unzip /root/euca2-admin-x509.zip

There is a file containing aliases for several commands you run that needs to be sourced every time you login. It is easier to do this automatically, so edit the file ~/.bash_profile and add this line to the end:

source /root/.euca/eucarc

Synchronise the keys between all the hosts:

/opt/eucalyptus/usr/sbin/euca_sync_key -c /opt/eucalyptus/etc/eucalyptus/eucalyptus.conf

Install ruby to be able to run the EC2 command line tools:

yum install ruby

You can now see if the EC2 command line tools and Eucalyptus are working:

ec2-describe-availability-zones verbose

This should give you an overview of the running system, with a listing of the compute nodes and the capacity of them in terms of number of running instances they can hold.

Extract the example VM image from Eucalyptus:

tar -zxvf euca-ttylinux.tgz

Bundle the image, upload it and register it:

ec2-bundle-image --image /root/ttylinux/vmlinuz-2.6.16.33-xen --kernel true
ec2-upload-bundle --bucket kernel-bucket --manifest /tmp/vmlinuz-2.6.16.33-xen.manifest.xml
ec2-register kernel-bucket/vmlinuz-2.6.16.33-xen.manifest.xml

ec2-bundle-image --image /root/ttylinux/ttylinux.img
ec2-upload-bundle --bucket image-bucket --manifest /tmp/ttylinux.img.manifest.xml
ec2-register image-bucket/ttylinux.img.manifest.xml

If all went well, you can list the images just uploaded:

ec2-describe-images

Add a keypair for the current user to Eucalyptus:

ec2-add-keypair admin_key > /root/admin_key.private
chmod 0600 /root/admin_key.private

Run a VM (change emi-F4CB118E with the emi-identifier returned by ec2-describe-images):

ec2-run-instances emi-F4CB118E -k admin_key

Get a list of instances:

ec2-describe-instances

To see the console output of the running instance (change i-3F170798 with the instance-identifier returned by ec2-describe-instances):

ec2-get-console-output i-3F170798

Everybody wants their stuff to be secure, until they actually try to use it. That’s the reason for this tidbit about disabling the default security on CentOS.

SELinux is enabled by default on CentOS. To disable it, we need to edit /etc/selinux/config:

SELINUX=disabled

We need to reboot the machine for this to take effect.

The firewall iptables has some default rules we want to get rid of:

iptables --flush
iptables --table nat --flush
iptables --delete-chain
iptables --table nat --delete-chain

To make these changes permanent, we can save the current rules:

/etc/init.d/iptables save

Static IP address

Assigning a static IP address on CentOS is accomplished by editing the files in the directory /etc/sysconfig/network-scripts. For example, assign a static IP address to interface eth0 by editing /etc/sysconfig/network-scripts/ifcfg-eth0:

DEVICE=eth0
BOOTPROTO=none
IPADDR=192.168.0.123
NETMASK=255.255.255.0
GATEWAY=192.168.0.1
ONBOOT=yes

Network interface bonding

To achieve higher bandwidth and/or reliability, network interfaces can be bonded. First, edit the file /etc/sysconfig/network-scripts/ifcfg-bond0:

DEVICE=bond0
BOOTPROTO=none
IPADDR=10.0.0.1
NETMASK=255.255.255.0
ONBOOT=yes

After that, you need to edit the network interfaces that are part of this bond, e.g. eth1 and eth2. Let’s start with /etc/sysconfig/network-scripts/ifcfg-eth1:

DEVICE=eth1
BOOTPROTO=none
MASTER=bond0
SLAVE=yes
ONBOOT=yes

Then edit /etc/sysconfig/network-scripts/ifcfg-eth2:

DEVICE=eth2
BOOTPROTO=none
MASTER=bond0
SLAVE=yes
ONBOOT=yes

The last file to edit is /etc/modprobe.conf:

alias bond0 bonding
options bond0 mode=active-backup miimon=100

The mode value in this last file can be one of several:

• balance-rr
• active-backup
• balance-xor
• 802.3ad
• balance-tlb
• balance-alb

In the active-backup mode shown in the example, only one slave in the bond is active. A different slave becomes active if, and only if, the active slave fails. The bond’s MAC address is externally visible on only one port (network interface) to avoid confusing the switch.

The option miimon (media-independent interface monitoring) defines how often, in milliseconds, link monitoring occurs.

Here are the steps to take to enable the Network Time Protocol (NTP) daemon on CentOS. Change the timeserver value to one that is close to you.

yum install system-config-date
ntpdate nl.pool.ntp.org
service ntpd start
chkconfig ntpd on