The trick we use here is to create a view that points to the current table we want to use. To be able to atomically point the view to another table, we use a stored procedure.

We start with the SQL code to create the stored procedure createMyTable. This stored procedure has one argument: the name of the table to create, e.g. mytable_2010_11_22.

DROP PROCEDURE IF EXISTS createMyTable;

DELIMITER //
CREATE PROCEDURE createMyTable(IN table_name text)
BEGIN
   SET @sql = CONCAT("CREATE TABLE ", table_name, "(", "id int,", "name text");");

   PREPARE stmt FROM @sql;
   EXECUTE stmt;

   DEALLOCATE PREPARE stmt;
END //
DELIMITER ;

At this point we can create a new table and fill it:

CALL createMyTable("mytable_2010_11_22");
INSERT INTO mytable_2010_11_22 (id, name) VALUES (1, "Alice");
INSERT ...

The second part we need is the stored procedure that changes the view to point to the newly created table. This stored procedure also has one argument: the name of the table to point to, e.g. mytable_2010_11_22.

DROP PROCEDURE IF EXISTS createMyView;

DELIMITER //
CREATE PROCEDURE createMyView(IN name text)
BEGIN
    SET @sql = CONCAT("CREATE OR REPLACE VIEW mytable AS SELECT * FROM ", name);

    PREPARE stmt FROM @sql;
    EXECUTE stmt;

    DEALLOCATE PREPARE stmt;
END //
DELIMITER ;

Call this stored procedure to point to our newly created table:

CALL createMyView("mytable_2010_11_22");

Now, if we want to list the content of the table that the view currently points to, we can use something like this:

SELECT * FROM mytable;

In a previous article we looked at LVM. This short article describes how to grow an existing volume by adding an extra disk to the system. Specifically, we use a XenServer guest VM with CentOS installed.

The first disk of a XenServer guest VM is called xvda, the second disk (the one we added) is called xvdb. We start with the creation of a Physical Volume (PV) on the extra disk.

pvcreate /dev/xvdb

Then we extend the existing Volume Group (VG) called VolGroup00.

vgextend VolGroup00 /dev/xvdb

Then we extend the Logical Volume (LV) by the size of the extra disk, in this case 8GB.

lvextend -L8G /dev/VolGroup00/LogVol00

Finally we resize the filesystem that uses this LV.

resize2fs /dev/VolGroup00/LogVol00

Logical Volume Management (LVM) is a method of allocating space on mass storage devices that is more flexible than conventional partitioning schemes. To understand how LVM works we will first take a look at some terminology:

• PV: Physical Volume. This can be an entire harddisk, a partition on a harddisk, a RAID device (software or hardware), a LUN, etc.
• VG: Volume Group. This is an aggregation of one or more PVs, summing small storage spaces into a bigger consolidated one.
• LV: Logical Volume. This is a part of a VG that can be used to put a filesystem on.

As an example of how these terms work together, have a look at the following picture:

Breaking things up into these parts means that we can do fun stuff with LVM. First of all, we can have a filesystem that is bigger than the biggest harddisk we have in the system. We can also grow and shrink the space that is given to a filesystem and even do this while the filesystem is being used. Finally, we can create snapshots, read-only ones for backup purposes and read-write ones that allow us to have multiple versions of a filesystem.

Creating

We start by creating the physical volumes (PVs). If you are using a full harddisk as a physical volume it is best to create 1 partition on it that fills the whole disk. The type of the partition can best be set to “8E”, which is the type for “Linux LVM”. You can do this by running fdisk (“n” for new partition, “t” to change the type and “w” to write changes). Afterwards run partprobe to detect the new partitions:

fdisk /dev/sdb
partprobe

Create the physical volume on the partitions. Let’s assume we have three volumes: /dev/sdb1, /dev/sdb2 and /dev/sdb3:

pvcreate /dev/sdb{1,2,3}
pvdisplay /dev/sdb{1,2,3}

Create the volume group “vg0″ that will group these physical volumes together:

vgcreate /dev/vg0 /dev/sdb{1,2,3}
vgdisplay /dev/vg0

Create a logical volume named “lv0″ from the volume group “vg0″ with a size of 100MiB:

lvcreate -L 100M -n lv0 /dev/vg0
lvdisplay /dev/vg0/lv0

Create a filesystem on top of the logical volume “lv0″ and mount it:

mke2fs -j /dev/vg0/lv0
mount /dev/vg0/lv0 /mnt

Growing

We will now grow the filesystem. This can be done when the filesystem is offline, but ext3 also supports online growing. We start by growing the space that is available to the filesystem in the logical volume “lv0″:

lvextend -L 120M /dev/vg0/lv0

Now we can ask the filesystem to occupy the extra space it has been given:

resize2fs /dev/vg0/lv0

Shrinking

Shrinking a filesystem is a bit more difficult. First we need to unmount the filesystem:

umount /mnt

Run a filesystem check and resize it:

e2fsck -f /dev/vg0/lv0
resize2fs /dev/vg0/lv0 80M

Mount the filesystem again:

mount /dev/vg0/lv0 /mnt

At this time, the filesystem is smaller than its logical volume, so we can shrink it:

lvreduce -L 80M /dev/vg0/lv0

Characteristics

I wanted to make my own Network Attached Storage (NAS) device with the following characteristics:

1. It should have lots of disk space and new disk space should be easy to add.
2. The primary storage should be redundant: if one disk fails, the data should still be there.
3. Important data on the primary storage should be backed up automatically and incrementally to secondary storage.
4. It should be able to run VMware.

Hardware

The following hardware was used to construct the NAS:

• Antec NSK4480 mini tower with 380W power supply
• Asus M3N78-CM GeForce 8200 SATA2 mainboard
• AMD Athlon X2 5050e 2.6GHz 1MB 45W AM2 processor
• Scythe Ninja Mini processor cooler
• Kingston 2x2GB DDR2 SDRAM PC6400 memory
• One Samsung HD501LJ 500GB disk
• Three Western Digital WD10EACS 1TB disks
  
• Samsung SH-S223F DVD player

The 500GB disk will be used to hold the operating system, Ubuntu 8.10 desktop, and the backup of the most important files. The three 1TB disks will be bundled together into a software RAID5 set, as shown in the following diagram:

Software RAID

We start by installing the operating system, Ubuntu 8.10 desktop for i386 systems. I choose the desktop version because I wanted to be able to access the VMware GUI on the NAS itself. I started with the 64 bit version of Ubuntu 8.10, but it was harder to install and it seemed slower than the i386 version. It does mean that we loose some of the 4GB RAM, but that is not a big concern for me.

After installing the OS, we move on to the partitioning of the three disks that will form the RAID5 set. We do this by installing the program gparted which is ideal for partitioning disks from a GUI:

sudo su -
apt-get install gparted
gparted

In gparted, create a single partition on the three RAID5 disks.
Next, create the RAID5 set:

apt-get install mdadm
mdadm --create /dev/md0 --level=5 --chunk=128 --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sdd1

Create a filesystem on the RAID5 set we just created:

mkfs.ext3 -b 4096 -R stride=32 /dev/md0

Create a mountpoint for the RAID5 set:

mkdir /data

Add an fstab entry by editing /etc/fstab:

/dev/md0   /data   /ext3   defaults   0   0

Finally, mount the RAID5 set:

mount /data

Incremental backup

The most important files on the NAS need to be backed up to the disk that also holds the operating system. The program rshapsnot is ideal for this task. It will do fast incremental backups that are browsable on the filesystem. So there’s no need to use a special program to restore the backup. Just copy the files and you’re done.

Install rsnapshot:

apt-get install rsnapshot

Edit the rsnapshot configuration file /etc/rsnapshot.conf:

snapshot_root /backup/
cmd_cp /bin/cp
cmd_ssh /usr/bin/ssh
cmd_du /usr/bin/du
cmd_rsnapshot_diff /usr/bin/rsnapshot-diff
interval hourly 6
interval daily 7
interval weekly 4
interval monthly 12
backup /data/important_files/ localhost/

This configuration file tells rsnapshot some basic things like the whereabouts of some needed programs and which directories it should backup. In our case, the directory /data/important_files is backed up to the directory /backup. It also tells that there should be 6 “hourly backups”, 7 “daily backups”, 4 “weekly backups” and 12 “monthly backups”. The reason I use quotation marks here, is that you yourself have to setup the cron jobs to actually run rsnapshot at the time you want. Run the following command:

crontab -e

and add the following lines:

# m h  dom mon dow   command
0 */4 * * * /usr/bin/rsnapshot hourly
45 23 * * * /usr/bin/rsnapshot daily
30 23 * * 0 /usr/bin/rsnapshot weekly
15 23 1 * * /usr/bin/rsnapshot monthly

This crontab will run the different backups in the following way:

• The “hourly backup” is run every four hours (denoted by the hour value of */4) on the hour (denoted by the minute value of 0).
• Every day at 23:45, the “daily backup” is run.
• Every sunday (denoted by the day-of-the-week value of 0) at 23:30 the “weekly backup” is run.
• Every first day of the month (denoted by the day-of-the-month value of 1) at 23:15 the “monthly backup” is run.

In combination with the “interval hourly 6″ line in the rsnapshot configuration file, this means that the last 24 hours are covered with 6 backups. In the same way we can see that 7 days are covered in the last week, 4 weeks are covered in the last month and 12 months are covered in the last year.

File sharing

To actually start using the NAS, we will now install the Samba package:

apt-get install samba

Finally, we edit the configuration file /etc/samba/smb.conf:

security = user
[data]
   comment = Data
   browseable = yes
   path = /data
   guest ok = no
   read only = no
[backup]
   comment = Backup
   browseable = yes
   path = /backup
   guest ok = no
   read only = yes

This will make the primary storage accessible read-write through \nasdata and the backup accessible read-only through \nasbackup.