Thursday, February 16, 2012

Howto resize XenServer LUN's Online

In the following procedure I will show how to extend iSCSI attached LUN's on XenServer (v 5.6 SP2 in my case) on the fly ,so no service restart or downtime are needed and the VDI's that reside on the resized LUN are not affected, read on!
First of all on the storage side (NetApp filer in my case)list the available LUN's, the LUN I want to resize is  called "my_lun30", with LUN ID of 22, which is currently allocated with 90GB.

For the test I will extend it by another 10GB, making it a 100GB LUN.

filer1> lun show
        /vol/vol1/my_lun10         300.0g (322163441664)  (r/w, online, mapped)
        /vol/vol2/my_lun20    200g (214748364800)  (r/w, online, mapped)
        /vol/vol2/my_lun30       90.0g (107388862464)  (r/w, online, mapped)


filer1> lun resize /vol/vol2/my_lun30 +10g
lun resize: resized to:  100.0g (107388862464)


As you can see, my_lun30 is now 100GB:

filer1> lun show
        /vol/vol1/my_lun10         300.0g (322163441664)  (r/w, online, mapped)
        /vol/vol2/my_lun20    200g (214748364800)  (r/w, online, mapped)
        /vol/vol2/my_lun30        100.0g (107388862464)  (r/w, online, mapped)

We are done with the storage side, let's head to the XenServer side.
 In case you work in pool mode, login to the pool master as root:

I suggest installing "lsscsi" which provides a nice way of viewing SCSI attached disks/LUNs:
[root@xen3]#yum install lsscsi -y

[root@xen3]# lsscsi
[0:0:0:0]    cd/dvd  Optiarc  DVD RW AD-7561S  AH52  /dev/scd0
[2:0:0:0]    disk    NETAPP   LUN              7340  /dev/sda
[2:0:0:22]   disk    NETAPP   LUN              7340  /dev/sdb
[2:0:0:33]   disk    NETAPP   LUN              7340  /dev/sdc
[3:0:0:1]    disk    NETAPP   LUN              7340  /dev/sdd
[3:0:0:3]    disk    NETAPP   LUN              7340  /dev/sde

If YUM traffic to a repository is blocked - It's also possible to see the SCSI id's under /proc via:

# cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: Optiarc  Model: DVD RW AD-7561S  Rev: AH52
  Type:   CD-ROM                           ANSI  SCSI revision: 05
Host: scsi2 Channel: 00 Id: 00 Lun: 00
  Vendor: NETAPP   Model: LUN              Rev: 7340
  Type:   Direct-Access                    ANSI  SCSI revision: 04
Host: scsi2 Channel: 00 Id: 00 Lun: 22
  Vendor: NETAPP   Model: LUN              Rev: 7340
  Type:   Direct-Access                    ANSI  SCSI revision: 04
Host: scsi2 Channel: 00 Id: 00 Lun: 33
  Vendor: NETAPP   Model: LUN              Rev: 7340
  Type:   Direct-Access                    ANSI  SCSI revision: 04
Host: scsi3 Channel: 00 Id: 00 Lun: 01
  Vendor: NETAPP   Model: LUN              Rev: 7340
  Type:   Direct-Access                    ANSI  SCSI revision: 04
Host: scsi3 Channel: 00 Id: 00 Lun: 03
  Vendor: NETAPP   Model: LUN              Rev: 7340
  Type:   Direct-Access                    ANSI  SCSI revision: 04

From the output we can see the SCSI ID of the LUN + it's corresponding device on the system, and check the device physical size, as you can see it is not updated yet:

[root@xen3 backup_scripts]# fdisk -l /dev/sdb

Disk /dev/sdb: 96.6 GB, 96647249920 bytes
255 heads, 63 sectors/track, 11750 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdb doesn't contain a valid partition table

Since XenServer uses LVM, we can see the physical volume info of the disk, as you can see the size is still 90GB.
  
[root@xen3]# pvdisplay /dev/sdb
  --- Physical volume ---
  PV Name               /dev/sdb
  VG Name               VG_XenStorage-e71389d1-4dc3-2518-aa30-9f5f0c70ba12
  PV Size               90.01 GB / not usable 6.12 MB
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              23039
  Free PE               16624
  Allocated PE          6415
  PV UUID               oSZlnA-VxlA-3qbp-07nI-Ql0b-3cG2-9wo2lC


Now, we will tell XenServer to rescan the SCSI bus, we will provide the SCSI id which we previously got from the "lsscsi" command (2:0:0:22):

[root@xen3]# echo 1 > /sys/class/scsi_disk/2:0:0:22/device/rescan

You can notice the immediate change of /dev/sdb:

[root@xen3]# fdisk -l /dev/sdb

Disk /dev/sdb: 107.3 GB, 107388862464 bytes
255 heads, 63 sectors/track, 13055 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdb doesn't contain a valid partition table

Now, resize the physical volume with:

[root@xen3]# pvresize /dev/sdb

  Physical volume "/dev/sdb" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized

Check again the physical volume size, notice the change:

[root@xen3]#  pvdisplay /dev/sdb
  --- Physical volume ---
  PV Name               /dev/sdb
  VG Name               VG_XenStorage-e71389d1-4dc3-2518-aa30-9f5f0c70ba12
  PV Size               100.01 GB / not usable 6.12 MB
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              25600
  Free PE               19185
  Allocated PE          6415
  PV UUID               oSZlnA-VxlA-3qbp-07nI-Ql0b-3cG2-9wo2lC

Let's get the SR uuid, I know the LUN ID is 22, so:

[root@xen3]# xe sr-list|grep lun22 -B1
uuid ( RO)                : e71389d1-4dc3-2518-aa30-9f5f0c70ba12
          name-label ( RW): iSCSI_filer1_lun22

Notice that the SR size has yet to be updated:

[root@xen3]#  xe sr-param-list uuid=e71389d1-4dc3-2518-aa30-9f5f0c70ba12|grep physical-size
           physical-size ( RO): 96632569856

Now, finally update the relevant SR:

[root@xen3]#  xe sr-update uuid=e71389d1-4dc3-2518-aa30-9f5f0c70ba12

And at last, the SR is updated with the correct new LUN size :
[root@xen3]#  xe sr-param-list uuid=e71389d1-4dc3-2518-aa30-9f5f0c70ba12|grep physical-size
           physical-size ( RO): 107374182400


You are done!

Monday, February 13, 2012

Getting Started with Amazon EC2 (part 1)

In the next series of tutorials I will document some of my experiences with Amazon EC2 cloud services, and provide a small guide which will hopefully help you with your first steps with Amazon.
 Being a command line guy I immediately wanted to put my hands on the Amazon CLI tool set called  "ec2-api-tools" ,which allows us to fully interact with the EC2 services.

It is of course possible to use the "traditional" Web GUI -
AWS Management Console, but in case you are a developer who really want to understand and feel EC2 true capabilities or a sys admin who is planning to create a decent automation solution for your instances in the Cloud, you will need to master the EC2 command line , so get it from here.

For my tests I have used a CentOS v5.5 x64 client.

Some pre-requirements first.

Check that Java is installed and operational (if not "yum install" it):
# rpm -qa|grep jdk
java-1.6.0-openjdk-1.6.0.0-1.23.1.9.10.el5_7

# java -version
java version "1.6.0_20"
OpenJDK Runtime Environment (IcedTea6 1.9.10) (rhel-1.23.1.9.10.el5_7-x86_64)
OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)

We will create a folder that will be dedicated to Amazon related stuff:
#mkdir ~/.amazon
#mv ~/ec2-api-tools.zip ~/.amazon/;cd ~/.amazon
#unzip ec2-api-tools.zip

Connection to Amazon EC2 services is secured using .x509 certificates.
So, in order to be able to interact with EC2 from your client you will need to generate public and private keys and put them into  ~/.amazon folder.

Log-in to Amazon Web Services , and go to your account settings, from there select "Security Credentials" option.

Inside you should find "Access Credentials", with a tab called "x.509 Certificates".
Select "Create a new Certificate" and download both of the keys into ~/.amazon directory.












Your ~/.amazon folder should now contain the unzipped tools folder + the 2 keys:
ec2-api-tools-1.5.2.4
pk-YOURID.pem
cert-YOURID.pem

Make sure to set the appropriate permissions (private key should be only readable by it's owner!) on the keys:
#chmod 0400 pk-YOURID.pem
#chmod 0644 cert-YOURID.pem

Next, we will need to modify your ~/.bashrc file with the appropriate environment variables:
#vi ~/.bashrc

#Amazon related variables...
export EC2_HOME="~/.amazon/ec2-api-tools-1.5.2.4"
export PATH=$PATH:${EC2_HOME}/bin
export EC2_PRIVATE_KEY=pk-YOURID.pem
export EC2_CERT=cert-YOURID.pem
export JAVA_HOME="/usr" #or wherever "java" binary resides

Save the file and trigger the shell to re-read the changes:
#source ~/.bashrc


Lastly we need to configure a private key for SSH sessions into the instances:
#cd ~/.amazon
#ec2-add-keypair name-of-keypair

Copy the contents of the generated private key and paste them into a file:
#cat > ~/.amazon/id_rsa_name-of-keypair

And set the correct permissions on it:
#chmod 0400 ~/.amazon/id_rsa_name-of-keypair

We are now ready to begin Interacting with Amazon via ec2-* commands.
You can test that everything works as should with the following command:
  
#ec2-version
1.5.2.4 2011-12-15


More about the CLI basics and further explorations are soon to come, so stay tuned!

Monday, February 6, 2012

CFEngine - Beginner's Guide (Book Review)

These days, when terms such as “Cloud Computing” are not just a buzzwords but a reality, the rules of the game change. The IT staff is required to re-think their strategy and general approach towards system administration in order to stay efficient and be able to sustain these large-scale, demanding and extremely dynamic computing environments.

CFEngine is a tool that provides the IT staff the operational agility, efficiency, and insight to be able to cope with the demands of the largest infrastructure environments. It provides an incredible solution for automating various system administration tasks, thus allowing the IT staff to be able to utilize their time better and focus on creativity instead of configuring the same services over and over again.
CFEngine is ideal for large-scale computing environments - Cloud Computing providers, Private Clouds & HPC clusters being the best examples.

Whether if you’re an IT manager, System Engineer/Administrator who works in such environment, most chances you will sooner or later run into CFEngine and need to be familiar with its concepts and potential.
A book I highly recommend as a great study guide is “Cfengine – Beginners Guide” (PACKT Publishing).



The book covers the latest version of CFEngine (version 3), and explains in detail how to make your first steps with CFEngine from the point of initial deployment to the stage where you need to sustain a large scale compute environment being able to bring up services in no time.

The book starts with description of CFEngine architecture, explaining the basic CFEngine functionalities describing how CFEngine various daemons are correlated.
Later on the book provides very practical, real-life scenarios and examples, explained carefully step by step by demonstrating each configuration in action.

Book’s chapters deal with various systems administration tasks and explain how CFEngine makes possible to automate them, for example:

  •       Configuring Systems - Deploying services (such as MySQL, NFS and many more),     network configuration, package management, adding/removing users.
  •        Security Audit – Modifying Iptables rules, service hardening, editing tcpwrappers.
  •       System Audit – Log rotation, Apache modifications

There is also a whole chapter dealing with CFEngine best practices such as – policy creations, potential pitfalls, integration with version control and more, giving the reader a wider picture, thus letting him achieve more efficient implementation of CFEngine in his environment.


Don’t let the name “Beginners Guide” to delude you - the coverage of the book will suit for the most advanced users covering not-trivial topics such as:
Writing new functions, Working with variables inside policies and much much more.

Another cool feature of the book is that each chapter includes a small quiz (answers are also provided), so the reader can test his understanding – thus mastering the tool better.

Bottom line - If you’re looking for a reliable CFEngine learning source look no further.