Monday, April 23, 2012

Getting Started with Amazon EC2 (part 2)

In the previous Amazon EC2 post we went through all the needed steps in order to start interacting with Amazon EC2, in this tutorial we will continue our exploration of EC2, we will go through some EC2 basics such as administration of instances and also some best practices, read on!

First we would like to create a test instance, instances are created out of pre-created images.

In order to see all the existing images on Amazon EC2 (which is quite a lot) and also some details of each image - such as it's architecture, kernel version, etc run:
$ec2-describe-images -a

Chose the image you would like to create your instance from, you will need to remember the image AMI id (ami-xxxx).

Create your instance from the selected image:

$ec2-run-instances ami-e565ba8c -k mykeypair

You can verify your instance is running with:

RESERVATION     r-3e2b245d      576950081803    quick-start-1
INSTANCE        i-c20b47a5      ami-e565ba8c  domU-12-31-39-15-0C-F7.compute-1.internal    running mytest   0               t1.micro        2012-04-22T11:52:04+0000        us-east-1d      aki-88aa75e1monitoring-disabled                     ebs                                     paravirtual xen              sg-448b4b2c     default
BLOCKDEVICE     /dev/sda1       vol-ba088bd5    2012-04-22T11:52:29.000Z        true

As we can see the instance is up and running. It's been assigned with a public DNS entry:

Now let's SSH into our instance; since by default SSH authentication to EC2 machines is key based we will need to specify our key (mytest.pem in my case):

$ssh -i mytest.pem  -l ec2-user

Last login: Sun Apr 22 11:54:03 2012 from

       __|  __|_  )
       _|  (     /   Amazon Linux AMI

Pay attention that I login as ec2-user, this is a special user which has full sudo permissions on the Amazon Linux AMI's:

[ec2-user@domU-12-31-39-15-0C-F7 ~]$whoami
[ec2-user@domU-12-31-39-15-0C-F7 ~]$ec2-user
[ec2-user@domU-12-31-39-15-0C-F7 ~]$ sudo su -
[root@domU-12-31-39-15-0C-F7 ~]# whoami

You can allow root login (which is not recommended due to security risks) on your instances inside /etc/ssh/sshd_conf , changing PermitRootLogin to "yes" and HUP ssh-pid.

Just for general knowledge - Amazon EC2 use Xen as it's hypervisor ,an obvious choice - since Xen is on of the most mature Open source hypervisors out there.
The disk layout of the instance is very simple, one big root partition:

[root@domU-12-31-39-15-0C-F7 ~]#df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/xvda1             5904752   1550536   4294228  27% /
none                    856180         0    856180   0% /dev/shm

Beware not to fill the root (/) partition , a good idea will be to monitor the disk usage of each created instance.

Another important point -

For critical applications such as production Web Servers and Databases I would warmly suggest connecting an external volume called EBS (Elastic Block Store) to your instance, which is a special volume that persists independently from the life of an instance.
Even if your instance gets destroyed (intentionally or not), the EBS still persists and can be re-connected to other instance.
Deploying your services in the cloud requires a different thinking, since the environment is now more "liquid" and dynamic you need to plan ahead and be ready to bring services up at no time since instances may come and go, this is one of the keys for any production deployment in the cloud.

I will talk about EBS in my future tutorials.

Let's create a basic default security policy, I would like to enable SSH,HTTP,HTTPS traffic from everywhere to my new created instance(s):

$for i in 22 80 443;do ec2-authorize default -p $i;done

You can see the applied security rules with:


... some output omitted ... 

PERMISSION      576950081803    default ALLOWS  tcp     22      22      FROM    CIDR       ingress
PERMISSION      576950081803    default ALLOWS  tcp     80      80      FROM    CIDR       ingress
PERMISSION      576950081803    default ALLOWS  tcp     443     443     FROM    CIDR       ingress

As we can see the rules have been implemented , so far so good.

Remember that Amazon charges you for your instance usage (pay as you go, so if we no longer require an instance it will be a good idea to get rid of it).
In order to terminate our instance (be sure to backup your files before that), verify your instance id with:

$ec2-describe-instances|grep -i instance|awk '{print $2}'

... and terminate it with:

$ec2-terminate-instances i-c20b47a5
INSTANCE        i-c20b47a5      running shutting-down

That's it for now.
Stay tuned for more updates.

Wednesday, April 18, 2012

Checkpoint FW :Failed to load Policy on Module

While not being exactly a security expert once in a while I have to deal with some security appliances especially in test environments (where FW rules need to be adjusted quite frequently).

Most of the time I was extremely pleased with Checkpoint products ,at least for me their products were rock solid - until one day I wasn't able to install new policies on my Checkpoint FW.  The symptom was quite awkward - after saving the policy and verifying it successfully, during the installation process I always got an error saying "Installation failed. Failed to load Policy on Module" no matter what I tried, no additional info was specified which complicated things a bit.

Here is my workaround for the problem:

After you've logged in into the appliance as admin user (either via console or ssh) ,type:
# Expert

In order to get into privileged (Expert) mode (which basically allows you to work as "root" user on the appliance , as it was a regular Linux box).

After you got into expert mode the prompt will change to:

Now, you need to locate the "fwm" process (which is the FW management), kill it and then restart it.

Please note that if your SmartDashboard (or any other Checkpoint applications) are connected to the FW ,it will terminate them, yet the FW traffic (including any established VPN connections) will not be affected, so proceed without worries:

[Expert@firewall]#ps -ef|grep fwm
[Expert@firewall]#kill fwm-pid
[Expert@firewall]#fwm &

After fwm was started successfully on your FW box, try installing the policy again - usually this should do the trick.

If restarting fwm did not help, as a last resort  only, you will need to restart the CP services.
This will of course disconnect any sessions and every established  VPN connections, so think twice before executing it:

[Expert@firewall]#cpstop && cpstart.

The CP restart process takes around ~1 minute during which the FW may seem unresponsive.

This did the trick for me and I hope it helped some one out there too.
If you have more elegant solution for this issue, please let me know.