AWS Command Line Resources

http://aws.amazon.com/developertools

AWS Auto Scaling Part 2 - Auto Scaling Based on Fixed Number of Instances

In part 1 - AWS Auto Scaling Part 1 - Configuring Auto Scaling Command Line Tools, we have spinned a new ubuntu machine and installed the auto scaling command line tools.

We will create two things

• a launch configuration (defines what AMI to be launched)
• an auto scaling group (defines the number of instances to be launched, etc)

Creating a Launch Configuration

We will create a launch configuration.

as-create-launch-config --image-id --instance-type

Choose an AMI of your choice:

as-create-launch-config NodeJS --image-id ami-111111 --instance-type t1.micro

Check the launch configuration

as-describe-launch-configs --headers

Note: In the AWS EC2 Console, you can create an AMI by right-clicking one of your EC2 instances and click Create Image.


Creating an Auto Scaling Group

The auto scaling group takes the following as its parameters:

• name for the group
• a launch configuration
• one or more availability zones
• a minimum group size
• a maximum group size

We will create a group called NodeJSGroup. It will launch the NodeJS configuration we created above. We will use us-east-1d as the region and we want to spin 1 instance.

as-create-auto-scaling-group NodeJSGroup --launch-configuration NodeJS --availability-zones us-east-1d --min-size 1 --max-size 1

Check the status of the group by:

as-describe-auto-scaling-groups --headers

Check the health of the auto scaling instances:

as-describe-auto-scaling-instances --headers 

You should see the health of the launched instances. If you don't see any, check the activity log

as-describe-scaling-activities

Deleting launch configurations and auto scaling groups

We will first remove all the instances from the Auto Scaling Group NodeJSGroup. Then we will delete the launch config and the group.

First update the group setting to terminate all the instances.

as-update-auto-scaling-group NodeJSGroup --min-size 0 --max-size 0

Now delete the group and the launch config.

as-delete-auto-scaling-group NodeJSGroup
as-delete-launch-config NodeJS

A more complicated example (with device mapping, security group

The launch configuration above doesn't take into account of the security groups and device-block-mappings. We will create a more complicated example below.

To check the device-block-mappings of an AMI, you will need to install the EC2 API Tools and use the ec2-describe-images command.

ssh connect to the machine you want to use auto-scaling on.

Run either of the following commands.

ec2-describe-images -o self
ec2-describe-images

You will get something like the following

IMAGE ami-17acc4ee 140591131275/nodejs-production-20130522 240591131275 available private x86_64 machine aki-125ea7eb ebs paravirtual xen
BLOCKDEVICEMAPPING EBS /dev/sda1 snap-1f356ee2 8 true standard
BLOCKDEVICEMAPPING EBS /dev/sdf snap-18356ee5 20 false standard
BLOCKDEVICEMAPPING EBS /dev/sdg snap-15356ee8 20 false standard
BLOCKDEVICEMAPPING EPHEMERAL /dev/sdb ephemeral0

For the above, the block-device-mapping will be

block-device-mapping=/dev/sda1=snap-1f356ee2, /dev/sdf=snap-b8356ee5, /dev/sdg=snap-b5356ee8, /dev/sdb=ephemeral0

Find the instance's security group in the AWS EC2 console

--group security_group

You will need to specify a key pair to ssh into this instance as well

--key key_pair

The whole command will be:

as-create-launch-config NodeJS --image-id --instance-type m1.large --block-device-mapping="/dev/sda1=snap- 1f356ee2, /dev/sdf=snap-18356ee5, /dev/sdg=snap-15356ee8, /dev/sdb=ephemeral0" --group security_group  --key key_pair

Now create the auto scaling group.

as-create-auto-scaling-group NodeJSGroup --launch-configuration NodeJS --availability-zones us-east-1d --min-size 1 --max-size 5 --tag "k=Name, v=AsNodeJSProd, p=true"

The tag name will propagate to all the instances. If you don't specify a tag, your instances will have no human readable names. Read more about tags here. k=Name must have Name capitalized, else you won't see the human readable names in your AWS EC2 console.

Check the status by using the following commands

as-describe-launch-configs
as-describe-auto-scaling-groups
as-describe-auto-scaling-instances

You should have one machine launched.

Keep in mind that in some newer instances, the volume names are different. For example,

/xvda1 = /sda1
/xvdb = /sdb
/xvdf = /sdf
/xvdg = /sdg

When using the as-create-launch-config command, use the names returned from as-describe-images even if the actual volumes are different names.

For device-block-mapping, you can specify different kinds of volumes with different sizes. Read Block Device Mapping for more information.

Now let's stop the auto scaling group, else you will need to pay.

as-update-auto-scaling-group NodeJSGroup --min-size 0 --max-size 0
as-delete-auto-scaling-group NodeJSGroup
as-delete-launch-config NodeJS

In the next post, we will go into auto scaling based on metrics like CPU utilizations via the CloudWatch API.

Amazon EC2 Command Line Tools Installation Guide

Begin by ssh into your EC2 instance.

Create the following directory /opt/tools

mkdir /opt/tools

Download EC2 API tools.

wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip
unzip ec2-api-tools.zip

Install the latest version of Java. Read Install Java OpenJDK 7 on Amazon EC2 Ubuntu.

Set the EC2_HOME directory in ~/.bashrc

export EC2_HOME=/opt/tools/ec2-api-tools-1.6.7.3
export PATH=$PATH:$EC2_HOME/bin

Add the Security Credentials to ~/.bashrc. You can get your security credentials in you AWS console.

export AWS_ACCESS_KEY=your_AWS_ACCESS_KEY_ID
export AWS_SECRET_KEY=your_AWS_SECRET_KEY

Try this command:

ec2-describe-regions 

AWS Auto Scaling Part 1 - Configuring Auto Scaling Command Line Tools

In this post, we will experiment with Amazon's auto scaling service.

We will first begin by installing the Auto Scaling Command Line Tools in a new Ubuntu machine.

Connect to your machine by ssh.


Download and Unzip the Auto Scaling Command Line Tools

mkdir /opt/tools
cd /opt/tools
wget http://ec2-downloads.s3.amazonaws.com/AutoScaling-2011-01-01.zip

sudo apt-get install unzip
unzip AutoScaling-2011-01-01.zip


Install Java

Read Install Java OpenJDK 7 on Amazon EC2 Ubuntu.


Setting the environment variables

In your  ~/.bashrc file, append the following lines to the end of the file.

export JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::")
export PATH=$JAVA_HOME/bin:$PATH

Set the AWS_AUTO_SCALING_HOME to the location where you unzipped the command line tools.

export AWS_AUTO_SCALING_HOME=/opt/tools/AutoScaling-1.0.61.2
export PATH=$PATH:$AWS_AUTO_SCALING_HOME/bin

Install the Security Credentials

Go to the AWS security console.

Scroll to the Access Credentials section.

Note down an active pair of Access Key ID and Secret Access Key (Click show to see the secret access key).

vi /opt/tools/AutoScaling-1.0.61.2/credential-file-path.template

Paste your keys:

AWSAccessKeyId=
AWSSecretKey=

Append to ~/.bashrc

export AWS_CREDENTIAL_FILE=/opt/tools/AutoScaling-1.0.61.2/credential-file-path.template

Setting the Auto Scaling Region

By default, the auto scaling region is us-east-1.

If want to use a different region, you need to change to your region. Note down the region endpoint here: Regions and Endpoints.

vi ~/.bashrc
export AWS_AUTO_SCALING_URL=https://autoscaling.us-east-1.amazonaws.com

Test your configuration

as-cmd

You should see a panel of commands like the following:

Command Name                                Description
------------                                -----------
as-create-auto-scaling-group                Create a new Auto Scaling group.
as-create-launch-config                     Creates a new launch configuration.
as-create-or-update-tags                    Create or update tags.
as-delete-auto-scaling-group                Deletes the specified Auto Scaling group.
as-delete-launch-config                     Deletes the specified launch configuration.
as-delete-notification-configuration        Deletes the specified notification configuration.
as-delete-policy                            Deletes the specified policy.
as-delete-scheduled-action                  Deletes the specified scheduled action.
as-delete-tags                              Delete the specified tags
as-describe-adjustment-types                Describes all policy adjustment types.
as-describe-auto-scaling-groups             Describes the specified Auto Scaling groups.
as-describe-auto-scaling-instances          Describes the specified Auto Scaling instances.
as-describe-auto-scaling-notification-types Describes all Auto Scaling notification types.
as-describe-launch-configs                  Describes the specified launch configurations.
as-describe-metric-collection-types         Describes all metric colle... metric granularity types.
as-describe-notification-configurations     Describes all notification...given Auto Scaling groups.
as-describe-policies                        Describes the specified policies.
as-describe-process-types                   Describes all Auto Scaling process types.
as-describe-scaling-activities              Describes a set of activit...ties belonging to a group.
as-describe-scheduled-actions               Describes the specified scheduled actions.
as-describe-tags                            Describes tags
as-describe-termination-policy-types        Describes all Auto Scaling termination policy types.
as-disable-metrics-collection               Disables collection of Auto Scaling group metrics.
as-enable-metrics-collection                Enables collection of Auto Scaling group metrics.
as-execute-policy                           Executes the specified policy.
as-put-notification-configuration           Creates or replaces notifi...or the Auto Scaling group.
as-put-scaling-policy                       Creates or updates an Auto Scaling policy.
as-put-scheduled-update-group-action        Creates or updates a scheduled update group action.
as-resume-processes                         Resumes all suspended Auto... given Auto Scaling group.
as-set-desired-capacity                     Sets the desired capacity of the Auto Scaling group.
as-set-instance-health                      Sets the health of the instance.
as-suspend-processes                        Suspends all Auto Scaling ... given Auto Scaling group.
as-terminate-instance-in-auto-scaling-group Terminates a given instance.
as-update-auto-scaling-group                Updates the specified Auto Scaling group.
help
version                                     Prints the version of the CLI tool and the API.

    For help on a specific command, type ' --help'

In Part 2, we will go through an example that will launch an instance via a auto scaling group.

Symfony 2 Forms - validate at least one field is non empty

Say you have a form that allows you to add a post to an existing list or a new list. The form will have a dropdown box with existing lists and an input box asking the user to input the name of the new list. When the user clicks on submit, the post will be added to that list.

In earlier version of Symfony 2, you can use the CallbackValidator to do that, but it's deprecated. You will need to use event listener now.

In the following example, the dropdown box has the element name 'listId' while the input box has the element name 'listName'.

In your form's buildForm function, add the following EventListener.

public function buildForm(FormBuilderInterface $builder, array $options)

{

        $builder->addEventListener(FormEvents::POST_BIND, function (DataEvent $event) {

            $form = $event->getForm();

            $listName = $form['listName']->getData();

            $originalListId = $form['listId']->getData();

            if( ($listName == null || $listName == '') && ($listId == null || $listId == '') ) {

                $form['listId']->addError(new FormError("Please select a list or create a new list."));

            }

        });

}

Windows Server 2012 - Internet Explorer Change Internet Security

I was spinning a Windows Machine on EC2 the other day, and I came across this annoying problem - every time I try to browse a site, there's this pop up to ask me to add the domain as a trusted source. Download is also blocked.

The first thing I tried is going to Internet Options -> Security -> Internal. The options to set the security level to lower than high are all grey-ed out.

At the bottom left corner, click on the Server Manager icon.

Click on the Local Server text at the left sidebar.

In this Tasks table, search for IE Enhanced Security Configuration (2nd column, around the 7th row).

Click on the "On" link.

Click the "Off" radio button for Administrators.

Now restart IE and try downloading a file.

Mac Java Could not start Tomcat - Address already in use

Could not start Tomcat: Protocol handler initialization failed: java.net.BindException: Address already in use :8080 -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:tomcat-maven-plugin:1.1:run (default-cli) on project hello: Could not start Tomcat

If you are running on the Mac, try this

pkill -9 java