Setting up Lighttpd Load Balancer on EC2 Ubuntu

Lighttpd is an asynchronous server. Along with Nginx, Lighttpd is one of the fast servers designed to counter the C10k problem. If you want to set up Nginx, read Setting up Nginx on EC2 Ubuntu.

This tutorial will demonstrate how to use Lighttpd to load balance application servers.


Creating a EC2 Instance

In the AWS Management Console, begin by creating a t1.micro Ubuntu Server 12.04.1 LTS 64-bit. (If you don't know how to create an instance, read Amazon EC2 - Launching Ubuntu Server 12.04.1 LTS step by step guide.

Here are some guidelines:

• Uncheck Delete on Termination for the root volume
• Add port 22, 80 and 443 to the Security Group, call it lighttpd.

Install Lighttpd

ssh -i {key} ubuntu@{your_ec2_public_address}

sudo apt-get update -y

sudo apt-get install -y lighttpd

Lighttpd should be running. To check its status, run

service lighttpd status

All the configuration files are located in /etc/lighttpd

To enable/disable a module

• Use /usr/sbin/lighty-enable-mod and /usr/sbin/lighty-disable-mod
• Or create a symbolic link from /etc/lighttpd/conf-available/{module} to /etc/lighttpd/conf-enabled/module

To load balance application servers, we will be using the 10-proxy.conf file as a template.

cd /etc/lighttpd/conf-available

cp 10-proxy.conf 11-proxy.conf

vi 11-proxy.conf

We are interesting in the following two variables:

• proxy.balance - choose from hash, round-robin or fair
• proxy.server - put the servers you want to load balance to

For example:

proxy.balance     = "hash"
proxy.server     = ( "" =>
                     (
                       ( "host" => "10.204.199.85",
                         "port" => 80
                       ),
                       ( "host" => "10.202.111.140",
                         "port" => 80
                       )
                     )
                    )

The above settings will load balance to two other servers based on IP.

Restart the server.

service lighttpd restart

Test the server.

To check the status:

netstat -ntulp

Setting up Nginx on EC2 Ubuntu

Nginx is a high performance Web server and a reverse proxy. It is one of the top servers that can counter the C10K problem. It can be used to load balancer application servers and serve static assets.

There are many ways to set up your load balancers in AWS. Here are some examples:

1. Elastic Loader Balancer -> Application and Nginx on each server
2. Three layers: Elastic Load Balancer -> Nginx Servers(cache, load balancers) -> Application Servers
3. Elastic Loader Blancer -> Application servers; using ElasticFront to server assets
4. Nginx -> Application servers

Instagram uses the 2nd approach above.

This tutorial will focus on setting up Nginx on a single EC2 instance, while load balancing the application servers.

Creating a EC2 Instance

In the AWS Management Console, begin by creating a t1.micro Ubuntu Server 12.04.1 LTS 64-bit. (If you don't know how to create an instance, read Amazon EC2 - Launching Ubuntu Server 12.04.1 LTS step by step guide.

Here are some guidelines:

• Uncheck Delete on Termination for the root volume
• Add port 22, 80 and 443 to the Security Group, call it Nginx.

Installing Nginx

ssh into your instance.

ssh -i {your key} ubuntu@{your_ec2_public_address}

sudo apt-get update

sudo apt-get install -y nginx

Check the nginx version

nginx -v

If this is not the latest version, do the following:

sudo vi /etc/apt/sources.list

Add:

deb http://nginx.org/packages/ubuntu/ precise nginx
deb-src http://nginx.org/packages/ubuntu/ precise nginx 

sudo apt-get update

You will get:

W: GPG error: http://nginx.org precise Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62

Add the public key:

wget http://nginx.org/packages/keys/nginx_signing.key
cat nginx_signing.key | sudo apt-key add -

sudo apt-get install nginx

You may get the following:

dpkg: error processing /var/cache/apt/archives/nginx_1.2.6-1~precise_amd64.deb (--unpack):
 trying to overwrite '/etc/logrotate.d/nginx', which is also in package nginx-common 1.1.19-1ubuntu0.1
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
Errors were encountered while processing:
 /var/cache/apt/archives/nginx_1.2.6-1~precise_amd64.deb

apt-get remove nginx-common
sudo apt-get install nginx

Check your version to make sure it's the latest version (nginx -v).

Make Nginx start on boot.

update-rc.d nginx defaults

Nginx Basic Commands

sudo service nginx start
sudo service nginx stop
sudo service nginx restart
sudo service nginx status


Checking IP of your browser:

ifconfig eth0 | grep inet | awk '{ print $2 }'

Load balancing servers

We will have the Nginx server to load balance two servers (backend1.example.com and backend2.example.com) in a round robin fashion.

Begin by creating a new virtual host configuration file.

cp /etc/nginx/sites-available/default /etc/nginx/sites-available/{domain}

Put the following into the file:


upstream domain {
    ip_hash;
    server backend1.example.com:8080;
    server backend2.example.com:8080;
}
server {
    listen 80;
    server_name domain
    access_log /var/log/nginx/web_portal.access.log;
    location / {
            proxy_pass      http://domain/;
            proxy_next_upstream error timeout invalid_header http_500;
            proxy_connect_timeout 2;
            proxy_set_header        Host            $host;
            proxy_set_header        X-Real-IP       $remote_addr;
            proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_intercept_errors on;
    }
}

Make sure the domain above match with your request domain.

It is very important to have the following two attributes. It defines what happens when a server is down. In this case, it would redirect the client request to the next machine if the server is not responding within 2 secs.

proxy_next_upstream error timeout invalid_header http_500;
proxy_connect_timeout 1;

Check out proxy_read_timeout as well.

ip_hash will always send the client back to the same server based on the IP.

Check the Nginx Wiki for more info.

Disable the default config.

rm /etc/nginx/sites-enabled/default

Enable the configuration by symbolic link to sites-enabled.

sudo ln -s /etc/nginx/sites-available/{domain} /etc/nginx/sites-enabled/{domain}

If Nginx doesn't seem to pick up on the configuration, make sure /etc/nginx/nginx.conf has the following within the http block:

include /etc/nginx/sites-enabled/*;

Restart the server.

service nginx restart

To deploy code without service interruption, read Nginx - How to deploy code without service disruption.

Nginx - How to deploy code without service disruption

Let's say you have the following upstream settings (Nginx load balancing two servers):

# /etc/nginx/sites-available/backend
upstream backend {
        server backend1.example.com:8080;
        server backend2.example.com:8080;
}

To update backend1.example.com, mark it as down,

# /etc/nginx/sites-available/backend
upstream backend {
        server backend1.example.com:8080 down;
        server backend2.example.com:8080;
}

> service nginx reload

Now update your code and reload the server configs.

> service nginx reload

Similarly, for the second server

upstream backend {
        server backend1.example.com:8080;
        server backend2.example.com:8080 down;
}

> service nginx reload

Update code on second server

> service nginx reload

The reload command does not restart the Nginx server.