Quickly Validating a Load-Balanced Website Architecture

Feb 02, 2014

devops website load balance tutorial vagrant testing

I am in the process of consolidating a number of websites that I use into a smaller set of servers. This is partly being done to reduce costs as well as support less resources, but it also provides me with a chance to try out new things.

In particular, I wanted to test out a load balanced architecture that will work within any VPS environment. Before I begin provisioning "real" servers and having companies bill me, I wanted to test that my ideal architecture works. I'm using the word "real" here to denote servers that will be public-facing and acting in a production-like capacity, as well as incurring billing charges.

To validate the test I used VirtualBox (via Vagrant) and provisioned 3 servers, one to act as a load balancer and two to act as back-end Web servers. This article will act as a guide for anyone wanted to test out this type of architecture before using real servers.

Requirements

Ubuntu (tested on 13.10 but should work with 12.04 and newer)
Vagrant
VirtualBox
A host machine with at least 6 GB of free RAM to allocate to VMs. ** The host machine can be any operating system, but I will be using POSIX commands. Substitute for Windows commands where necessary.
Nginx

Before we begin

Ensure that you are familiar with all of the required tools, or else you may find yourself lost when I run some of these commands.

Some conventions will be used here so a glossary is needed:

LB: load balancer
Web: Web server

Provision VMs

Using Vagrant, we can easily provision a set of 3 VMs (1 LB, 2 Web) inside one Vagrantfile.

First we create ourselves a play area:

mkdir /tmp/lb-test
cd /tmp/lb-test

Then we initialize a new vagrant repository.

vagrant init

We will be using Ubuntu 13.10, so download the vagrant-compatible box (found using http://vagrantbox.es).

vagrant box add ubuntu-13.10 http://cloud-images.ubuntu.com/vagrant/saucy/current/saucy-server-cloudimg-amd64-vagrant-disk1.box

We are going to overwrite the default Vagrantfile with the following text. Note that I have removed all superfluous comments to make the file smaller and more readable:

# -*- mode: ruby -*-
# vi: set ft=ruby :

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.define :lb do |vps|
    vps.vm.box = "ubuntu-13.10"
    vps.vm.network :private_network, ip: "192.168.33.10"
    vps.vm.provider :virtualbox do |vb|
      vb.gui = false
      vb.customize ["modifyvm", :id, "--memory", "1024"]
    end
  end

  config.vm.define :web1 do |vps|
    vps.vm.box = "ubuntu-13.10"
    vps.vm.network :private_network, ip: "192.168.33.11"
    vps.vm.provider :virtualbox do |vb|
      vb.gui = false
      vb.customize ["modifyvm", :id, "--memory", "2048"]
    end
  end

  config.vm.define :web2 do |vps|
    vps.vm.box = "ubuntu-13.10"
    vps.vm.network :private_network, ip: "192.168.33.12"
    vps.vm.network :public_network
    vps.vm.provider :virtualbox do |vb|
      vb.gui = false
      vb.customize ["modifyvm", :id, "--memory", "2048"]
    end
  end
end

Most people only use a Vagrantfile to create a single VM, but using the syntax above allows us to create multiple machines, each with their own IP address.

The LB will be created with only 1GB of RAM since it will only be using nginx to load balancer the back-end Web servers. The other VMs are created with 2GB of RAM which should be enough for the majority of what we need to test.

I have not included a provisioning script (provisioning is what configures a server with a specific set of software) because I want to show how this can be done manually.

The last thing to do is run the following command to create the virtual machines and start them.

vagrant up

Configure the Load Balancer

Login to the load balancer using the following shortcut provided by Vagrant (if you are on Windows this won't connect to an SSH terminal, but it will provide you with details about how you can do it manually).

vagrant ssh lb

To make things easier, and to prevent having to type IP addresses everywhere, we update our hosts file to use custom domain names. From now on, we can use these domain names when referring to the back-end Web servers.

sudo vi /etc/hosts

192.168.33.11   web1
192.168.33.12   web2

Next we need to setup Nginx to act as the load balancer. This is a very fast server that provides a simple configuration script for us to specify back-end hosts.

sudo apt-get install -y nginx

Nginx is now installed and ready to be configured. It has not yet been started. We will update the default site configuration (partly because it already exists with default values, and because it is set to listen on port 80).

vi /etc/nginx/sites-available/default

Overwrite the entire file with the following text. I will go over each statement afterward.

# tell nginx the names/IPs of your backend servers
# default load balancing algorithm is round-robin
upstream backend {
        server web1;
        server web2;
}

server {
        listen 80 default_server;
        listen [::]:80 default_server ipv6only=on;

        server_name localhost;

        location / {
                # here we pass all requests to the backend, defined earlier
                proxy_pass http://backend;
        }

        access_log /var/log/nginx/access.log main;
        error_log /var/log/nginx/error.log error;
}

With the configuration done, we can now start the server.

sudo service nginx start

Configure Web Servers

Since there are two Web servers, the following section will need to be performed once on each server.

sudo apt-get install -y nginx

sudo mkdir /var/www
sudo touch /var/www/index.html

Depending on which server this is, change the server number accordingly:

sudo echo "Server 1" >> /var/www/index.html

We will need to configure Nginx to serve a static website (it could be any site, but this is just a simple test).

vi /etc/nginx/sites-available/default

Overwrite the file with the following text. Comments help to show what is going on.

server {
  # tell nginx which port to listen on
  listen 80 default_server;
  listen [::]:80 default_server ipv6only=on;


  # Make site accessible from http://localhost/
  server_name localhost;

  location / {
    # tell nginx where all HTML files are located
    root /var/www;

    # nginx will load these files by default if not given a specific file
    index index.html index.htm;
  }
}

With nginx configured to serve our static website, we can now turn it on and start accepting requests.

sudo service nginx start

tail -f /var/log/nginx/access.log

Test the Setup

In order to test that our server is working correctly, we'll install a text-based web browser called eLinks. Login to the load balancer and run the following:

sudo apt-get install -y elinks

elinks http://localhost

The browser will open and display either "Server 1" or "Server 2". If you press Ctrl-R it will perform a page refresh and send a new request through the load balancer. You will see the "Server 1" change to "Server 2". Repeat this refresh multiple times and you will see that your request is being round-robin'd to each server.

You will notice that, upon each page refresh, the logs that you are tailing on each web server will grow, according to which server is responding to the request.

Now we are going to test the failover capability of the load balancer. Login to web1 and turn off the web server.

sudo service nginx stop

If you switch back to the load balancer and refresh the page multiple times you will notice that you only receive "Server 2" as a result. This mean the load balancer has recognized the downed back-end server and is now sending all requests to the remaining one.

If you are still tailing the access logs on each web server, you will also notice that only web2's logs are growing.

Conclusion

With this setup in place, I can expand on it to add more services, or simply deploy these changes to a set of production servers. My advice for you is to write up a set of scripts (your choice, mine is Ansible) so that all of this is automated.

This setup is far from complete for a production system. Each server needs to be locked down and secured from unwanted access. Also, each back-end web server should accept requests only from the load balancer, to keep out any prying eyes. These things are all left as exercises for the reader.