The following information is now out of date. For current information on how to deploy to AWS using Cloud Foundry BOSH, please refer to the documentation at http://docs.cloudfoundry.org/deploying/aws/index.html.

 

Cloud Foundry was designed and built to support distributed applications that can be moved between multiple clouds, including those running on different IaaS infrastructures such as vSphere, vCloud, OpenStack and Amazon Web Services. The key to supporting multiple clouds is BOSH, a cloud deployment and lifecycle management tool that was announced earlier this year.

This blog post will walk you through the steps to start using BOSH on AWS and then show you how to deploy a sample three-tier application.

BOSH

To start using BOSH, you will need to bootstrap your environment. The Cloud Foundry engineers created BOSH to deploy and manage distributed applications, such as Cloud Foundry, and we wanted to be able to start from scratch and create a working environment that is manageable and upgradable in a controlled manner. BOSH is a distributed application, so it’s only natural that we use BOSH to bootstrap and deploy BOSH itself. For this, we have a notion of a BOSH kernel, a single system running all the BOSH components called micro BOSH. While this is sufficient to manage simple applications, a full BOSH environment would be required for a production grade system. But for the sake of demonstrating BOSH on AWS, we will deploy a sample application directly from the micro BOSH instance.

Prerequisites

To get started with BOSH on AWS you need three things:

1. an AWS account
2. AWS credentials, i.e. access_key_id and secret_access_key
3. a Mac or *NIX computer

Installing the BOSH Deployer

We assume you already have Ruby (1.9.2) and rubygems (1.8) installed. To install the BOSH deployer gem (which includes the BOSH cli):

gem install bosh_deployer

Micro BOSH AMIs

We have published micro BOSH AMIs in each AWS region. When you are ready to use the BOSH deployer, you will use the AMI that corresponds the EC2 region you are using from the table below.

Region	AMI
ap-northeast-1	ami-7656eb77
ap-southeast-1	ami-64d59436
eu-west-1	ami-874c4af3
sa-east-1	ami-6280597f
us-east-1	ami-69dd6900
us-west-1	ami-4f3e1a0a
us-west-2	ami-7ac7494a

Deploying Micro BOSH

The first thing we need to do is deploy a micro BOSH instance on AWS. Since you can’t set the IP of an AWS instance, you need to create an elastic IP to use with micro BOSH, as the instances it will deploy need to communicate with the micro BOSH instance and the AWS public IP will change.

Creating an Elastic IP

1. Go to the AWS Dashboard and look for the Elastic IP link in the upper right hand corner
2. On the Allocate New Address pop-up window keep the EC2 default and click “Allocate”

1. Now that you have one IP allocated, repeat the steps to create three more (one for micro BOSH and three for the WordPress demo application we are deploying).

Create the Directory Structure

The BOSH deployer will deploy applications based on files in expected directory locations:

mkdir ~/deployments
cd ~/deployments
mkdir aws

Create Micro BOSH Config File

Micro BOSH configurations are set in the micro_bosh.yml, which you need to create.

1. Create ~/deployments/aws/micro_bosh.yml using the template that appears below

   ---
   name: aws
   
   logging:
     level: DEBUG
   
   network:
     type: dynamic
     vip: x.x.x.x
   
   resources:
     persistent_disk: 20000
     cloud_properties:
       instance_type: m1.small
       availability_zone: sa-east-1a
   
   cloud:
     plugin: aws
     properties:
       aws:
         access_key_id: AKIAIYJWVDUP4KRWBESQ
         secret_access_key: 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
         default_key_name: bosh
         default_security_groups: ["bosh"]
         ec2_private_key: ~/.ssh/bosh
         ec2_endpoint: ec2.sa-east-1.amazonaws.com
   
   apply_spec:
     agent:
       blobstore:
         address: x.x.x.x
       nats:
         address: x.x.x.x
     properties:
       aws_registry:
         address: x.x.x.x

2. Update all instances of x.x.x.x with one of the elastic IPs you created and the region you are deploying to. Add your AWS credentials under the aws section.
3. Update the availability zone and endpoint settings to match your availability zone. For instance, if you deploy to us-east-1 you should set the availability zone to either us-east-1a or us-east-1b.
4. Save the file

Create a Keypair

If you already have a Key Pair for EC2 you can skip this section.

1. Go to the AWS Dashboard and click Key Pair
2. Click on the Create Key Pair button
3. Name the Key Pair “bosh”
4. Your browser should automatically download a private key named bosh.pem
5. Rename the private key file to bosh and save it in your ~/.ssh directory

Create a Security Group

1. In the AWS Dashboard, create a security group named bosh
2. Add the following inbound rules from 0.0.0.0/0or more restrictive if you like:
   1. port 25555 (BOSH director REST API)
   2. port 6868 (BOSH agent HTTP interface)
   3. port 22 (ssh) for troubleshooting and poking around in the micro BOSH instance
3. Click “Apply Rule Changes” to make the rules take effect

Deploying Micro BOSH

Micro BOSH can now be deployed from your deployments directory.

1. Make sure you are in your deployments directory:

   cd ~/deployments

2. Select the deployment you created:

   bosh micro deployment aws

   Note: don’t be concerned by seemingly inaccurate message WARNING! Your target has been changed to `http://aws:25555!

3. Start the deployment using the AMI from the BOSH AMIs section above:

   bosh micro deploy ami-xxxxxxxx

4. Within 20 minutes your instance of micro BOSH will be deployed. After the ‘Done’ message appears, you have a running micro BOSH instance.

   Done             11/11 00:07:10
   WARNING! Your target has been changed to `http://x.x.x.x.:25555'!
   Deployment set to '/Users/bosh/deployments/aws/micro_bosh.yml'
   Deployed `aws/micro_bosh.yml' to `http://aws:25555', took 00:07:10 to complete

5. If your deployment failed for some reason use the following to clean up:

   bosh micro delete

6. Log in to the Micro BOSH:

   bosh login

7. Type the default account name is admin and the password is admin
8. Change the account name and password using the command below. Don’t say we didn’t tell you if someone deletes your deployment!

   bosh create user <username> <password>

Deploying an Application Using BOSH

We have created a sample three-tier application (Nginx, Apache + PHP with WordPress, and MySQL) to demonstrate how you can use BOSH, and the next step is to deploy it using your newly created micro BOSH instance.

Uploading the Sample Release

The sample release is on Github for your cloning convenience:

1. First make a git clone of the sample application release repository:

   cd ~
   git clone git://github.com/cloudfoundry/bosh-sample-release.git
   cd bosh-sample-release

2. Upload the release to micro BOSH:

   bosh upload release releases/wordpress-1.yml

Uploading the Latest Stem Cell

Now we download the latest stem cell to upload to our micro BOSH instance.

1. Download the latest BOSH stem cell for AWS:

   bosh download public stemcell bosh-stemcell-aws-0.6.4.tgz

2. Upload it to your micro BOSH instance:

   bosh upload stemcell bosh-stemcell-aws-0.6.4.tgz

Create a Deployment Manifest

1. Create a security group called “open” which allows inter-security group communication, port 80 from anywhere so the web server can reached from the Internet, and port 22 from anywhere if you want to be able to log on using ssh.
2. Get the director UUID using the following command:

   bosh status

3. Make a copy of the wordpress-aws.yml in the bosh-same-releasedirectory and update the sections in red.

   ---
   name: wordpress
   director_uuid: 718f5344-c8e4-4266-ab10-3a75e1638e25
   
   release:
     name: wordpress
     version: latest
   
   compilation:
     workers: 3 # only the required number are provisioned
     network: default
     reuse_compilation_vms: true
     cloud_properties:
       instance_type: m1.small
   
   # this section describes how updates are handled
   update:
     canaries: 1
     canary_watch_time: 3000-120000
     update_watch_time: 3000-120000
     max_in_flight: 1
     max_errors: 1
   
   networks:
     - name: elastic
       type: vip
       cloud_properties: {}
     - name: default
       type: dynamic
       cloud_properties:
         security_groups:
         - open
   
   resource_pools:
     - name: common
       network: default
       size: 4
       stemcell:
         name: bosh-stemcell
         version: 0.6.4
       cloud_properties:
         instance_type: m1.small
         availability_zone:
         key_name:
   
   jobs:
     - name: mysql
       template: mysql
       instances: 1
       resource_pool: common
       networks:
         - name: default
           default: [dns, gateway]
         - name: elastic
           static_ips:
           - 107.21.118.59
     - name: nfs
       template: debian_nfs_server
       instances: 1
       resource_pool: common
       networks:
         - name: default
           default: [dns, gateway]
         - name: elastic
           static_ips:
           - 107.21.118.229
     - name: wordpress
       template: wordpress
       instances: 1
       resource_pool: common
       networks:
         - name: default
           default: [dns, gateway]
         - name: elastic
           static_ips:
           - 107.21.118.223
     - name: nginx
       template: nginx
       instances: 1
       resource_pool: common
       networks:
         - name: default
           default: [dns, gateway]
         - name: elastic
           static_ips:
           - 107.21.118.225
   
   properties:
     nginx:
       workers: 1
     wordpress:
       admin: foo@bar.com
       port: 8008
       servers:
         - 107.21.118.223
       servername: 107.21.118.225
       db:
         name: wp
         user: wordpress
         pass: w0rdpr3ss
       auth_key: random key
       secure_auth_key: random key
       logged_in_key: random key
       nonce_key: random key
       auth_salt: random key
       secure_auth_salt: random key
       logged_in_salt: random key
       nonce_salt: random key
     mysql:
       address: 107.21.118.59
       port: 3306
       password: rootpass
     nfs_server:
       address: 107.21.118.229
       network: 107.21.118.223/255.255.255.255
     debian_nfs_server:
       no_root_squash: true

Deploy

1. Select the deployment manifest you just created:

   bosh deployment ~/wordpress-aws.yml

2. Initiate the deployment:

   bosh deploy

3. Sit back and enjoy the show!

Conclusion

Congratulations. You just used BOSH to deploy an application to AWS! For now, a sample application is a good place to start. Soon we will post similar instructions for deploying Cloud Foundry. In the meantime, Dr Nic has written a number of getting started guides for BOSH which we highly recommend.

–Martin Englund, Cloud Foundry BOSH team