# Sunday, 27 May 2012

The announcement that Facebook intended to acquire Instagram made headlines. While the focus of these headlines primarily centered around the $1B valuation and 50 Million user audience, the true story went largely untold.

Instagram's story marks an inflection point in computing history whereby a start-up embracing cloud infrastructure is no longer the exception, but now considered the rule for building a successful company.

Consider the fundamentals of their meteoric rise over 2 years:
  • 2 people start with an idea
  • Create an iPhone application with hosting on Amazon Web Services (AWS)
  • Distribute the app on Apple's app store
  • Grow an audience
  • Evolve and enhance the product
  • Leverage elastic scalability on AWS
The costs for a v1 product were extraordinarily low and gave them an opportunity to pivot at any point without being grounded by capitalized infrastructure investments (i.e. building their own colo servers and doing their own hosting).

At the time of Instagram's acquisition they had added only 11 more employees, mostly hired in the few months leading up to their acquisition.

Some tips and lessons learned for other start-ups considering embracing the cloud as their development and hosting platform:

Choose a platform and master it: Instagram selected Amazon Web Services, but there are many options available. Don't hedge your bets by designing for platform portability. Dig deep into the capabilities of your chosen platform and exploit them.

Think Ephemeral: Your cloud web servers and storage can disappear at any point. Anticipate and design for this fact. If you have a client server background, then take a step back and grasp the concepts of ephemeral storage and computing before applying old world concepts to the cloud.

Share Knowledge: This is still a new frontier and we're all constantly learning new tips and tricks about how to best utilize cloud computing. Many people at Facebook (including myself) were fans of the Instagram Engineering blog long before the acquisition. Share what you've learned and others will reciprocate.

Build for the long term: The Instagram team did not build a company to be sold. They built a company that could have continued to grow indefinitely, and perhaps even overtaken competing services. There are so many legacy business processes and consumer applications whose growth is restricted by their architecture. Embrace the cloud with the intent to build something enduring and everlasting.

Sunday, 27 May 2012 10:26:04 (Pacific Daylight Time, UTC-07:00)
# Sunday, 05 December 2010

Update: About one year after writing this article I switched to hosting node.js apps on Heroku. Check it out.

Node.js is an impressively fast and lightweight web server based on the Google V8 Javascript engine. I really enjoy working with node.js for the simple elegance of language parity between client and server. The use of server-side Javascript also means taking advantage of common JS patterns, such as event-driven programming and closures (there's just something reassuring and pure about functional programming on the server that gives me a greater sense of confidence when there are no state dependencies between expressions).

In the spirit of the CloudStock event tomorrow, I set out to install Node.js on an Amazon EC2 instance. Amazon is running a promotion on free EC2 micro instances, otherwise micros can be leased for about $0.02 per hour or ~$20 per month.

Step 1

Signup for Amazon EC2 and click on "Launch Instance" to get started.

Step 2

I prefer the default Amazon Linux machine image, but any Linux distribution should work. The default Linux AMI is stripped down and secure out of the box.

Step 3

Select the type of instance. I recommend starting with a Micro for creating a simple Node sandbox.

Step 4

Accept the default advanced instance options by clicking "Continue"

Step 5

Give your instance a name, such as "Node Sandbox".

Step 6

This is the most critical part of the instance provisioning process. If a key pair has not already been defined in EC2, create one by entering a key name then downloading the resulting *.pem file to the local desktop.

Step 7

If this is your first time at Amazon, you'll need to generate a firewall profile for use by the Node Sandbox instance. Allow SSH (port 22) and HTTP (port 80).
We'll initially be hosting Node on port 8080, which unfortunately is not configurable in the instance request wizard. Make a mental note that we'll be coming back to security groups in step 10 to allow port 8080.

Step 8

Review the request and press "Launch" to fire up your Linux virtual machine. Amazon says it could take several minutes to provision the VM. In my experience, the micro instances have only taken seconds.

Step 9

Confirm the new instance is running. Copy the "Public DNS" URL of the instance into a text editor; you'll be using it frequently in the next steps. (Note: Make sure to copy the Public DNS and not the Private DNS, which is only used for internal EC2 connections).

Step 10

Select the instance then click on Security Groups. Modify the group to allow tcp traffic over port 8080. That's it for EC2 configuration.

Step 11

SSH. The remaining steps all require SSH access to the newly provisioned EC2 instance. This article uses the bash terminal on Apple OSX for demonstration.

Remote access for root user is not enabled. Instead, the user "ec2-user" is made available with sudo permissions. Login via SSH using syntax:

ssh -i keyFilePath/keyFile.pem ec2-user@ec2-public-dns

Switch i (-i) authenticates using the identity of the key file created in step 6.
keyFilePath is the path to the key file generated in step 6.
ec2-public-dns is the domain name for the EC2 instance retrieved in step 9.

Note: You may receive the following error when attempting to SSH to EC2.

Amazon requires the keyfile to be truly private, therefore only readable by you and no other user on the local machine. To fix the issue, change the file mode with:
chmod 700 keyFileName.pem

A successful login will present the following prompt.

Step 12  Download/Copy Node.JS.

Download Node.js to your local file system. In this example, I've downloaded Stable: 2010.11.16 node-v0.2.5.tar.gz

Open a local shell window and copy the package to the EC2 instance using secure copy.
scp -p -i ../keyPath/keyFile.pem node-v0.2.5.tar.gz ec2-user@ec2-204-236-155-210.us-west-1.compute.amazonaws.com:node.tar.gz

Step 13 Extract

The scp copy should copy node to the home/ec2-user directory. You can extract node and configure/install from this directory. The following steps assume extraction to the root opt directory so the following commands are all executed in the "Super User" sudo context to override the permission errors you'll otherwise encounter while logged in as the ec2-user user.

Copy to opt
sudo cp node.tar.gz ../../opt
cd ../../opt
sudo gunzip -d node.tar.gz
sudo tar -xf node.tar

Change to the node install directory. Listing the contents will display the following:

[ec2-user@ip-xxx-yyy-zzz node-v0.2.5]$ ll
total 112
-rw-r--r-- 1 1000 1000  4674 Nov 17 05:46 AUTHORS
drwxr-xr-x 3 1000 1000  4096 Nov 17 05:46 benchmark
drwxr-xr-x 2 1000 1000  4096 Nov 17 05:46 bin
-rw-r--r-- 1 1000 1000 31504 Nov 17 05:46 ChangeLog
-rwxr-xr-x 1 1000 1000   387 Nov 17 05:46 configure
drwxr-xr-x 7 1000 1000  4096 Nov 17 05:46 deps
drwxr-xr-x 2 1000 1000  4096 Nov 17 05:47 doc
drwxr-xr-x 2 1000 1000  4096 Nov 17 05:46 lib
-rw-r--r-- 1 1000 1000  2861 Nov 17 05:46 LICENSE
-rw-r--r-- 1 1000 1000  2218 Nov 17 05:46 Makefile
-rw-r--r-- 1 1000 1000   413 Nov 17 05:46 README
drwxr-xr-x 2 1000 1000  4096 Nov 17 05:46 src
drwxr-xr-x 8 1000 1000  4096 Nov 17 05:46 test
-rw-r--r-- 1 1000 1000  1027 Nov 17 05:46 TODO
drwxr-xr-x 5 1000 1000  4096 Nov 17 05:46 tools
-rw-r--r-- 1 1000 1000 19952 Nov 17 05:46 wscript

Step 14 Configure and Install

The next step is to configure the node environment. Typing the following will result in a dependency error
sudo ./configure

/opt/node-v0.2.5/wscript:138: error: could not configure a cxx compiler!

To fix this error, we need only to install the GCC compiler from the YUM repository hosted by Amazon

sudo yum install gcc-c++

We're not going to be installing any SSL certificates in this sandbox, so run config without support for Open SSL

sudo ./configure --without-ssl

Now you should be able to make the Node.js package

sudo make

At this point I recommend getting up and going for a walk, making a sandwich, or anything else that will kill up to 10 minutes. The small and micro instances on EC2 have limited access to CPU computing resources, making this part of the install process lengthy.

Once the make is complete, the final step is

sudo make install

You can optionally run sudo make test to verify the install.

Step 15 Hello World

Verify the installation is working by creating a file named example.js with the following contents.

var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
console.log('Server running at http://ec2-204-236-155-210.us-west-1.compute.amazonaws.com:8080');

Then run node from the command line:

node example.js

Open a browser to confirm the domain is accessible from port 8080. That's it!

Because node is running as a shell process you may want to launch the server using the "no hangup" util (or 'forever') to ensure node runs beyond the shell session.

nohup node example.js &

There's huge potential for creating scalable cloud services when combining Amazon Web Services and node.js. Enjoy!

Sunday, 05 December 2010 12:05:57 (Pacific Standard Time, UTC-08:00)
# Sunday, 13 December 2009
Update Jan 2012: This article is way out of date. I've since moved to a MacPro for development, using VMWare Fusion to run some Windows apps.

What would be your ultimate machine for developing applications in the cloud?

I've been mulling over this question for a few weeks and finally got around to putting a solution together over a weekend. There are both hardware and software components:

  • Something in between a netbook and laptop, around 4 pounds.
  • 8+ hour battery life
  • 1" thin (Easy to tote)
  • SSD (Solid State Disk 256 GB+)
  • 4 GB RAM
  • 2 GHz CPU+
  • ~$700
I settled on a new Acer 4810 Timeline which met most of my requirements. The exceptions being the Acer has a 1.3 GHz CPU and doesn't have SSD. I wanted the SSD primarily for a speedy boot time, but some tuning of the Win7 sleep options along with the Acer's 8+ hour battery life means the laptop is rarely ever turned off and can quickly recover from sleep mode.


Next up was the software required to fully embrace the cloud. My list of essential cloud tools is no where near as prolific as Scott Hanselman's tool list. But hey, this is a nascent craft and we're just getting started. These tools are essential, in my opinion, for doing development on the leading cloud platforms.

Windows 7
Whoa! I can already hear my Mac toting friends clicking away from this blog post. "WTF? Why Windows 7 for a cloud development machine?" Well, there are several reasons:
  • First of all, even if you do develop on a Mac you likely have a Windows VM or dual boot configured for Windows anyway.
  • Windows has been running on x86 architecture for years. Mac only made the jump a few years ago and is still playing catch up on peripheral device and driver support.
  • Even Google, a huge cloud player, consistently will develop, test, and release all versions of Chrome and Google App Engine tools on Windows before any other platform. Windows developers typically get access to these tools months in advance of Mac users.
  • Eclipse is another tool that is well supported on Windows, above all other platforms.
  • Silverlight support. This is my RIA environment of choice going forward.

Visual Studio 2010 Beta 2
By the time you read this blog, perhaps this version of Visual Studio will be outdated, but it is the first release of VS designed with complete support for Azure.

Windows Azure
You'll need an Azure account to upload your application to the cloud for hosting.

Azure SDK
Great article here for getting started on installing/configuring your machine for the latest Azure bits

Azure Storage Explorer
Azure Storage Explorer is a useful GUI tool for inspecting and altering the data in your Azure cloud storage projects including the logs of your cloud-hosted applications.

Eclipse is a Java-based IDE that requires the Java runtime

Rather than downloading the latest and greatest version of Eclipse, I recommend downloading whatever version is currently supported by the next essential cloud development tool (below)...

Force IDE
Eclipse Plug-in that supports development and management of Apex/Visualforce code on Salesforce.com's platform (aka Force.com)

Google App Engine
Currently, there are both Python and Java development environments for GAE. Like Azure, GAE supports development and debugging on localhost before uploading to the cloud, so the installation package provides a nice local cloud emulation environment.

I have a need for both the Windows Explorer Tortoise shell plug-in and the Eclipse plug-in. You may need only one or the other. All the Force.com open source projects are accessible via SVN.

Git on Windows (msysgit)
It seems gitHub is becoming the defacto standard for managing the source code for many web frameworks and projects. Excellent article here on how to install and configure Git on Windows

Amazon Web Services (AWS) Elastic Fox
Nice Firefox plug-in for managing EC2 and S3 instances on AWS. I mostly just use it for setting up temporary RDP whitelist rules on EC2 instances as I connect remotely from untrusted IP addresses (like airports/hotels/conferences).

I highly recommend using the browser based AWS console for all other provisioning and instance management. There's also an AWS management app for Android users called decaf.

If you're doing real world web development, then you already know the drill. Download and install Internet Explorer 8, Firefox, and Chrome at minimum. In addition to the Elastic Fox plug-in (above) you'll want to install Firebug. IE 8 now has Firebug-like functionality built-in, called the Developer Toolbar. Just hit F12 to access it (see comparison with Firebug here).

I personally use JQuery for all web development that requires DOM manipulation to handle cross-browser incompatibilities and anomalies.

There are 6-10 other utilities and tools I installed on this machine that aren't specific to the cloud. Installing everything on this list took about 90 minutes (plus a couple hours to pull down all my SVN project folders for Passage and other related projects I manage).

Given that cloud development is all about distributing resources over servers and clients, I like to take a minimalist approach and reduce the surface area and drag of my local environment as much as possible. This improves OS and IDE boot time as well as eliminates a lot of common debugging issues as a result of version incompatibilities.

What about you? What hardware/tools/applications are essential to your cloud development projects?

Sunday, 13 December 2009 12:01:04 (Pacific Standard Time, UTC-08:00)