# Wednesday, 30 December 2009

Occam's Razor is the principle that "entities must not be multiplied beyond necessity" and the conclusion thereof, that the simplest explanation or strategy tends to be the best one. (See Wikipedia)

The simplest constructs for cloud computing are storage and processing. These 2 basic services are consumed by any number of Internet-connected clients, often described as the '3 screens' of PC, Mobile, and TV.

In Azure, the storage construct is sub-categorized into it's simplest components of Table, Blob (file), and Queue storage. Processing is sub-categorized as either a 'Web Role' or a 'Worker Role'.

Compositions may utilize these native constructs in any combination. A web role may work independently or utilize table storage. A client may directly access cloud storage or a processing service.

These 5 basic constructs can be arranged in 120 different combinations (5! factorial). When designed to be stateless, basic compositions can be scaled by adding additional storage or processing resources on-demand.

Wednesday, 30 December 2009 12:28:55 (Pacific Standard Time, UTC-08:00)
# Tuesday, 29 December 2009

Every year during the holiday break I look forward to digging into a fun software project and learning something new. This year I developed a simple application called "Up or Down" using the Microsoft Windows Azure cloud platform. Windows Azure represents a huge bottom-up R&D effort by Microsoft to support highly scalable Internet-connected applications across a number of devices and platforms.

(Jump to the 5 minute video walkthrough)
Download the source code from Codeplex.

Up or down is a simple service monitoring application that periodically polls a number of websites and reports their status as "Up" or "Down". This application has 2 basic roles: a web role for serving web pages and a worker role that runs in the background polling services. Both the web and worker roles have access to the same Azure tables storage service.

Project Structure
The web and worker roles each have their own separate project spaces. Because the Azure table storage is shared by both roles, a 3rd library project is created for representing the data model and is referenced by both roles.

This project was developed using Visual Studio 2010 Beta 2 with the November 2009 Azure SDK.

Azure Table Storage
2 tables are defined. One for storing information about websites to be monitored (ServiceRequestDefinition) and a table for storing ping responses (ServiceResponses).

The service definition table is very simple and contains just a few rows in this example. The partition key is based on the HTTP request verb used to monitor a service. Up or Down currently only makes HEAD requests for content, but can be expanded to do full page GET requests and conduct string matching on the response.

The response table is designed to be more scalable. Monitoring 10 websites once per minute will create 10 x 60 x 24 = 14,400 response records per day and over 5 million records per year. The partition key is in the format YYYY-MM for the fast retrieval of response records by month; for example "2009-12". The response table also captures the response time in milliseconds for each request.

User Interface
Up or Down is a simple one page application that communicates with the server via JQuery AJAX calls.

  • JQuery UI is used for the tabs and style sheet.
  • Click events are handled using JQuery.
  • The "Add New Service" tab is a simple web form for defining a service name and URL.
  • Each monitored service has a toolbar of options to view a report, edit, or delete a service.
  • Anonymous users can view dashboard status and drill down to recent response history
  • Support for simple authentication that requires a password to update/delete data.
  • The current status dashboard can be refreshed.
  • Report sparklines generated using the Google charting API

Data Access : Server side AJAX Handler
An ASMX webservice named db.asmx is included in the web role project for handling browser requests, CRUD operations against the Azure table storage, and formatting JSON responses back to the browser. Each C# model class has a ToJSON() method for serializing responses.

  • The web service declares the [System.Web.Script.Services.ScriptService] attribute to support calls from javascript on the browser.
  • Web service methods are declared with the [ScriptMethod(ResponseFormat = ResponseFormat.Json)] attribute to allow JSON to be passed back to the browser.
  • The open source JSON2 parse and stringify library is used by the browser client to serialize javascript objects to JSON for passing to db.asmx.
  • The Json.NET library is used server-side to format JSON responses from Azure tables.

The jQuery AJAX method asynchronously handles both the success and error responses from the server and in turn updates the user interface.

Configuration and Initialization
The polling frequency is externalized to the Service definition file. When the worker role starts, it looks for this configuration setting or otherwise using a default polling frequency of 60 seconds.

Evolution
The current state of this project is just functional enough to prove some basic concepts. Some likely subsequent features to be added:

  • Multi-tenant. Modify the partition keys to include an OrgId or ClientId to support multiple unique accounts
  • Profile and permissions model: Support multiple profiles with varying access permissions per account.
  • Subscription and Email notification. Notify profiles when sites go down.
  • Average response time: Use the history of response times to update performance metrics, such as average response time.
  • Additional reports: Add ability to query response history by date range.

 

5 Minute Video Walkthrough

This 5 minute video walks through some of the high level points of "Up or Down".

<
Tuesday, 29 December 2009 16:24:38 (Pacific Standard Time, UTC-08:00)
# Sunday, 13 December 2009

Well, almost everything ;-)

A basic iteration with Azure consists of:

  • Develop
  • Debug and Test
  • Deploy to Staging
  • Deploy to Production
Here it is compressed into 5 minutes.

Sunday, 13 December 2009 18:25:13 (Pacific Standard Time, UTC-08:00)
Update Jan 2012: This article is way out of date. I've since moved to a MacPro for development, using VMWare Fusion to run some Windows apps.

What would be your ultimate machine for developing applications in the cloud?

I've been mulling over this question for a few weeks and finally got around to putting a solution together over a weekend. There are both hardware and software components:

Hardware
  • Something in between a netbook and laptop, around 4 pounds.
  • 8+ hour battery life
  • 1" thin (Easy to tote)
  • SSD (Solid State Disk 256 GB+)
  • 4 GB RAM
  • 2 GHz CPU+
  • ~$700
I settled on a new Acer 4810 Timeline which met most of my requirements. The exceptions being the Acer has a 1.3 GHz CPU and doesn't have SSD. I wanted the SSD primarily for a speedy boot time, but some tuning of the Win7 sleep options along with the Acer's 8+ hour battery life means the laptop is rarely ever turned off and can quickly recover from sleep mode.

Software

Next up was the software required to fully embrace the cloud. My list of essential cloud tools is no where near as prolific as Scott Hanselman's tool list. But hey, this is a nascent craft and we're just getting started. These tools are essential, in my opinion, for doing development on the leading cloud platforms.

Windows 7
Whoa! I can already hear my Mac toting friends clicking away from this blog post. "WTF? Why Windows 7 for a cloud development machine?" Well, there are several reasons:
  • First of all, even if you do develop on a Mac you likely have a Windows VM or dual boot configured for Windows anyway.
  • Windows has been running on x86 architecture for years. Mac only made the jump a few years ago and is still playing catch up on peripheral device and driver support.
  • Even Google, a huge cloud player, consistently will develop, test, and release all versions of Chrome and Google App Engine tools on Windows before any other platform. Windows developers typically get access to these tools months in advance of Mac users.
  • Eclipse is another tool that is well supported on Windows, above all other platforms.
  • Silverlight support. This is my RIA environment of choice going forward.

Visual Studio 2010 Beta 2
By the time you read this blog, perhaps this version of Visual Studio will be outdated, but it is the first release of VS designed with complete support for Azure.

Windows Azure
You'll need an Azure account to upload your application to the cloud for hosting.

Azure SDK
Great article here for getting started on installing/configuring your machine for the latest Azure bits

Azure Storage Explorer
Azure Storage Explorer is a useful GUI tool for inspecting and altering the data in your Azure cloud storage projects including the logs of your cloud-hosted applications.

Eclipse 
Eclipse is a Java-based IDE that requires the Java runtime

Rather than downloading the latest and greatest version of Eclipse, I recommend downloading whatever version is currently supported by the next essential cloud development tool (below)...

Force IDE
Eclipse Plug-in that supports development and management of Apex/Visualforce code on Salesforce.com's platform (aka Force.com)

Google App Engine
Currently, there are both Python and Java development environments for GAE. Like Azure, GAE supports development and debugging on localhost before uploading to the cloud, so the installation package provides a nice local cloud emulation environment.

SVN
I have a need for both the Windows Explorer Tortoise shell plug-in and the Eclipse plug-in. You may need only one or the other. All the Force.com open source projects are accessible via SVN.

Git on Windows (msysgit)
It seems gitHub is becoming the defacto standard for managing the source code for many web frameworks and projects. Excellent article here on how to install and configure Git on Windows

Amazon Web Services (AWS) Elastic Fox
Nice Firefox plug-in for managing EC2 and S3 instances on AWS. I mostly just use it for setting up temporary RDP whitelist rules on EC2 instances as I connect remotely from untrusted IP addresses (like airports/hotels/conferences).

I highly recommend using the browser based AWS console for all other provisioning and instance management. There's also an AWS management app for Android users called decaf.

Browsers
If you're doing real world web development, then you already know the drill. Download and install Internet Explorer 8, Firefox, and Chrome at minimum. In addition to the Elastic Fox plug-in (above) you'll want to install Firebug. IE 8 now has Firebug-like functionality built-in, called the Developer Toolbar. Just hit F12 to access it (see comparison with Firebug here).

I personally use JQuery for all web development that requires DOM manipulation to handle cross-browser incompatibilities and anomalies.

There are 6-10 other utilities and tools I installed on this machine that aren't specific to the cloud. Installing everything on this list took about 90 minutes (plus a couple hours to pull down all my SVN project folders for Passage and other related projects I manage).

Given that cloud development is all about distributing resources over servers and clients, I like to take a minimalist approach and reduce the surface area and drag of my local environment as much as possible. This improves OS and IDE boot time as well as eliminates a lot of common debugging issues as a result of version incompatibilities.

What about you? What hardware/tools/applications are essential to your cloud development projects?

Sunday, 13 December 2009 12:01:04 (Pacific Standard Time, UTC-08:00)
# Sunday, 06 December 2009

Just a reminder that effective January 2010, you will start receiving a bill for Azure hosting, although the amount due for January's bill will be $0. This will provide you with a line item breakdown of how your application consumes hosting and storage resources and will help in capacity planning and forecasting exercises.

Promoting your CTP apps to production will apparently require some manual intervention, so there should be no possibility of suddenly getting billed for an idle app you haven't been using.

Actual billing for Azure doesn't start until February, so before the December holiday break, take advantage of the free month of hosting and signup here for your Azure account.

You can still continue to develop/run your Azure apps on a localhost Windows machine using free development tool (check out the complete list of system requirements here).

Sunday, 06 December 2009 21:40:20 (Pacific Standard Time, UTC-08:00)
# Saturday, 05 December 2009

Can't get any simpler than this (from Steve Marx's blog)

Saturday, 05 December 2009 17:11:42 (Pacific Standard Time, UTC-08:00)
# Friday, 27 November 2009
Microsoft announced at last weeks PDC that the project code-named Oslo is now SQL Server Modelling Services.

As any Salesforce.com Developer or Admin will tell you, the cloud has become synonymous with declarative database definitions and rapid application development.

Oslo, which originally started as a domain specific language modeling environment, will now apparently be positioned as a data and application modeling tool, complete with Visio-like drag and drop capabilities for developing Azure applications.

The disconnect between tools like Rational Rose and actual working object-oriented code has long been a problem. It'll be interesting to see if Oslo's models, err SQL Server Modelling Services, are co-dependent on actual code to ensure they stay in sync.

Friday, 27 November 2009 18:05:45 (Pacific Standard Time, UTC-08:00)

This video from the recent Microsoft developers conference provides an excellent overview of Azure. It's a little over an hour, but worth it.

Some takeaway points:

  • The Azure Table service that was introduced early on as a core platform feature, largely in response to Google's BigTable and Amazon's SimpleDB, no longer appears to be strategic to Azure.
  • SQL Azure lets Developers reuse existing SQL Server databases, stored procedures, and skills for storage in the cloud.
  • Applications are developed locally and pushed to the cloud.
  • Both staging and production environments are available for a 2 phase publishing workflow (nicely done!)
  • Azure instances run on virtual machines.
  • Provisioning is fairly low-level. Developers control how many CPU cores, RAM, and bandwidth resources. Pay for more as needed. Scale back when not needed.
  • Load balancing is built-in (although not clear to me how multiple SQL instances stay in sync)
  • Azure instances are language agnostic. Developers can run Ruby on Rails, PHP, or .NET apps. Basically anything that can run on IIS will run on Azure
  • Developer can choose from among several geographies when provisioning cloud apps.
  • All demo apps I've seen are running in the URL cloudapps.net. Not clear on how DNS fits in the mix, but apparently it's possible to have multiple domain bindings on a single VM for multi-tenant architecture
  • Copies exist of all data and apps for disaster recovery
  • Logging is done using existing native .NET APIs
  • It appears almost all APIs are accessible via PowerShell scripts. GUIs are starting to come online, demonstrating Microsoft's patience of getting the system level aspects done right before working on eye candy
Bottom line is that Azure is the real deal and is squarely looking to make up the ground lost to Amazon EC2. If you get a chance, I recommend checking out this guided tour of a next gen Azure container.

Friday, 27 November 2009 17:31:08 (Pacific Standard Time, UTC-08:00)
# Friday, 06 November 2009

So, an email arrives in my inbox today inviting me to try Windows Azure. I've tried Azure a couple times in the past with not much luck. I got the impression from reading some white papers that there was bottom-up innovation going on at the hardware and storage layer and that they had solved the scalability challenges first, but there wasn't much to show from the top-down (unfortunately, where first impressions are made).

But it is Microsoft, so if they're really serious about cloud computing, you got to at least sit up and listen. Here's my 5 minute unedited foray back into the world of Azure.

(Yes, I know the transparency of my video reveals management keys, yadda, yadda. It's a sandbox. I trust you...)

Friday, 06 November 2009 18:09:02 (Pacific Standard Time, UTC-08:00)