Cloud Computing – Guide 2 IT Managers

A Guide to Amazon Web Services for Corporate IT Managers

 

There are several compelling reasons to employ AWS, even for those IT managers that haven't considered the cloud yet.

by David Strom

Tags: Amazon Web Services, cloud computing, web services
February 16, 2010, 02:05 PM — 

Amazon may be the world's largest bookstore, but in the past eight years it has quietly built up a series of more than a dozen cloud-based computing services as part of its Amazon Web Services (AWS) product offerings. Some of the services are older and better known, such as renting online storage using Simple Storage Service (S3) or setting up virtual supercomputers to work on knotty CPU-intensive computations using the Elastic Compute Cloud (EC2). But there are others that are newer that serve important niches, such as the ability to stream videos using CloudFront, tie Amazon's resources with your own data center with its Virtual Private Cloud service, bring up database servers with SimpleDB and Relational Database Services, and the ability to automatically add or subtract computing resources as they are needed with its Auto Scale and Elastic Load Balancer features. (See the summary chart for the complete list of all AWS services and links to more information.)

AWS certainly isn't the only cloud-computing vendor around: Google has its App Engine, Microsoft's Live and Azure are both coming of age, and Rackspace is one example of a managed services provider that is offering more Web services. What makes AWS fascinating – and perhaps the gold standard of cloud computing -- is seeing how it continues to evolve and complement its core offerings with additional services that can be knitted together to provide a very robust offering for IT managers that previously haven't given the cloud much of a second thought. And while it is challenging to evaluate cloud services because they are always being tweaked, enhanced, and augmented, now is the time to take a closer look at what Amazon offers.

There are several compelling reasons to employ AWS, even for those IT managers that haven't considered the cloud yet.

• 
  
  First, they are easy to learn and setup, with a limited number of programming interfaces and controls. There is copious documentation, including a constellation of support services, reams of sample code, and discussion forums galore on each of their services. Should this not be sufficient, Amazon has two different premium support plans for $100 or $400 a month. The higher plan includes round-the-clock live phone support and one hour response times. Few cloud-based vendors have a community this rich and helpful.
• 
  
  Second, they are designed to scale with your demands, making them ideal for peak load projects or to deal with unexpected heavy demands that your in-house servers weren't designed to handle. And given the size of Amazon's data centers to support its own operations, they can scale upwards better than many other cloud vendors that have smaller footprints across the planet. For example, when the National Archives released thousands of pages of Hillary Clinton's schedule, the Washington Post created 200 EC2 server instances to process the images so that reporters and the public could search them. The project took a little more than a day and cost the Post about $150, happening as one Post developer said, "at the speed of breaking news."
  Amazon even has a bulk-loading service called Import/Export, where you mail them a physical hard drive that they then connect to their network temporarily and upload your data. For getting started with datasets of hundreds of gigabytes, this can be very quick and cheap to setup.
  Some of the applications deployed using AWS can get rather sophisticated. Rather than purchasing their own server hardware, scientific instrument supplier Varian was able to run a complex series of several week-long mathematical simulations in under a day using Amazon's CycleCloud. They were able to dynamically scale their processing up to execute the simulation, then shut down when calculations completed.
• 
  
  Third, they are built for rapid deployment and heavily rely on automation. Talk Market, an online shopping vendor, integrated payment transaction controls directly into their own interface using Amazon's Flexible Payment Service. When vendors sign up with them, they are connected with a quick, free credit card payment processor using an Amazon Payments Business account.
• 
  
  Fourth, they operate around the clock and in different data centers around the world, too, making them appealing to global businesses or those that want to be thought of that way. Amazon has three main data centers in California, Virginia and Ireland, but not all components of AWS are hosted in all three, and some, such as Cloud Front media streaming, are hosted elsewhere such as Hong Kong and Japan. This could be an issue for some IT managers, who want to drive to see their data center, and in some cases don't know exactly where their data lives in the cloud.
• 
  
  Fifth, they are reasonably priced, especially when compared with traditional outsourced or managed hosting providers. When the IT staff at the Indianapolis 500 car races needed extra bandwidth to stream videos of its races, they turned to AWS and saved more than half of their hosting bills. Much of this was due to the flexible nature of Amazon's servers and how they could automate their on-demand needs. Snap My Life, a photo social media site, was also able to halve their costs using a variety of AWS offerings.
  Eric Quanstrom, vice president, marketing and strategy with Sorenson Media in Carlsbad, Calif., has been using a variety of AWS, including EC2, CloudFront and S3. "We have a heritage of providing cloud services going back to 2000. We see video ripe for the cloud and chose AWS because of their flexibility, scalability and reliability. It would cost us a lot more if we went with someone else, and CloudFront's points of presence and cost/performance were excellent. It also helps that they are looking at the market the same way we are."
• 
  
  Finally, they are becoming the industry standard and their interfaces are or will be incorporated into a variety of third party providers. One example is Linux distro vendor Ubuntu. They have an Enterprise Cloud offering that makes use of the same AWS programming interfaces, making it easier for developers to port their cloud applications to a private server running Ubuntu inside your corporate data center. Another is coming from Racemi, which plans on having tools that can import VMware virtual machines into and out of AWS later this year.

What are the downsides? Certainly security and having possession of all of your computing resources, but that isn't specific to AWS. This is where having the Virtual Private Cloud VPN service can help, so that any cloud-based resources can sit behind the corporate firewall and other network protective devices. And pricing can be difficult to calculate too.

Each of AWS' component services has its own price list that breaks down into pieces for data that is stored on Amazon's servers and bandwidth consumed both inbound and outbound. The good news is that there are no monthly minimums and no long-term contracts. Bills are calculated monthly, and Amazon frequently has price changes, mostly – and refreshingly -- reducing their fees charged. There are different fees for hosting at its two domestic data centers and at its data center in Ireland. And if you are looking for bargains, SimpleDB is free to get started until June 30. All inbound data transfers to all AWS are also free until June 30, too.

To help figure all this out, Amazon has provided its own Web-based calculator here.

Amazon's Web Services Universe

Service and URL	Description	Newest and notable features
Elastic Compute Cloud (EC2)	On demand computing
Elastic MapReduce	Large scale number crunching	Job flow debugging
Simple Storage Service (S3)	Online storage	File versioning
Elastic Block Storage (EBS)	Persistent storage used with EC2 applications
Mechanical Turk	Send small tasks to humans to process
CloudFront	Streaming media servers
SimpleDB	Data indexing and querying	Free until June 30 for new customers
Relational Database Service (RDS)	MySQL servers in the sky
Fulfillment Web Service (FWS)	Use Amazon's warehouses to fulfil your own orders	Free but warehouse fees apply
Simple Queue Service (SQS)	Workflow and messaging applications
CloudWatch	Monitor all your AWS services
Virtual Private Cloud (VPC)	VPN connecting the AWS cloud with in-house servers
Elastic Load Balancing	Distributed load across EC2 instances for fault tolerance
Auto Scaling	EC2 scaling	Free and included with EC2
Flexible Payments Service (FPS)	Integrated payment processing
DevPay	Online billing
Import/Export	Mail your hardware to Amazon and have them transfer your data

References and Useful Links:

http://aws.amazon.com/startupchallenge/

Cloud Computing – Related Technology

Technologies Related to Cloud Computing

Cloud computing typically has characteristics of all these technologies:

1. Grid computing
2. Virtualization
3. Utility Computing
4. Autonomic Computing

Grid Computing

Grid Computing involves a network of computers that are utilized together to gain large supercomputing type computing resources. Using this network of computers large and complex computing operations can be performed. In grid computing these networks of computers may be present in different locations.

A famous Grid Computing project is http://folding.stanford.edu/. The project involves utilizing unused computing powers of thousands of computers to perform a complex scientific problem. The goal of the project is "to understand protein folding, misfolding, and related diseases".

Virtualization

Virtualization introduces a layer between Hardware and operating system. During the sixties mainframe started supporting many users using virtual machines. These virtual machines simulated behaviour of an operating system for each user. VMWare launched a product called VMware Workstation in 1999 that allows multiple operating systems to run on personal computers.

The virtualization forms the foundation of cloud technology. Using virtualization, users can access servers or storage without knowing specific server or storage details. The virtualization layer will execute user request for computing resources by accessing appropriate resources.

Typically server utilization in data centres can be as low as 10%. Virtualization can help in significantly improving server utilization.

Utility Computing

Utility Computing defines a "pay-per-use" model for using computing services. In utility computing, billing model of computing resources is similar to how utilities like electricity are traditionally billed. When we procure electricity from a vendor, the initial cost required is minimal. Based upon the usage of electricity, electricity companies bills the customer (typically monthly). In utility computing billing is done using a similar protocol.

Various billing models are being explored. A few common ones are:

1. Billing per user count. As an example if an organization of 100 people uses Google's gmail or Microsoft Live as their internal email system with email residing on servers in the cloud, Google/Microsoft may bill the organization on per user basis.
2. Billing per Gigabyte. If an organization is using Amazon to host their data on the cloud, Amazon may bill the organization on the disk space usage.
3. Billing per hour/day. As an example a user may pay for usage of virtual servers by time utilized in hours.

In reality pricing on cloud computing can be very complex. Utility computing helps in reducing initial investment. As the computing requirements for an individual or an organization changes, the billing changes accordingly, without incurring any additional cost. If the usage has reduced, then billing will also reduce accordingly.

Autonomic Computing

Autonomic computing is an initiative started by IBM in 2001. Autonomic means "self-managing" computers. In Autonomic computing, computers can automatically correct themselves without human intervention. As an example consider is a network of computers running a set of programs. When there is a hardware failure on one of the computers on the network, the programs running on that computer are "transferred" to other computers in the network. This is an example of "self-correction" or autonomic computing. The analogy typically used is that of human biological systems. Our biological systems take action in self-correcting mode without our explicit knowledge. In the same way the goal of autonomic computing is for computing infrastructure to self-correct itself in unforeseen situations.

Cloud Computing - Types

Types of Cloud Computing

Cloud computing is typically classified in two ways:

1. Location of the cloud computing
2. Type of services offered

Location of the Cloud Computing

Cloud computing is typically classified in the following three ways:

1. 
   
   Public cloud: In Public cloud the computing infrastructure is hosted by the cloud vendor at the vendor's premises. The customer has no visibility and control over where the computing infrastructure is hosted. The computing infrastructure is shared between any organizations.
2. 
   
   Private cloud: The computing infrastructure is dedicated to a particular organization and not shared with other organizations. Some experts consider that private clouds are not real examples of cloud computing. Private clouds are more expensive and more secure when compared to public clouds.
   Private clouds are of two types: On-premise private clouds and externally hosted private clouds. Externally hosted private clouds are also exclusively used by one organization, but are hosted by a third party specializing in cloud infrastructure. Externally hosted private clouds are cheaper than On-premise private clouds.
3. 
   
   Hybrid cloud Organizations may host critical applications on private clouds and applications with relatively less security concerns on the public cloud. The usage of both private and public clouds together is called hybrid cloud. A related term is Cloud Bursting. In Cloud bursting organization use their own computing infrastructure for normal usage, but access the cloud for high/peak load requirements. This ensures that a sudden increase in computing requirement is handled gracefully.
4. 
   
   Community cloud involves sharing of computing infrastructure in between organizations of the same community. For example all Government organizations within the state of California may share computing infrastructure on the cloud to manage data related to citizens residing in California.

Classification based on Service Provided

Based upon the services offered, clouds are classified in the following ways:

1. 
   
   Infrastructure as a service (IaaS) involves offering hardware related services using the principles of cloud computing. These could include some kind of storage services (database or disk storage) or virtual servers. Leading vendors that provide Infrastructure as a service are Amazon EC2, Amazon S3, Rackspace Cloud Servers and Flexiscale.
2. 
   
   Platform as a Service (PaaS) involves offering a development platform on the cloud. Platforms provided by different vendors are typically not compatible. Typical players in PaaS are Google's Application Engine, Microsofts Azure, Salesforce.com's force.com .
3. 
   
   Software as a service (SaaS) includes a complete software offering on the cloud. Users can access a software application hosted by the cloud vendor on pay-per-use basis. This is a well-established sector. The pioneer in this field has been Salesforce.coms offering in the online Customer Relationship Management (CRM) space. Other examples are online email providers like Googles gmail and Microsofts hotmail, Google docs and Microsofts online version of office called BPOS (Business Productivity Online Standard Suite).

The above classification is well accepted in the industry. David Linthicum describes a more granular classification on the basis of service provided. These are listed below:

1. Storage-as-a-service
2. Database-as-a-service
3. Information-as-a-service
4. Process-as-a-service
5. Application-as-a-service
6. Platform-as-a-service
7. Integration-as-a-service
8. Security-as-a-service
9. Management/Governance-as-a-service
10. Testing-as-a-service
11. Infrastructure-as-a-service

Cloud Computing

Cloud Computing

Cloud Computing makes computer infrastructure and services available "on-need" basis. The computing infrastructure could include hard disk, development platform, database, computing power or complete software applications. To access these resources from the cloud vendors, organizations do not need to make any large scale capital expenditures. Organization need to "pay per use" i.e. organization need to pay only as much for the computing infrastructure as they use. The billing model of cloud computing is similar to the electricity payment that we do on the basis of usage. In the description below vendor is used for cloud computing service provide and organization is used for user of cloud computing services.

Cloud Computing has the following characteristics:

1. 
   
   Availability of large computing infrastructure on need basis: Cloud vendors provide appearance of infinite computing infrastructure availability. This is available to organizations on need basis. This ensures that organizations do not need to set up servers for their peak requirements. As an example consider the official Wimbledon site. The site gets extremely high traffic in the two weeks when the championship happens. For this two weeks period this site will have high server usage. For rest of the year the site will need to only pay for the reduced usage. In general organizations do not need to bear the cost of computing infrastructure for their peak loads. The usage of computing resources can be increased or reduced on need basis, is called elastic computing.
2. 
   
   Cloud computing uses a "pay-per-use" billing model. Cloud billing model are very different when compared to traditional IT billing techniques. Typical billing models include per user billing, per GB billing or per-use billing (i.e. an organization is billed on each usage of the computing service).
3. 
   
   Cloud computing typically does not involve long-term commitment to use the computing infrastructure. The vendor does not enforce long-term usage of services.
4. 
   
   Cloud computing does not involve any significant capital expenditure for the organization. Unlike traditional IT infrastructure, in cloud computing organizations just use the computing services without procuring it. In some sense cloud computing involves renting the computing resources instead of buying them. As the figure below displays, unlike traditional computing model, Cloud computing requires no capital expenditure to acquire initial computing resources. The figure below is from cloud computing wiki.
   
   
5. 
   
   Since the cloud computing vendor provides services over the web, these are available from any location.
6. 
   
   Cloud computing can be ordered online without detailed formal contracts.

Cloud computing provides a level-playing field for smaller organizations. It allows smaller organization access to computing infrastructure without making any significant initial investment.
Many experts believe that cloud computing will lead to increased commoditization of computing resources.

Origin of term

Cloud traditionally depicts the internet. Since cloud computing is built using internet, hence the name.

Examples of Cloud Computing Applications

Hotmail.com was launched in 1996, It is widely considered as the first cloud computing application. The data is stored at the vendor servers, and users could pay incrementally to increase disk space usage. Many other services have emerged in the last decade that allows users to store information (or perform processing) without paying any upfront charges. These are typically consumer oriented services. Twitter, MySpace, Wikipedia, YouTube, Facebook, LinkedIn, Google docs and blogger all have the characteristics explained above and are examples of cloud computing.

Salesforce.com, founded in 1999, was the first successful example of providing software as a service in the business to business domain. Salesforce is a CRM tool for sales executives providing features like managing customer details, running promotions etc.

Google and Microsoft provide development platforms that can be accessed with "pay-per-use" billing model. All these services are examples of Cloud computing.

Amazon.com was one of the first vendors to provide storage space and computing resources following the cloud computing model.