Today at ZDNET Mary Jo Foley wrote an interesting piece entitled:
 

Microsoft, HP to unveil 'solutions built on new infrastructure-to-application model'

Seriously,

Microsoft is once again late to the game.

So, they are attempting to make yet another 'empty' promise for future services that don't exist today.

Microsoft would like nothing better than to have CFOs freeze their Cloud development initiatives and wait for them to catch up.


Amazon Web Services is continuing its expansion and has a total of over three years experience in Cloud Infrastructure, Operating System as a service with 'on demand' pay for only the resources you need being its main attractor.

Don't limit yourself to a Microsoft-only proprietary Cloud.

AWS is agnostic and gives you flexibility to choose which O/S to use, including Microsoft Windows Server 2008, without any license, if you so choose.

Amazon Web Services is here today.

Mary Jo was kind enough to reply to my Talkback with:

Hi. I agree with you that Microsoft is chasing Amazon in the cloud.

As an Amazon partner, I'm curious whether you agree or have anything to say about a recent blog post claiming Amazon is hitting a scalability wall. Do you think it's true?

Here it is: http://alan.blog-city.com/has_amazon_ec2_become_over_subscribed.htm

MJ

To which I replied:

There are two sides to a coin, and an edge.

The author of the above-referenced article, fails to mention whether they contacted Amazon, and if so, were AWS technical support able to corroborate any of their issues and/or what remediation may have taken place in the AWS data center to rectify their latency issues.

Image via CrunchBase

Also, it should be noted latency, by itself, does not imply a scalability issue.

For example, if a web app user spawned a thread for a 'poorly-optimized' back-end database SQL query, that query might be contributory to latency seen in an app, if the query doesn't return its results in a timely fashion.

Moreover, SQL correlated queries are preferred over nested dynamic queries and if developers of the web app's database were diligent in testing their SQLs (e.g., using Explain) they'd have readily seen where their DB bottlenecks are.

Something as simple as identifying a slow-running query and observing that it is running a 'brute-force' sequential table scan vs indexed read can be remedied with the creation of a new primary key index on the column or columns of the database which are part of a SQL select/insert/delete query statement.

It is stuff like the above that has hit concerns like Twitter. The tools they were using were first suspect (Ruby on Rails), but it was really a matter of their coming to terms with their own programming and scalability issues that ultimately allowed them to become the success they are today. Their scalability issues were not due to the inability of RoR to scale, it was only that they had to go through their own 'learning curve' on how best to configure their servers for peak demand usage that was at issue.

No fault of the tools/resources--just a 'learning experience'.

So, while I won't rule out the possibility that AWS may have intermittent 'growing pains', I am always concerned when only one side of a story is given in a blog.

To be fair, the author could have given Amazon the opportunity to provide a response which could have been included in the story.

What do you think?
 

Related articles by Zemanta

• EngineYard - Cloud Based Hosting Platform for Ruby on Rails Web Apps (gajeebo.com)
• Winners And Losers of 2009 - III (cloudave.com)
• MegaRoundup: Windows Unveils Azure Pricing (datacenterknowledge.com)

Mary Jo Foley of ZDNET's 'All About Microsoft' blog writes today an article entitled:

Is VMware the real motivator behind latest Microsoft-HP deal?

I am inclined to believe that Red Hat will factor heavily this year and following more than any other player into the growing market for server consolidation using virtual machine technology.

Keep in mind that every copy of the Linux kernel now includes Kernel Virtual Machine for free.

KVM is a Type 1 hypervisor and has performance characteristics as good as or better than VMware ESX.

Couple that to Red Hat's open sourcing SPICE and you can readily see they have positioned themselves to meet the demand for server consolidation, particularly in a recession when the pressure is on IT to find any way possible to curb or even better 'reduce' software and capital equipment expenditure.

One only need watch the video benchmarks here that compare Citrix ICA, RDP, and SPICE to get a feel for the level of competition coming from open source for Virtual Machine/Thin Client software.

That said, 2010 will be another GREAT year for Red Hat.

What do you think?

Related articles by Zemanta

• Red Hat takes on VMware with server virtualization solution (arstechnica.com)
• Red Hat debuts virtualization management (news.cnet.com)
• Red Hat promotes cloud projects, pans Microsoft and VMware (infoworld.com)

Press Release

Related articles by Zemanta

• Alfresco Wants Enterprise Content Management in the Cloud (readwriteweb.com)
• Is there a best CMS tool for your industry? (cmswatch.com)
• Amazon EC2 Shows Amazing Growth (cloudave.com)

September 17, 2009 | By Charlie Schluting

While some say the cloud doesn't perform well enough and isn't stable enough, others say it is perfect for their usage model. Why not take advantage of such a scalable architecture? It can make a great place to "stuff" data, such as backups.

In this article we use Amazon EC2 (the computing cloud) and S3 (the storage) as the example cloud provider, but any similar cloud service can be used the same way. There are many ways to backup your data to the cloud, including:

• simple file copies onto the remote OS
• remote API calls to send data into the cloud
• normal block-based replication onto volumes in your cloud OS

File Copies

Running a Linux image in the cloud means you have access to implement whatever type of backup strategy you like. The most simple, brute force method, is to create a volume of a few hundred gigabytes, and simply copy files to it. You can also copy disk images, database backup dumps, and anything else that falls outside the traditional category of "file copy."

Many backup programs, especially open source ones, support disk-based backups. You can simply point a backup program at a remote volume with SSH access, and it will treat it as its backup volume. If you're implementing a quick-and-dirty backup solution for the first time, tools like rsync are wonderful. Simply rsync a directory every hour via a cron job, and it will copy only the data that has changed. Many forms of rsync backup scripts exist, posted to various Internet forums and newsgroups, ranging from simple to extremely complex.

Cloud Storage APIs

In addition to simply copying data into your cloud OS, you can also leverage the S3 storage grid directly. API calls will allow you to store and retrieve buckets of data, but that doesn't mean you need to spend time programming a solution.

Included in your EC2 OS image are two scripts: ec2-bundle-vol and ec2-upload-bundle. They can be run in sequence to upload a volume on your EC2 instance into S3 storage. If you're using the EC2 OS to copy in large amounts of backup data, as in the previous example, most likely you will eventually want to ship some of that data onto S3 storage directly. It is less expensive, and also easier to manage.

Aside from running scripts to send huge chunks of data to S3, which quickly becomes a scheduling and management nightmare, you can also just point backup software at your S3 account. Instead of disk-based backups in EC2, it often makes more sense to simply talk straight to S3. For example, the open source backup product, Amanda, supports S3 as a backup device in addition to the traditional disk and tape mediums.

Real-Time Replication With DRBD

Before we get started, let us first point out that replication is not backup. With careful planning, however, you can have easy access to an up-to-date copy of data, and lessen the need for certain types of backup.

Block-level replication can be thought of as RAID-1 over TCP. In Linux, the standard software that implements this type of real-time replication is called DRBD, which stands for Distributed Replicated Block Device [Editor's note: the author works for LINBIT, the authors of DRBD]. Conceptually, data is written to one server, and immediately replicated to a second. DRBD is free and open source, and according to the Linux Kernel Mailing List, slated to be integrated into kernel 2.6.32 this month.

DRBD is most often used to create high-availability clusters, with no shared storage, and therefore no single points of failure. Should a server in the pair crash due to OS bug or failed hardware, the other can be configured to automatically take over its duties--with an up-to-date copy of the application or database data. DRBD's third-node feature is most often used to locate a third copy of data off-site. Should an entire site go down, another up-to-date (as of the last file system write) copy of the data is ready to go. So how does this apply to the cloud?

A few models exist to replicate your data into the cloud. A pair of servers, configured to failover using ClusterLabs' Pacemaker cluster manager software, can have one node local, and one node in the cloud. Getting failover to work in this instance is non-trivial, and requires some fancy BGP routing knowledge, or VPN tunnels to migrate an IP address. The VPN option assumes the local site isn't completely down, though.

More often, a slower failover in the case of a datacenter meltdown is acceptable. Maybe DNS has to be changed, and the propagation time is acceptable. In this case, the general 3-node DRBD setup is used. One highly available cluster (pair of servers) at the local datacenter site serves up MySQL, Oracle, NFS, iSCSI, Apache or whatever application is needed, and a third node sitting in the cloud quietly sits receiving a copy of every transaction.

One problem with real-time replication is that the performance of your Internet connection is extremely important. Writes to a DRBD volume are immediately sent to replica nodes, and the primary node must wait for acknowledgement before proceeding with too many more data writes, to ensure data integrity. This can dramatically slow down file system performance in the face of the most common bottleneck: your Internet connection. Enter DRBD Proxy (not free, not open source) to alleviate this pain. The proxy runs on the cluster, buffering data very much like the write cache of a SAN storage controller. The primary node(s) can now write huge amounts of data, without waiting for the off-site copy to catch up.

In summary, if you need a crude but effective way to simply ensure that some data exists off-site, a quick rsync to the cloud may work. You can also use robust backup software to schedule "real" backups via cloud APIs. If, however, you want to have an up-to-date copy of your data to implement whatever clever real-time failover or disaster-mode fallback mechanism you like, you should consider DRBD.

Source

Related articles by Zemanta

• Realizing Microsoft's potential in the cloud (venturebeat.com)

The announcement of Apps.gov is the first step in a long journey to the clouds by the U.S. government

The federal government has jumped into cloud computing with both feet with the announcement of Apps.gov, the first service in the U.S. government's movement to the cloud. Apps.gov is, in essence, a new Web site where agencies can purchase online applications and -- soon -- infrastructure services.

[ Get the no-nonsense explanations and advice you need to take real advantage of cloud computing in the InfoWorld editors' 21-page Cloud Computing Deep Dive PDF special report. | Stay up on the cloud with InfoWorld's Cloud Computing Report newsletter. ]

According to IDG:

White House CIO Vivek Kundra unveiled the first service in the U.S. government's new cloud computing initiative on Tuesday, launching a new Web site where federal agencies can buy online apps and basic computing services. Run by the U.S. General Services Administration, Apps.gov is an online storefront where government agencies can buy online applications from companies such as Google and Salesforce.com. IT services such as storage, Web hosting, and virtual machines will eventually be offered here as well.

I'm not sure that anyone was surprised by the announcement; the U.S. government has a huge interest in cloud computing, and the GSA is leading the way. Existing cloud computing providers see the opportunity as well with Google creating a government version of its cloud offering. The rest of the large and small cloud computing minions are looking hard at the emerging government marketplace too. We could be heading to a time where the U.S. government is actually leading the way with an emerging technology, creating best practices well ahead of the commercial world.

More

Related articles by Zemanta

• Government eyes big savings with first cloud service (infoworld.com)
• Obama Administration Gets Their Own Cloud Computing App Store [Government] (gizmodo.com)
• Apps.gov Gives Cloud Computing a Slice of the $75 Billion IT Pie (readwriteweb.com)

Cloud Computing is being used contrary to the perceived issues.  Perception unfortunately is sometimes not in line with reality and I can't think of a better example than Salesforce.com to hold up as proof that the Cloud works for you, if you so choose to embrace it.

A white paper written by Salesforce.com strips out some of the chaff and distills to  a handful of compelling major reasons why CIOs are adopting Cloud Computing.

Please take the time to read the paper in its entirety and decide for yourself if these aren't good reasons to consider Cloud Computing:

1. Delivers Faster Time to Value
2. Requires No Up-Front Capital Expense
3. Minimizes Operational Costs
4. Requires Fewer Technical Resources
5. Simplifies Integration

Please feel free to give your feedback in the comments section.
Thank you.
--Dietrich

Related articles by Zemanta

• There's no escaping the cloud (theregister.co.uk)
• Salesforce.com sets sights on the smallest businesses (infoworld.com)
• Virtualization and the cloud: Tech, talk to converge (news.cnet.com)

Related articles by Zemanta

• Bits: Server Makers Go Retro as Sales Plummet (bits.blogs.nytimes.com)
• Not every cloud has a silver lining (guardian.co.uk)
• Server sales continue to plummet (news.cnet.com)

by Leena Rao on September 3, 2009

Last year, Google rolled out forms that link into Google Doc's spreadsheets, providing elementary database-style form support for its online office suite. Forms basically let you add data to a spreadsheet without having to enter it directly into the spreadsheet itself, or even having to log in because you can add the data through a survey.

Today, Google is upgrading its Forms tool in Google Docs, adding a number of new features. Forms is basically a way to conduct a survey, with responses added automatically to a spreadsheet. Users now have a more compact, grid-like form in which to collect data. They can now quickly gather responses for a group of similar questions by simply labeling a few columns and creating as many rows as they like.

Summary charts also have clearer formatting of statistics and now support right-to-left text input, helping out those users whose written languages go from right-to-left. Developers can also integrate forms with their own applications and pre-populate a form with data.

Since its launch, Google forms has been an easy and accessible way to collect large amounts of data. Of course, the obvious use for forms is for surveys where you are collecting a massive amount of data and then need to make sense of it. You can either embed surveys into a blog post or site or you can share a link to the survey. Any responses are collected in a spreadsheet.

These new features make forms a little bit more user-friendly and attractive. Forms aren't the most popular Google app out there, but I'm sure to try them out the next time I post a survey on TechCrunch, instead of using SurveyMonkey or another survey application. I actually created a survey (see below) but my one complaint is that it doesn't show respondents the results, or at least if it does, it is hard to figure out and is not an automatic function.

Source

Related articles by Zemanta

• Sharing with Google Groups (googleblog.blogspot.com)
• Translate Google Docs And Share Them Around The World (techcrunch.com)
• You're Already Running Google Chrome OS (thaibrother.com)

All Things Distributed

By Werner Vogels on August 25, 2009 11:00 PM

At this 3^rd anniversary of the launch of Amazon Elastic Compute Cloud (Amazon EC2), it is amazing to see the impact this service has had on the industry. It is truly disruptive technology and its impact has reached far beyond a pure technology offering as the benefits of the cloud have changed the way we view IT Infrastructure. As one of the CIOs at the ACM Cloud Computing Roundtable summarized it: "IT used to be the blocker in anything we did, but with our shift to the cloud IT is now the enabler." From young businesses and established enterprises to hospitals and governments agencies, all are equally enthusiastic cloud customers for whom IT infrastructure has changed forever.

Even though we keep rolling out new services and features, and several existing AWS services are already very successful, this is still Day One. We are only at the brink of what is possible to deliver in the cloud and at Amazon we continue to innovate to make this future a reality.

We continuously listen to our customers to make sure our roadmap matches their needs. One important piece of feedback that mainly came from our enterprise customers was that the transition to the cloud of more complex enterprise environments was challenging. We made it a priority to address this and have worked hard in the past year to find new ways to help our customers transition applications and services to the cloud, while protecting their investments in their existing IT infrastructure.

Protecting investments during the transition

Most enterprises with a datacenter practice have invested significantly over the past decade into the management of their systems and applications. CIOs of Fortune 500 companies are responsible for hundreds if not thousands of applications running in a variety of locations. Keeping track of those resources and managing access to them is a daunting task that continues to require significant investment.

The CIO of a large financial services company in the Northeast explained to me that his teams manage close to 3000 applications and services in 27 different locations.  Consolidation of applications, resources and locations is a process that never stops in a world where mergers and acquisitions happen frequently.  For him the cloud is attractive as a target for his consolidated services: it allows him to significantly reduce both his capital and operational costs, while gaining significant flexibility and reliability with resources that are globally distributed, without the headache of owning and maintaining them.

He has set the guideline that their current data center infrastructure should not expand any further and that all new development will target the cloud. He expects that the process of moving his existing applications and services to the cloud will take time to complete, as his road map is driven by many internal and external factors.  And there are certainly some legacy applications that may never move. He has set the goal of moving 20% of his applications into the cloud by the end of 2010, but to meet this goal he needed to find a solution for a significant obstacle: how to integrate applications running in the cloud into his existing management frameworks. In his world, this especially applies to those management practices that manage policy-driven access controls and required, cross-application regulatory auditing.

This story is typical of many of the conversations I have had with CIOs around the globe. They have bought into the cloud as a target for a significant portion of their services, as the benefits are too obvious to ignore, and most expect that their transition will be a continuous process.  They would accelerate the adoption of cloud services if they could access a form of cloud that would give them the best of both worlds: the flexibility and cost-effectiveness of accessing a virtually infinite pool of resources without owning it, while being able to integrate those resources into their existing datacenter environments such that they could continue to leverage existing investments in their management and control infrastructure.

Private Cloud is not the Cloud

These CIOs know that what is sometimes dubbed "private cloud" does not meet their goal as it does not give them the benefits of the cloud: true elasticity and capex elimination. Virtualization and increased automation may give them some improvements in utilization, but they would still be holding the capital, and the operational cost would still be significantly higher.

I often get asked to define "The Cloud," especially because of the many permutations that different vendors use in trying to make their existing businesses look like a cloud offering. I define the cloud by it benefits, as those are very clear. What are called private clouds have little of these benefits and as such, I don't think of them as true clouds.

The cloud:

• Eliminates Cost. The cloud changes capital expense to variable expense and lowers operating costs. The utility-based pricing model of the cloud combined with its on-demand access to resources eliminates the needs for capital investments in IT Infrastructure. And because resources can be released when no longer needed, effective utilization rises dramatically and our customers see a significant reduction in operational costs.
• Is Elastic. The ready access to vast cloud resources eliminates the need for complex procurement cycles, improving the time-to-market for its users. Many organizations have deployment cycles that are counted in weeks or months, while cloud resources such as Amazon EC2 only take minutes to deploy. The scalability of the cloud no longer forces designers and architects to think in resource-constrained ways and they can now pursue opportunities without having to worry how to grow their infrastructure if their product becomes successful.
• Removes Undifferentiated "Heavy Lifting."The cloud let its users focus on delivering differentiating business value instead of wasting valuable resources on the undifferentiated heavy lifting that makes up most of IT infrastructure. Over time Amazon has invested over $2B in developing technologies that could deliver security, reliability and performance at tremendous scale and at low cost. Our teams have created a culture of operational excellence that power some of the world's largest distributed systems. All of this expertise is instantly available to customers through the AWS services.

Elasticity is one of the fundamental properties of the cloud that drives many of its benefits. While virtualization has tremendous benefits to the enterprise, certainly as an important tool in server consolidation, it by itself is not sufficient to give the benefits of the cloud. To achieve true cloud-like elasticity in a private cloud, such that you can rapidly scale up and down in your own datacenter, will require you to allocate significant hardware capacity. While to your internal customers it may appear that they have increased efficiency, at the company level you still own all the capital expense of the IT infrastructure. Without the diversity and heterogeneity of the large number of AWS cloud customers to drive a high utilization level, it can never be a cost-effective solution.

We have been listening very closely to the real requirements that our customers have and have worked closely with many of these CIOs and their teams to understand what solution would allow them to treat the cloud as a seamless extension of their datacenter, where their standard management practices can be applied with limited or no modifications. This needs to be a solution where they get all the benefits of cloud as mentioned above while treating it as a part of their datacenter.

Introducing Amazon Virtual Private Cloud

We have developed Amazon Virtual Private Cloud (Amazon VPC) to allow our customers to seamlessly extend their IT infrastructure into the cloud while maintaining the levels of isolation required for their enterprise management tools to do their work.

With Amazon VPC you can:

• Create a Virtual Private Cloud and assign an IP address block to the VPC. The address block needs to be CIDR block such that it will be easy for your internal networking to route traffic to and from the VPC instance. These are addresses you own and control, most likely as part of your current datacenter addressing practice.
• Divide the VPC addressing up into subnets in a manner that is convenient for managing the applications and services you want run in the VPC.
• Create a VPN connection between the VPN Gateway that is part of the VPC instance and an IPSec-based VPN router on your own premises. Configure your internal routers such that traffic for the VPC address block will flow over the VPN.
• Start adding AWS cloud resources to your VPC. These resources are fully isolated and can only communicate to other resources in the same VPC and with those resources accessible via the VPN router. Accessibility of other resources, including those on the public internet, is subject to the standard enterprise routing and firewall policies.

Amazon VPC offers customers the best of both the cloud and the enterprise managed data center:

• Full flexibility in creating a network layout in the cloud that complies with the manner in which IT resources are managed in your own infrastructure.
• Isolating resources allocated in the cloud by only making them accessible through industry standard IPSec VPNs.
• Familiar cloud paradigm to acquire and release resources on demand within your VPC, making sure that you only use those resources you really need.
• Only pay for what you use. The resources that you place within a VPC are metered and billed using the familiar pay-as-you-go approach at the standard pricing levels published for all cloud customers. The creation of VPCs, subnets and VPN gateways is free of charge. VPN usage and VPN traffic are also priced at the familiar usage based structure
• All the benefits from the cloud with respect to scalability and reliability, freeing up your engineers to work on things that really matter to your business.

For more details on Amazon Virtual Private Cloud, visit the Amazon VPC detail page and the posting on the AWS developer weblog. For how our partners view Amazon VPC see for example the posting at RightScale

And happy birthday to Amazon EC2!

Source

Related articles by Zemanta

• The Pros and Cons of Cloud Computing (technosailor.com)
• Information Is The Lifeblood Of A Business (slideshare.net)
• Elastic Load Balancing, Auto Scaling, and CloudWatch Resources (aws.typepad.com)

July 22, 2009

By Leon Erlanger

As if outsourcing, virtualization, utility computing, automation, hosted applications, and a recession weren't enough to stress out the average IT professional, there's the emerging threat of cloud computing to take away even more IT jobs. As time progresses, analyst firms foresee the cloud becoming more prevalent, absorbing functions traditionally done by IT. IDC predicts that worldwide IT spending on cloud services will grow almost threef

old by 2012 to $42 billion. Gartner has even predicted that, for IT, cloud computing will become as influential as e-business has been.

So exactly how -- and when -- will cloud computing reshape IT organizations and IT jobs? And what should the typical IT staffer do to protect his or her career?

[ Prepare yourself for the changing IT landscape with InfoWorld's tech career survival guide. | Get sage career advice each week from Bob Lewis' Advice Line blog. ]

First, don't panic. Any large-scale shift to cloud computing is a decade or more away, says Gartner analyst Ben Pring. "For now I look at software as a service [Saas] and cloud computing as an extension of the company's network, not a replacement," says Kim Terry, president of Terrosa Technologies, which helps software vendors make their wares available through the cloud. "In most organization, it's likely to be five years before anyone is ready to change out a company's financial systems [to the cloud]," he says.

The cloud will create a few jobs, at first
In the short term, cloud computing today may actually create some IT jobs, says James Staten, a principal analyst at Forrester Research. The reason: Today, cloud computing is mostly used for new applications. And, even as you rely on cloud providers for the back-end work, "you still have to know a lot about this infrastructure; you just don't have to manage it yourself," he says.

"Some percentage of the jobs actually performing infrastructure services, monitoring, and datacenter operations in-house will shift to cloud service providers like Google, Amazon, and the telcos," says Mark McDonald, Gartner's group vice president of executive programs. But there won't be much growth in these infrastructure jobs at the cloud providers, he notes, due to the economies of scale that come from massive, highly automated and virtualized service-based infrastructures. "There will likely be fewer people needed per thousand transactions," he says.

"But if you're able to get one of these jobs, in many cases it's a skill set that is less technical and more managerial and administrative, with days full of conference calls and putting out fires. [In such jobs,] few technical skills [are] added to your repertoire," says Carole Schlocker, president of IT staffing services at iSpace. Even management growth will be limited since most of the big decisions have been made in the service contracts, she notes.

Even if you do manage to get a job at a cloud provider's datacenter, it most likely won't be where you live today. Terry points out that many of these jobs will move outside of the major cities. "Instead of running a small datacenter near Washington, D.C., or New York City, cloud providers will tend toward an area that makes sense from a facilities perspective, such as near a large dam or close to a power-generation source in a place like Idaho where electricity is less expensive," says Terrosa's Terry.

Read more

Related articles by Zemanta

• Delivering Security on Demand (technologyreview.com)
• Gartner Asks, Can The Cloud Save The World? (elasticvapor.com)
• Is Everything Cloud Computing? (berkeleyclouds.blogspot.com)