Disclaimer: I work for the supercomputing vendor Cray Inc. The opinions expressed in this article are my own and do not represent my employer.
This week Amazon announced the addition of a GPU compute cluster instance to their Elastic Compute Cloud (EC2) service. This is an exciting and not unexpected addition to EC2 and has drawn a lot of attention at the 2010 Supercomputing conference (SC10). I read an article this week over at the Cycle Computing blog declaring the new AWS GPU instances as A Couple More Nails in the Coffin of the Private Compute Cluster. Do Amazon’s HPC and GPU instances mean the death to HPC vendors? Should I, as an employee of possibly the most iconic company in supercomputing, be concerned that Amazon is going to drive HPC vendors out of business? No. I, for one, welcome Amazon into the HPC community.
So what does all of this mean to HPC? These new offerings mean a much lower barrier to entry into High Performance Computing, much like cluster computers did in the mid-90s. They mean that universities and businesses that have neither the money nor expertise to run an HPC system can begin using HPC to meet their needs. Could this reduce the number of privately owned clusters? Yes, but only on the very lowest end of the market. I can’t see the national labs, NSF, weather services around the globe, or countless other existing HPC users moving to the cloud. For some, the cloud will not provide the necessary security, but for many it will not provide the necessary capabilities. There are classes of problems that will run extremely well on these low-cost, cloud-based offerings, but there are also classes of problems that can’t even be solved on the largest machines currently being built.
I see both of Amazon’s HPC offerings as an incredible boon to the industry. As more companies begin to use HPC, they will gain experience with and reap the benefits of HPC. This will encourage these users to develop higher resolution models, or to model additional chemical species, or simply to want a faster time to solution on their existing models. As their needs grow, these users may decide to purchase larger, in-house systems or to pursue Teragrid or INCITE allocations. Perhaps they will eventually develop models so revolutionary that they will receive the ACM Gordon Bell Prize, all because they experienced HPC in the cloud.
Will these new HPC offerings kill privately owned HPC systems? I believe they will eventually lead to more HPC installations. Just as the low cost and ease of deployment of Amazon Web Services led to an enormous growth in web-based companies, these new HPC EC2 instances will lead to new growth in the HPC market. Welcome Amazon and thank you for exposing more people to HPC.
So, what do you think of Amazon’s HPC in the cloud? Should traditional HPC vendors fear the cloud or welcome a new generation of HPC users?