Yahoo, HP and Intel are going to do cloud research together using a global, multi-data center, open source Cloud Computing Test Bed bigger than anything put together for such a purpose before, they said.
The whole testbed could potentially scale to 24,000 cores, 18 terabytes of memory and 9 petabytes of disk, roughly 164 teraFLOPS of power, big enough, the threesome said, for Internet-scale tests, at least tests of short duration.
There will be six - God willing always-available - sites: one at each of the vendors and one each at the state-run Infocomm Development Authority of Singapore (IDA), the University of Illinois at Urbana-Champaign and Karlsruhe Institute of Technology (KIT) in Germany.
Each site - and, remember, academic researchers have lacked the hardware and software infrastructure to support Internet-scale systems software research before - is supposed to consist of 1,000-4,000 cores.
The hardware will be almost exclusively Intel-based HP widgetry, though what exactly they're not saying since one of the points of the exercise is to figure out works best.
HP of course has its new cloud-bound Xeon-based ProLiant BL2x220c G5, the first server blade to combine two independent servers in a single blade, and its StorageWorks 9100 Extreme Data Storage System (ExDS9100), a highly scalable storage system designed to simplify the management of multiple petabytes.
Intel, which already supports Tashi, the open source cluster management system for cloud computing, will test and perhaps concoct some mojo we haven't seen yet beyond its Data Center Management Interface (DCMI), Node Manager (NM) and virtualization stuff.
There's always the possibility that a radically new architecture could merge.
Yahoo's supercomputing cluster, dubbed M45 after one of the star clusters, has been up and working since November when Yahoo opened it up to research by Carnegie Mellon. It appears to be the model or proof point for the other five data centers, all which are supposed to be operational by the end of the year.
As Yahoo said in November, it planned to make M45 available to researchers from other universities for "open, collaborative research."
Yahoo's purpose with M45 was to advance Hadoop, the Apache Software Foundation's open source sub-project, an open source distributed file system and parallel execution environment that processes massive amounts of data.
Yahoo has been Hadoop's primary contributor and it's looking for other contributions so Hadoop will be integral to the HP-Intel-Yahoo R&D effort. Unless contributions are governed by an open source license the IP will belong to the developer.
The vendors, who are kicking in research talent themselves, said the breath of their research would be wider than, say, what IBM and Google are doing, which appears to be limited to the applications layer.
Besides hardware testing, Haddop will form the basis of the systems software research and the trio, particularly Intel, is interested in advancing the cause of parallel programming and software management.
Yahoo is also interested in advancing its Yahoo Research-developed Pig open source parallel programming language,
The trio wants to understand how systems software and hardware function in a cloud environment.
Obviously the results should turn up in applications software and services.
HP Labs says it will use the test bed for advanced research into intelligent infrastructure and dynamic cloud services, and stuff like massive storage and software deployment.
Under HP's concept of "Everything as a Service," devices and services are supposed to interact seamlessly through the cloud, and it figures businesses and individuals will use services that anticipate their needs based on location, preferences, calendar and communities.
According to Prith Banerjee, HP's senior vice-president of research and director of HP Labs, "To realize the full potential of cloud computing, the technology industry must think about the cloud as a platform for creating new services and experiences. This requires an entirely new approach to the way we design, deploy and manage cloud infrastructure and services."
In answer to a question on a conference call, Intel replied that the founders might be willing - pending discussions - to take in other partners.
They declined to talk about the size of the investment but it appears the National Science Foundation is picking up the chit for the University of Illinois.
Supposedly however the goal of the initiative is to "promote open collaboration among industry, academia and governments by removing the financial and logistical barriers to research in data-intensive, Internet-scale computing."
Subscribe to Eclipse Developer's Journal Email News Alerts
Subscribe via RSS
