《融智学导论》作者邹晓辉认为:
所谓“云计算”就是集群协同计算。
它实质上是一种具体的协同智能计算系统。
注1:协同智能计算系统是狭义融智学的研究对象。
注2:附录1和附录2分别是本人标读的英文和中文译稿的全文。
附录1:
Google and the Wisdom of Clouds
A lofty new strategy aims to put incredible computing power in the hands of many
by Stephen Baker
One simple question. That's all it took for Christophe Bisciglia to bewilder confident job applicants at Google (GOOG). Bisciglia, an angular 27-year-old senior software engineer with long wavy hair, wanted to see if these undergrads were ready to think like Googlers. "Tell me," he'd say, "what would you do if you had 1,000 times more data?"
What a strange idea. If they returned to their school projects and were foolish enough to cram formulas with a thousand times more details about shopping or maps or—heaven forbid—with video files, they'd slow their college servers to a crawl.
At that point in the interview, Bisciglia would explain his question. To thrive at Google, he told them, they would have to learn to work—and to dream—on a vastly larger scale. He described Google's globe-spanning network of computers. Yes, they answered search queries instantly. But together they also blitzed through mountains of data, looking for answers or intelligence faster than any machine on earth. Most of this hardware wasn't on the Google campus. It was just out there, somewhere on earth, whirring away in big refrigerated data centers. Folks at Google called it "the cloud." And one challenge of programming at Google was to leverage that cloud—to push it to do things that would overwhelm lesser machines. New hires at Google, Bisciglia says, usually take a few months to get used to this scale. "Then one day, you see someone suggest a wild job that needs a few thousand machines, and you say: Hey, he gets it.'"
What recruits needed, Bisciglia eventually decided, was advance training. So one autumn day a year ago, when he ran into Google CEO Eric E. Schmidt between meetings, he floated an idea. He would use his 20% time, the allotment Googlers have for independent projects, to launch a course. It would introduce students at his alma mater, the University of Washington, to programming at the scale of a cloud. Call it Google 101. Schmidt liked the plan. Over the following months, Bisciglia's Google 101 would evolve and grow. It would eventually lead to an ambitious partnership with IBM (IBM), announced in October, to plug universities around the world into Google-like computing clouds.
As this concept spreads, it promises to expand Google's footprint in industry far beyond search, media, and advertising, leading the giant into scientific research and perhaps into new businesses. In the process Google could become, in a sense, the world's primary computer.
"I had originally thought [Bisciglia] was going to work on education, which was fine," Schmidt says late one recent afternoon at Google headquarters. "Nine months later, he comes out with this new [cloud] strategy, which was completely unexpected." The idea, as it developed, was to deliver to students, researchers, and entrepreneurs the immense power of Google-style computing, either via Google's machines or others offering the same service.
What is Google's cloud? It's a network made of hundreds of thousands, or by some estimates 1 million, cheap servers, each not much more powerful than the PCs we have in our homes. It stores staggering amounts of data, including numerous copies of the World Wide Web. This makes search faster, helping ferret out answers to billions of queries in a fraction of a second. Unlike many traditional supercomputers, Google's system never ages. When its individual pieces die, usually after about three years, engineers pluck them out and replace them with new, faster boxes. This means the cloud regenerates as it grows, almost like a living thing.
A move towards clouds signals a fundamental shift in how we handle information. At the most basic level, it's the computing equivalent of the evolution in electricity a century ago when farms and businesses shut down their own generators and bought power instead from efficient industrial utilities. Google executives had long envisioned and prepared for this change. Cloud computing, with Google's machinery at the very center, fit neatly into the company's grand vision, established a decade ago by founders Sergey Brin and Larry Page: "to organize the world's information and make it universally accessible." Bisciglia's idea opened a pathway toward this future. "Maybe he had it in his brain and didn't tell me," Schmidt says. "I didn't realize he was going to try to change the way computer scientists thought about computing. That's a much more ambitious goal."
1
ONE-WAY STREET
For small companies and entrepreneurs, clouds mean opportunity—a leveling of the playing field in the most data-intensive forms of computing. To date, only a select group of cloud-wielding Internet giants has had the resources to scoop up huge masses of information and build businesses upon it. Our words, pictures, clicks, and searches are the raw material for this industry. But it has been largely a one-way street. Humanity emits the data, and a handful of companies—the likes of Google, Yahoo! (YHOO), or Amazon.com (AMZN)—transform the info into insights, services, and, ultimately, revenue.
This status quo is already starting to change. In the past year, Amazon has opened up its own networks of computers to paying customers, initiating new players, large and small, to cloud computing. Some users simply park their massive databases with Amazon. Others use Amazon's computers to mine data or create Web services. In November, Yahoo opened up a cluster of computers—a small cloud—for researchers at Carnegie Mellon University. And Microsoft (MSFT) has deepened its ties to communities of scientific researchers by providing them access to its own server farms. As these clouds grow, says Frank Gens, senior analyst at market research firm IDC, "A whole new community of Web startups will have access to these machines. It's like they're planting Google seeds." Many such startups will emerge in science and medicine, as data-crunching laboratories searching for new materials and drugs set up shop in the clouds.
For clouds to reach their potential, they should be nearly as easy to program and navigate as the Web. This, say analysts, should open up growing markets for cloud search and software tools—a natural business for Google and its competitors. Schmidt won't say how much of its own capacity Google will offer to outsiders, or under what conditions or at what prices. "Typically, we like to start with free," he says, adding that power users "should probably bear some of the costs." And how big will these clouds grow? "There's no limit," Schmidt says. As this strategy unfolds, more people are starting to see that Google is poised to become a dominant force in the next stage of computing. "Google aspires to be a large portion of the cloud, or a cloud that you would interact with every day," the CEO says. The business plan? For now, Google remains rooted in its core business, which gushes with advertising revenue. The cloud initiative is barely a blip in terms of investment. It hovers in the distance, large and hazy and still hard to piece together, but bristling with possibilities.
Changing the nature of computing and scientific research wasn't at the top of Bisciglia's agenda the day he collared Schmidt. What he really wanted, he says, was to go back to school. Unlike many of his colleagues at Google, a place teeming with PhDs, Bisciglia was snatched up by the company as soon as he graduated from the University of Washington, or U-Dub, as nearly everyone calls it. He'd never been a grad student. He ached for a break from his daily routines at Google—the 10-hour workdays building search algorithms in his cube in Building 44, the long commutes on Google buses from the apartment he shared with three roomies in San Francisco's Duboce Triangle. He wanted to return to Seattle, if only for one day a week, and work with his professor and mentor, Ed Lazowska. "I had an itch to teach," he says.
2
He didn't think twice before vaulting over the org chart and batting around his idea directly with the CEO. Bisciglia and Schmidt had known each other for years. Shortly after landing at Google five years ago as a 22-year-old programmer, Bisciglia worked in a cube across from the CEO's office. He'd wander in, he says, drawn in part by the model airplanes that reminded him of his mother's work as a United Airlines (UAUA) hostess. Naturally he talked with the soft-spoken, professorial CEO about computing. It was almost like college. And even after Bisciglia moved to other buildings, the two stayed in touch. ("He's never too hard to track down, and he's incredible about returning e-mails," Bisciglia says.)
On the day they first discussed Google 101, Schmidt offered one nugget of advice: Narrow down the project to something Bisciglia could have up and running in two months. "I actually didn't care what he did," Schmidt recalls. But he wanted the young engineer to get feedback in a hurry. Even if Bisciglia failed, he says, "he's smart, and he'd learn from it."
To launch Google 101, Bisciglia had to replicate the dynamics and a bit of the magic of Google's cloud—but without tapping into the cloud itself or revealing its deepest secrets. These secrets fuel endless speculation among computer scientists. But Google keeps much under cover. This immense computer, after all, runs the company. It automatically handles search, places ads, churns through e-mails. The computer does the work, and thousands of Google engineers, including Bisciglia, merely service the machine. They teach the system new tricks or find new markets for it to invade. And they add on new clusters—four new data centers this year alone, at an average cost of $600 million apiece.
In building this machine, Google, so famous for search, is poised to take on a new role in the computer industry. Not so many years ago scientists and researchers looked to national laboratories for the cutting-edge research on computing. Now, says Daniel Frye, vice-president of open systems development at IBM, "Google is doing the work that 10 years ago would have gone on in a national lab."
How was Bisciglia going to give students access to this machine? The easiest option would have been to plug his class directly into the Google computer. But the company wasn't about to let students loose in a machine loaded with proprietary software, brimming with personal data, and running a $10.6 billion business. So Bisciglia shopped for an affordable cluster of 40 computers. He placed the order, then set about figuring out how to pay for the servers. While the vendor was wiring the computers together, Bisciglia alerted a couple of Google managers that a bill was coming. Then he "kind of sent the expense report up the chain, and no one said no." He adds one of his favorite sayings: "It's far easier to beg for forgiveness than to ask for permission." ("If you're interested in someone who strictly follows the rules, Christophe's not your guy," says Lazowska, who refers to the cluster as "a gift from heaven.")
A FRENETIC LEARNER
On Nov. 10, 2006, the rack of computers appeared at U-Dub's Computer Science building. Bisciglia and a couple of tech administrators had to figure out how to hoist the 1-ton rack up four stories into the server room. They eventually made it, and then prepared for the start of classes, in January.
Bisciglia's mother, Brenda, says her son seemed marked for an unusual path from the start. He didn't speak until age 2, and then started with sentences. One of his first came as they were driving near their home in Gig Harbor, Wash. A bug flew in the open window, and a voice came from the car seat in back: "Mommy, there's something artificial in my mouth."
At school, the boy's endless questions and frenetic learning pace exasperated teachers. His parents, seeing him sad and frustrated, pulled him out and home-schooled him for three years. Bisciglia says he missed the company of kids during that time but developed as an entrepreneur. He had a passion for Icelandic horses and as an adolescent went into business raising them. Once, says his father, Jim, they drove far north into Manitoba and bought horses, without much idea about how to transport the animals back home. "The whole trip was like a scene from one of Chevy Chase's movies," he says. Christophe learned about computers developing Web pages for his horse sales and his father's luxury-cruise business. And after concluding that computers promised a brighter future than animal husbandry, he went off to U-Dub and signed up for as many math, physics, and computer courses as he could.
3
In late 2006, as he shuttled between the Googleplex and Seattle preparing for Google 101, Bisciglia used his entrepreneurial skills to piece together a sprawling team of volunteers. He worked with college interns to develop the curriculum, and he dragooned a couple of Google colleagues from the nearby Kirkland (Wash.) facility to use some of their 20% time to help him teach it. Following Schmidt's advice, Bisciglia worked to focus Google 101 on something students could learn quickly. "I was like, what's the one thing I could teach them in two months that would be useful and really important?" he recalls. His answer was "MapReduce."
Bisciglia adores MapReduce, the software at the heart of Google computing. While the company's famous search algorithms provide the intelligence for each search, MapReduce delivers the speed and industrial heft. It divides each task into hundreds, or even thousands, of tasks, and distributes them to legions of computers. In a fraction of a second, as each one comes back with its nugget of information, MapReduce quickly assembles the responses into an answer. Other programs do the same job. But MapReduce is faster and appears able to handle near limitless work. When the subject comes up, Bisciglia rhapsodizes. "I remember graduating, coming to Google, learning about MapReduce, and really just changing the way I thought about computer science and everything," he says. He calls it "a very simple, elegant model." It was developed by another Washington alumnus, Jeffrey Dean. By returning to U-Dub and teaching MapReduce, Bisciglia would be returning this software "and this way of thinking" back to its roots.
There was only one obstacle. MapReduce was anchored securely inside Google's machine—and it was not for outside consumption, even if the subject was Google 101. The company did share some information about it, though, to feed an open-source version of MapReduce called Hadoop. The idea was that, without divulging its crown jewel, Google could push for its standard to become the architecture of cloud computing.
The team that developed Hadoop belonged to a company, Nutch, that got acquired. Oddly, they were now working within the walls of Yahoo, which was counting on the MapReduce offspring to give its own computers a touch of Google magic. Hadoop remained open source, though, which meant the Google team could adapt it and install it for free on the U-Dub cluster.
Students rushed to sign up for Google 101 as soon as it appeared in the winter-semester syllabus. In the beginning, Bisciglia and his Google colleagues tried teaching. But in time they handed over the job to professional educators at U-Dub. "Their delivery is a lot clearer," Bisciglia says. Within weeks the students were learning how to configure their work for Google machines and designing ambitious Web-scale projects, from cataloguing the edits on Wikipedia to crawling the Internet to identify spam. Through the spring of 2007, as word about the course spread to other universities, departments elsewhere started asking for Google 101.
Many were dying for cloud knowhow and computing power—especially for scientific research. In practically every field, scientists were grappling with vast piles of new data issuing from a host of sensors, analytic equipment, and ever-finer measuring tools. Patterns in these troves could point to new medicines and therapies, new forms of clean energy. They could help predict earthquakes. But most scientists lacked the machinery to store and sift through these digital El Dorados. "We're drowning in data," said Jeannette Wing, assistant director of the National Science Foundation.
BIG BLUE LARGESSE
The hunger for Google computing put Bisciglia in a predicament. He had been fortunate to push through the order for the first cluster of computers. Could he do that again and again, eventually installing mini-Google clusters in each computer science department? Surely not. To extend Google 101 to universities around the world, the participants needed to plug into a shared resource. Bisciglia needed a bigger cloud.
4
That's when luck descended on the Googleplex in the person of IBM Chairman Samuel J. Palmisano. This was "Sam's day at Google," says an IBM researcher. The winter day was a bit chilly for beach volleyball in the center of campus, but Palmisano lunched on some of the fabled free cuisine in a cafeteria. Then he and his team sat down with Schmidt and a handful of Googlers, including Bisciglia. They drew on whiteboards and discussed cloud computing. It was no secret that IBM wanted to deploy clouds to provide data and services to business customers. At the same time, under Palmisano, IBM had been a leading promoter of open-source software, including Linux. This was a key in Big Blue's software battles, especially against Microsoft. If Google and IBM teamed up on a cloud venture, they could construct the future of this type of computing on Google-based standards, including Hadoop.
Google, of course, had a running start on such a project: Bisciglia's Google 101. In the course of that one day, Bisciglia's small venture morphed into a major initiative backed at the CEO level by two tech titans. By the time Palmisano departed that afternoon, it was established that Bisciglia and his IBM counterpart, Dennis Quan, would build a prototype of a joint Google-IBM university cloud.
Over the next three months they worked together at Google headquarters. (It was around this time, Bisciglia says, that the cloud project evolved from 20% into his full-time job.) The work involved integrating IBM's business applications and Google servers, and equipping them with a host of open-source programs, including Hadoop. In February they unveiled the prototype for top brass in Mountain View, Calif., and for others on video from IBM headquarters in Armonk, N.Y. Quan wowed them by downloading data from the cloud to his cell phone. (It wasn't relevant to the core project, Bisciglia says, but a nice piece of theater.)
The Google 101 cloud got the green light. The plan was to spread cloud computing first to a handful of U.S. universities within a year and later to deploy it globally. The universities would develop the clouds, creating tools and applications while producing legions of computer scientists to continue building and managing them.
Those developers should be able to find jobs at a host of Web companies, including Google. Schmidt likes to compare the data centers to the prohibitively expensive particle accelerators known as cyclotrons. "There are only a few cyclotrons in physics," he says. "And every one if them is important, because if you're a top-flight physicist you need to be at the lab where that cyclotron is being run. That's where history's going to be made; that's where the inventions are going to come. So my idea is that if you think of these as supercomputers that happen to be assembled from smaller computers, we have the most attractive supercomputers, from a science perspective, for people to come work on."
As the sea of business and scientific data rises, computing power turns into a strategic resource, a form of capital. "In a sense," says Yahoo Research Chief Prabhakar Raghavan, "there are only five computers on earth." He lists Google, Yahoo, Microsoft, IBM, and Amazon. Few others, he says, can turn electricity into computing power with comparable efficiency.
All sorts of business models are sure to evolve. Google and its rivals could team up with customers, perhaps exchanging computing power for access to their data. They could recruit partners into their clouds for pet projects, such as the company's clean energy initiative, announced in November. With the electric bills at jumbo data centers running upwards of $20 million a year, according to industry analysts, it's only natural for Google to commit both brains and server capacity to the search for game-changing energy breakthroughs.
What will research clouds look like? Tony Hey, vice-president for external research at Microsoft, says they'll function as huge virtual laboratories, with a new generation of librarians—some of them human—"curating" troves of data, opening them to researchers with the right credentials. Authorized users, he says, will build new tools, haul in data, and share it with far-flung colleagues. In these new labs, he predicts, "you may win the Nobel prize by analyzing data assembled by someone else." Mark Dean, head of IBM's research operation in Almaden, Calif., says that the mixture of business and science will lead, in a few short years, to networks of clouds that will tax our imagination. "Compared to this," he says, "the Web is tiny. We'll be laughing at how small the Web is." And yet, if this "tiny" Web was big enough to spawn Google and its empire, there's no telling what opportunities could open up in the giant clouds.
It's a mid-November day at the Googleplex. A jetlagged Christophe Bisciglia is just back from China, where he has been talking to universities about Google 101. He's had a busy time, not only setting up the cloud with IBM but also working out deals with six universities—U-Dub, Berkeley, Stanford, MIT, Carnegie Mellon, and the University of Maryland—to launch it. Now he's got a camera crew in a conference room, with wires and lights spilling over a table. This is for a promotional video about cloud education that they'll release, at some point, on YouTube (GOOG).
Eric Schmidt comes in. At 52, he is nearly twice Bisciglia's age, and his body looks a bit padded next to his protégé's willowy frame. Bisciglia guides him to a chair across from the camera and explains the plan. They'll tape the audio from the interview and then set up Schmidt for some stand-alone face shots. "B-footage," Bisciglia calls it. Schmidt nods and sits down. Then he thinks better of it. He tells the cameramen to film the whole thing and skip stand-alone shots. He and Bisciglia are far too busy to stand around for B footage.
Baker is a senior writer for BusinessWeek in New York .
http://www.businessweek.com/magazine/content/07_52/b4064048925836.htm
附录2:
商业周刊:Google及其云智慧
这项全新的远大战略旨在把强大得超乎想象的计算能力分布到众人手中。
这是一个简单的问题,是克里斯托夫·比希利亚为信心十足的Google应聘者们出的一道题。作为Google公司的高级软件工程师,27岁的比希利亚留着一头卷曲的长发,他希望了解这些大学本科生是否已经准备好以Google人的方式去思考。“告诉我,”他问道,“如果有1000多倍的数据量,你将怎么办?” 真是个奇怪的问题。假如他们真的跑回学校,愚蠢地想要去处理容量多达1000多倍的细节信息,那么学校的服务器恐怕会被拖累得慢如爬虫。
比希利亚将在面试中阐释他的问题。他告诉应聘者,要想在Google发展,就必须学会从更宽广、更宏观的角度来工作和思考。他描述了Google全球运行的计算机网络。的确,这些设备可以实现对搜索需求的即时回馈;而当形成集群,它们则能更快地处理浩如烟海的数据,其检索答案或指令的速度将超过世界上任何一台单机。绝大部分硬件设备并非安放在Google公司园区,而是在园区之外,没准就在地球上某个大型冷却数据中心里高速运转着。Google内部把这种大规模计算机集群称作“云”。在Google,工程师编程过程中碰到的一大挑战便是如何驾驭“云”? 提高它的数据处理能力从而大幅领先于小型计算机群。比希利亚表示,Google的新员工通常要花费数月才能习惯从这种角度思考。比希利亚认为,Google的新人所需要的是高级培训课程。2006年秋季的一天,当他在会议间歇偶遇公司首席执行官埃里克·施米特时,他脑海里浮现出一个想法。他将利用自己的“20%时间”(即Google分配给员工用于独立开发项目的时间)来启动一门课程,这门课程将在他的母校华盛顿大学进行,着重引导学生们进行“云”系统的编程开发,他设想把这个项目命名为Google 101。施米特很是欣赏这一计划。在接下来的数月中,比希利亚的Google 101计划不断发展和深化,最终促成了Google与IBM在2007年10月开展了一次雄心勃勃的合作---把全球多所大学纳入类似Google的计算“云”中。
随着“云”概念影响的扩大,Google在产业中的足迹必然会远远超出搜索、媒体和广告领域,从而使这家IT巨头得以涉足科学研究甚至更新的业务领域。在这一过程中,Google在某种意义上可能会成为世界上首屈一指的超级计算机。“我最初以为(比希利亚)不过是想在教育上做点事情,这当然也不错,” 施米特最近一个下午在Google总部回想道,“9个月后他拿出了新战略(即‘云’计划),太出乎意料了。”随着自身的不断拓展,“云”计划将为学生、研究人员和企业家们提供Google式的无限的计算处理能力,不论是通过Google自身的设备或是通过提供相同服务的其他厂商。
Google的“云”到底是什么?它是由几十万甚至大约100万台廉价的服务器所组成的网络。这些机器单个而论的话,其性能并不比家用台式机强大多少。但是这个网络存储的数据量惊人,能容纳不计其数的网络数据拷贝,因此搜索速度能够更快,在眨眼之间便能为数十亿的搜索提交答案。与许多传统的超级计算机不同,Google的系统永远不会老化。如果网络中某一台机器落伍(通常在使用3年后),工程师们就会把它淘汰,而代之以性能更强的新款计算机。这意味着,“云”几乎就像生物一样能长生不老。
向“云”规模的数据处理迈进标志着我们在信息处理方面发生了翻天覆地的转变。从最基本的层面讲,“云”的发展就如同100年前人类用电的进程演变,当时的农场和公司逐渐关闭了自己的发电机,转而从高效的发电厂购买电力。Google的高管们很早前就开始展望这一转变并为之进行筹划准备。以Google设备为核心的“云计算”完全符合由该公司创始人谢尔盖·布林和拉里·佩奇10年前提出的远大构想:“构建起跨越全世界的信息,供人们随时随地访问。”比希利亚的想法刚好为实现这个构想开辟了一条道路。“没准他脑子里早就有数,只是没告诉我,”施米特表示,“我开始没有意识到他将试图改变计算机专家对于计算的固有想法。这个目标太伟大了。”
单行道
对于小型公司和企业主而言,“云”意味着机会,在密集型数据处理领域这一竞技场中,它就像是一道标准线。今天,掌控“云”系统的互联网巨头中,只有少数几家拥有吞吐海量信息并开展相关业务的资源。我们的文字、图片、点击和搜索全都是这个产业的原材料。一直以来,很大程度上这是一条单行道? 人们产出数据,Google、雅虎和亚马逊等公司则将信息转化成观点、服务,最终变成收入。这种状况已开始发生改变。2006年,亚马逊向付费用户开放了自己的计算机网络,调动新的参与者加入“云”计算,而无论其规模大小。一些用户只是简单地将数据库存储在亚马逊,另一些则使用亚马逊的服务器搜索数据或建立网络服务。2007年11月,雅虎也将一个电脑集群(即小规模的“云”)开放给卡内基-梅隆大学的研究人员。微软同样通过开放服务器群来加深与科学研究团体的联系。市场调查公司IDC的高级分析师弗兰克·金斯表示:随着这些“云”的发展,“新兴的网络公司将有望造访这些服务器。这就如同在播撒Google种子”。随着搜索新材料和药品的数据处理实验室把工作室搬到了“云”上,这些新公司很多将会出现在科学和医药领域。
要想让“云”发挥出潜能,与此相关的编程和操作就应该与使用互联网一样简单。分析家称,这给“云”搜索及其相关的软件工具打开了增长的市场,对于Google及其竞争对手来说,这可谓唾手可得的业务。Google将为用户提供多少存储容量,或以什么形式、什么价格提供,对于这些,施米特都不会明说。“通常来讲,我们开始时会采取免费策略,”他表示,并强调大客户“应该很有可能负担一些费用”。那么这些“云”能发展到多大的规模呢?“无限大,”施米特表示。随着“云”策略的展开,更多人看到Google随时做好了成为下一代计算的主导力量的准备。“Google渴望占据“云计算”市场中相当的份额,或成为每天都与普通人打交道的‘云’,”施米特说道。那么有什么样的商业计划呢?就目前而言,Google仍将继续植根于核心业务,这一业务给它带来了滚滚的广告收入。从投资角度来说,“云”计划在初始阶段不过是像雷达屏幕上显示的一个小光点。它在远处盘旋,目标很大却烟雾弥漫,很难拼凑在一起,但仍充满着无限可能。就在那次比希利亚趁机和施米特谈论Google 101时,改变计算和科研的现状还并不是他的主要意图。他自己说,当时真正想做的是返回学校。与公司里很多已拿到博士学位的同事不同,比希利亚刚从华盛顿大学毕业就被Google录用,他甚至没读过硕士研究生。因此他渴望从Google的日常工作中抽出时间换换脑子。在Google,比希利亚每天都需要从公寓搭乘班车长途跋涉到公司,然后开始10小时的搜索运算法则的编写工作。他想回到西雅图,哪怕每周只有一天,回到学校去和他的教授兼导师埃德·拉佐斯卡一起工作。
在突发灵感想到“云”计划并直接和老板详细讨论之前,比希利亚并没多加考虑。他和施米特已相识数年。他5年前刚入职Google时还只是一个年仅22岁的程序员,其工位就在首席执行官的办公室附近。比希利亚回忆说,他走进办公室时被一架飞机模型所吸引,这让他想起母亲在美国联合航空公司从事的空乘工作。自然而然地,他与话语温和、有学者派头的施米特聊起了数据计算,那种感觉就像在大学一样。后来虽然比希利亚搬到了其他办公楼,但两人仍然保持着联系。在他们第一次讨论“Google 101”计划的那一天,施米特提出了很好的建议:把项目缩减到比希利亚能在两个月内完成的规模。“我实际上没太在意他的话,”施米特回忆说,但是他想尽快给这位年轻的工程师发出反馈。他说,即使比希利亚失败了,但“他很聪明,一定能从失败中获得经验”。
要顺利启动“Google 101”计划,比希利亚必须把项目的来龙去脉和Google“云”的些许魔力透露给合作对象,同时又不能深入“云”本身或揭示出核心机密。这些机密会激发计算机学家无穷无尽的思考,Google对此守口如瓶,毕竟这台“超级计算机”是公司运营的支柱,它能自动处理搜索、放置广告、传递电子邮件等业务。计算机在从事这些工作,而包括比希利亚在内的上千名Google工程师仅仅只是“服侍”着它。他们“教授”系统新的技术或为它寻找新的主攻市场,同时在其中添加新的集群,2007年一年就增加了4个新的数据中心,平均每个成本达6亿美元。
在搭建这台“计算机”的过程中,在搜索领域名声大震的Google随时准备扮演计算机业的新角色。不久之前,科学家和研究人员曾期望国家实验室能启动数据计算方面的前沿研究。如今,IBM负责开放系统开发的副总裁丹尼尔·弗赖感叹:“Google现在做的事情10年前只有在国家实验室才能实现。”
那么,比希利亚如何让学生们访问这台机器呢?最容易的方案当然是直接从学校连接专线到Google服务器。然而公司并不准备彻底放手让学生们随意访问这台装有授权软件、存储着私人信息以及运营着106亿美元业务的计算机。比希利亚因此购买了价位适中的40台计算机组成集群。他发出订单后开始琢磨如何给这些服务器付钱。就在卖家组装电脑集群时,比希利亚告诉Google的几名经理将出现一大笔账单。之后他“拿着花销报告从下到上请示了一通,结果没人反对”。说到这里,他又加上自己喜欢的一句格言:“请求原谅比寻求批准容易得多。”
狂热的学习者 2006年11月10日,排成阵列的计算机群出现在华盛顿大学计算机科学学院的教学楼里。比希利亚和几个技术负责人得想办法把将近1吨重的机柜抬上4层放到机房里。他们最终解决了这个问题,并准备在第二年1月开始上课。
比希利亚的母亲布伦达说,她的儿子似乎从小就注定要走一条不平凡的道路。他直到两岁才开口说话,但很快就开始成句成句地说。最早的一次是家人开车行至离家不远的华盛顿吉格港时,一只小虫子从打开的车窗飞进来,只听到从后排座传来比利亚的声音:“妈妈,有一件物体在我嘴里。” 在学校里,这个男孩没完没了的提问和飞快的学习进度惹恼了老师。父母看到他很伤心、很受挫,便把他带回家教了3年。比希利亚说,那段时间他失去了很多小伙伴,但是学会了如何成为一个生意人。他对冰岛野马兴趣浓厚,并在十六七岁时投身到养马行当。他的父亲吉姆回忆道,一次,他们开车一直向北行驶到马尼托巴买了马匹,却并没有考虑如何把它们运回家。“整个旅行就像塞维·蔡斯电影里的场景,”他说。比希利亚学会了用计算机为他的贩马事业和父亲的豪华游艇业务制作网页。比希利亚断定计算机比养马更有前途,因此义无反顾地报考了华盛顿大学,并选修了尽可能多的学科,包括数学、物理和计算机相关学科。
2006年年末,当比希利亚往返于Google大厦和西雅图之间筹备“Google 101”计划时,他运用生意人的技巧,招募了一支组织松散的志愿者队伍。他和学院的实习生一起设计课程,还在Google公司位于学校附近的华盛顿州科克兰德分部拉拢部分同事,让他们抽出20%的时间来帮忙教课。比希利亚听从了施米特的建议,把“Google 101”集中在学生们在学习过程中容易上手的方面。“我基本想的是,什么课程我能在两个月里教会他们,同时又真正有用和重要?”他回忆道。最终他的答案是MapReduce。比希利亚十分推崇MapReduce,这是Google数据计算的核心软件。公司著名的搜索运算法为每一次搜索提供信息,MapReduce则传递出速度。它把每个任务分解为成百甚至上千块小任务,然后发送到计算机集群中。眨眼之间,每台计算机传送回自己的那部分信息,MapReduce则迅速整合这些反馈并形成答案。虽然也有一些技术具有同样的功能,但MapReduce速度更快且显示出几乎可以解决无限任务的能力。提到MapReduce,比希利亚变得十分兴奋和狂热:“我记得刚毕业时来到Google学习MapReduce,这的的确确改变了我对计算机科学乃至所有事情的想法。”他把该软件称为“非常简单却极其卓越的模型”。这个软件是由其华盛顿大学校友杰弗里·迪安开发的。因此通过回到母校教授MapReduce,比希利亚会将这个软件和“这种思考方式”带回源头。
只有一个阻碍。MapReduce曾经安全地“沉寂”在Google主机中? 而且不允许外界使用,对于“Google 101”项目也一视同仁。Google曾拿出一部分相关信息与他人共享,以开发开源版本“Hadoop”。当时的想法是在不泄露核心技术的前提下,推动自身的标准成为“云”计算的体系结构。开发Hadoop的团队属于一家名为Nutch的公司。说也奇怪,这家公司现在归入雅虎麾下,雅虎希望依靠MapReduce的衍生产物给自己的数据计算提供一点Google“云”的魔力。好在Hadoop仍然保持开源状态,这意味着Google团队能对其加以应用并可免费安装在华盛顿大学的计算机集群中。
“Google 101”一出现在冬季学期的课程安排中,学生们立即蜂拥而来选修这门课程。起初比希利亚和Google的同事们尝试自己教课,不过后来他们及时地把这一工作转交给华盛顿大学的专职教员。“他们的讲解更加清晰,”比希利亚表示。接下来的几周里,学生们学习如何调整自己的程序来适应Google计算机,并雄心勃勃地设计开发网络规模的项目,这些项目涵盖了从维基百科的编辑分类到互联网垃圾邮件的鉴别处理等各个方面。2007年的整个春天,有关这门课程的消息不胫而走,其他大学的院系也开始要求参与“Google 101”计划。很多人迫切渴望了解“云”的相关知识和计算能力,特别是在科研方面的计算。实际上在每个领域,从各种传感器、分析设备以及先进的测量工具产生的大量新数据浩如烟海,让科学家们大伤脑筋。这些数据可能用于开发新药品和疗法、制造新的清洁能源、甚至预测地震,然而绝大多数科学家缺少设备来存储和筛检这些“数据宝藏”。“我们真是被淹没在了数据里,”美国国家科学基金会的助理主任周以真(Jeannette Wing)表示。
IBM的慷慨
对Google计算能力的巨大需求倒是把比希利亚难住了。他能完成第一批计算机集群的采购安装已经算是很幸运了,可是他能像这样一次又一次、最终在每个计算机学院都装上一个微型的Google“云”吗?当然不现实。为了把“Google 101”计划扩展到全球各地的大学,各参与方必须要接入到共享的资源中。因此比希利亚需要一个更大的“云”集群。幸运之神随着IBM董事长彭明盛突访Google大厦而降临。这天成了“Google的彭明盛日”,一位IBM的研究员表示。那是一个冬日,如果要在Google园区里来场沙滩排球可能会有点寒冷,不过彭明盛中午在Google的餐厅体验到了传说中的免费大餐。随后,他和他的团队与施米特以及包括比希利亚在内的十几名Google工程师座谈交流,他们在白板上写写画画、讨论着“云计算”。IBM一直希望部署“云”系统来为企业客户提供数据和服务。与此同时,在彭明盛的领导下,IBM已经成为Linux系统等开源软件的领先倡导者。这可是蓝色巨人在软件战役中的重点,尤其是在对抗微软的战斗中。如果Google和IBM在“云”上合作,它们可能共创这种基于Google标准(包括Hadoop版本)的“云计算”的未来。
Google当然已在这个项目上先行一步,即比希利亚的“Google 101”计划。就在会面的当天,比希利亚小小的实践成为由两家技术巨头的首席执行官支持的一项重大计划的开端。当彭明盛那天下午离开Google时,比希利亚和IBM公司的丹尼斯·全就被指派组建Google-IBM的联合大学“云”的原型。在接下来的3个月中,他们在Google总部并肩作战。(比希利亚说,从那时起“云”计划从“20%时间”变成了他的全职工作。)他们的主要工作是把IBM的商用软件和Google的服务器进行整合,并装配大量包括Hadoop在内的开源程序。2007年2月,他们在加州山景城向高层领导、同时通过视频向位于纽约阿蒙克的IBM总部人员首次展示项目原型。丹尼斯·全用手机从“云”集群中下载数据,让在场人员赞叹不已。(比希利亚说,虽然与核心项目关系不大,但这的确是场很精彩的演出。)
“Google 101”计划获得了通过。这一计划是首先将“云计算”用一年时间扩展到全美的多家大学,之后在全球部署。各所大学将会继续开发“云”,创建工具和应用程序,同时培养出大批的计算机科学家来继续建设和管理“云”。那些开发者应该能在Google这样的网络公司找到工作。施米特喜欢把这些数据中心比作极其昂贵的粒子回旋加速器。“物理界只有几台粒子加速器,”他说,“每一台都十分重要,因为你如果是个顶尖的物理学家,你需要在有粒子加速器运行的实验室工作。那才是创造历史的地方,那才是诞生发明的地方。所以我想,假如你把‘云’当作由小型计算机群组成的超级计算机,那么从科学观点讲,我们拥有最能吸引人才的那种计算机。”
随着商用和科学数据量日益壮大,数据计算能力转变成一种战略种资源和一种资本。“从某种意义上说,”雅虎研究主管普拉巴卡·拉加万表示,“世界上不过有5台真正的计算机。”他意指Google、雅虎、微软、IBM和亚马逊这几家公司。他表示,除此之外,没有哪家能有相似的实力把“电流”转化为数据计算能力。毫无疑问,各种商业模式都会进化。Google及其对手能与客户合作、或许是通过交换计算能力来取得数据的访问权。它们可以在“云”中引入合作伙伴进行次级项目的开发,比如公司在2007年11月宣布的清洁能源计划。行业分析家表示,随着大型数据中心电费开支每年以2000万美元的速度上升,也只有Google能够以其智力资源和服务器容量来承担重任,以图在寻找创新能源方面取得突破。
用于研究的“云”会是什么样子?微软负责外部研究的副总裁Tony Hey介绍,他们把“云”建成大型的虚拟实验室,应用新一代管理程序配以适当人工来管理数据,分级别适度开放给研究人员。他说,授权用户将开发出新工具、补充新数据,并与各地同事广泛共享。据他预测,在这些新的实验室中,“你可以通过分析从别人那里汇集来的数据赢取诺贝尔奖”。位于加州Almaden的IBM研究运营部门负责人马克·迪安表示,短短几年内,商用和科学用途的结合将产生“云”网络,我们尽可放开想象。“与这个相比,”他说,“现在的网络微不足道。我们将会嘲笑现在的网络实在太小。”然而,如果这个“太小”的网络对于Google帝国发展都已经足够大了,那么没人能预测巨型的“云”网络可以提供什么样的机会。
2007年11月的一天,比希利亚刚刚从中国回到美国,还没来得及倒时差。他在中国的几所大学介绍了“Google 101”的计划。他的时间表排得满满当当,不仅要和IBM共同建立“云”集群,还需要处理华盛顿大学、加利福尼亚大学伯克利分校、斯坦福大学、麻省理工学院、卡内基-梅隆大学以及马里兰大学等6所高校“云”计划启动的相关事宜。此时,他把一位摄像师请到会议室,线缆和灯光占满了整张桌子。他将录制一段有关“云”教学的宣传片,这段视频或许会选择在YouTube网站发布。
埃里克·施米特走了进来。52岁的他几乎比比希利亚年长一倍,与爱徒清瘦的体格相比,他显得强壮不少。比希利亚把他引到摄像机面前并解释这个宣传计划。他们会录制采访中的音频,再拍几个施米特的单人正面镜头。比希利亚把这些镜头称为“B级胶片”。施米特同意了计划并落座下来。他想到了更好的主意,他告诉摄像师拍下全景而省略单人镜头。的确,他和比希利亚忙得怎么有时间单干呢?
本文来自CSDN博客,转载请标明出处:
http://news.csdn.net/n/20080319/114470.html
1. [PDF]
文件格式:PDF/Adobe Acrobat -快速查看
第六十六期. 2010 年10 月30 日.Google及其云智慧. 作者:斯蒂芬•贝克(Stephen Baker). 译/赵斌. 【编者按】“云”的想法,是创造一个平台,让所有大型的信...
ccga.pku.edu.cn/cms/uploads/soft/201103/79_20101253.pdf
2011年2月10日...商业周刊:Google及其云智慧. 减小字体 增大字体. 这项全新的远大战略旨在把强大得超乎想象的计算能力分布到众人手中。 这是一个简单的问题,是克里斯...
www.flatws.cn ›教程资料›程序开发›云计算-网页快照