WIll open source save the datacentre industry?

IT rivals stand cheek-by-jowl to praise Facebook’s Open Compute Project, but is the love-in a Damascene conversion or just damage limitation?

When Facebook launched the Open Compute Project (OCP) in 2011 to promote collaborative development of open source hardware for the datacentre, some observers scoffed. Why would suppliers share their intellectual capital with their rivals? 

IT engineers, by contrast, generally cheered. They understood how the software market had been transformed over the past two decades as a result of the open source movement and knew the hardware industry could benefit by following a similar trajectory.

At January’s Open Compute Summit in San Jose, Frank Frankovsy, Facebook’s head of hardware design and operations – and the driving force behind OCP – said: “Historically, those who embrace open source have been far more successful than those who fight it.” The initiative, he added, “helps us all collaborate, contribute and consume in a virtuous circle”.

But in the fairly conservative datacentre industry, it needed a big hyperscale customer like Facebook to set the ball rolling and cajole reluctant suppliers to take part. 

“This project has been driven directly from customer pull, rather than a supplier push. It highlights the growing influence that organisations of Facebook’s size have on the datacentre industry,” said Errol Rasit, a research director at analyst firm Gartner.

Kuba Stolarski, research manager for servers at analyst IDC, agrees. “Server vendors have limited involvement in the OCP, and they are not sharing their intellectual property with the community. It is the customers – those buying the servers – who are collaborating on datacentre hardware designs,” he said.

But with the weight of customer giants behind the OCP, suppliers have little choice but to respond. “They have to be involved in some capacity, because someone still needs to build those servers and they don’t want to lose out on that business,” said Stolarski. “Solutions are going to be more customised, as demanded by the customers.”

From the spirit of camaraderie evident at the summit, the push for collaboration seems to be working. As well as a raft of younger companies perfectly comfortable with the open source model, and big customers like Goldman Sacks and Bloomberg, OCP has also seen growing support from traditional, proprietary suppliers.

Open Compute Project goes from strength to strength

From the project’s humble beginnings in the Facebook canteen, it has steadily been recruiting big names spanning areas including servers, semiconductors, storage and networking. These include Intel, HP, IBM, EMC, Seagate and – this year’s coup – Microsoft, which has contributed its cloud server specification to the open source community and pledged to be a committed participant in the OCP.

Historically those who embrace open source have been far more successful than those who fight it

Frank Frankovsky, Facebook

“Microsoft’s participation does herald potential for increased participation from other large names, but what shouldn’t be ignored is that participation is a two-way street: to share and to receive. So there is benefit for many organisations to participate in programmes like this, much as there would be to participate in open source software code contribution,” said Rasit.

Microsoft’s involvement will certainly put more pressure on hardware manufacturers to play ball, and could even prompt other cloud giants to throw their weight behind OCP. “Amazon would be a big win and I can imagine it would be possible for it to join,” said Stolarski. “Google seems unlikely, but if it joined that would be a big coup.”

Addressing the summit, Frankovsky was characteristically bullish about OCP’s achievements. “We now have 150 official members spanning a cross-section of customers and suppliers in North America, Europe and Asia-Pacific,” he said, adding that regional OCP chapters were organising themselves across the globe in Japan, Europe, Australia, the Philippines, South Korea and elsewhere.

In addition, the scope of the OCP is fast broadening to encompass open source networking, storage, solution consultancy, certification and startup incubation. “The work we are doing together is fundamentally transforming the industry,” Frankovsky went on. “This is no longer an industry that’s one-sided. Suppliers and customers are working together to say what happens next.”

Can the momentum continue?

Andrew Donoghue, senior analyst for datacentre technologies at 451 Research, is more sceptical about the reasons why the big names are getting on board. “It looks good from a marketing perspective and helps to curry favour with potential customers like Facebook,” he said. “Big suppliers like Microsoft, Intel and others will co-operate – and pay lip-service – until they see some kind of threat to their existing business.”

Yet those taking the stage at the Open Compute Summit were generally IT engineers, not marketing executives. To anyone watching, their support for the project and willingness to participate in a true open source spirit of sharing for mutual benefit seemed perfectly genuine. Stolarski believes this engineering focus is key to the OCP’s success. “It allows engineers across companies, both customers and suppliers, to share best practices and work together towards solutions... At the same time it can accelerate development by putting effectively more resources on a problem and then letting everyone share the solution,” he said.

More on the Open Compute Project

But Donoghue doubts it can last. “Engineers may drive things initially, but they usually get co-opted by marketing along the way. I am sure the same will happen with OCP. The trick is how long they can sustain this engineering-led ethos as the organisation grows and companies such as Microsoft – with lots of marketing resource – get involved,” he said.

“The danger is that as OCP grows, the open source and engineering drive behind it will get gradually diluted and it will lose much of its ability to disrupt traditional approaches to hardware design and development.”

Others, though, think that collaboration is the only way forward if datacentres are to become sufficiently efficient to handle the vast influx of data that is on the way. Having hyped the coming business and social benefits of big data and the internet of things to death, the industry needs to be sure it can deliver the back-end datacentre infrastructure necessary for this vision to become reality – rather than face derision for another piece of overblown hype. 

As Facebook’s Frankovsky told delegates: “In this industry, there’s always historically been a pretty huge divide between what we can envision and what we can accomplish. A lot of the time, that’s because we’re focused on gratuitous differentiation.”

Ian Drew, corporate vice-president of ARM, made a similar point in his presentation to launch the company’s new 64-bit server standard. “In three years’ time there will be more exabytes of data coming into datacentres from mobile phones every day than there are grains of sand on this planet,” he said. 

“There are also going to be hundreds of millions of wearables, connected cars, smart TVs and other devices. There’s a huge slew of big data coming and organisations need to get ready for that. You need to have a model where one size does not fit all.”

The danger is that as OCP grows, the open source and engineering drive behind it will get gradually diluted and it will lose much of its ability to disrupt traditional approaches

Andrew Donoghue, 451 Research

“Innovation is important to ensure scalability, but the business model around it won’t be proprietary. The 21st century business model will be about working together – scalability by collaboration, rather than a dominant player in the industry saying ‘this is the way it will be’,” said Drew.

Of course, that’s not a surprising position for ARM to hold, since its business model is built around its ecosystem of partners, but it is an argument that chimes with many customers. And since those customers now have the OCP to act as a rallying point, the momentum may be unstoppable.

Gartner’s Rasit, however, is more equivocal. “The open source movement is largely focused on improving datacentre infrastructure hardware to increase efficiency and reduce cost. The incumbent suppliers’ goals are similar, but they increasingly focus on management software optimisation and integration value to achieve economies of scale. Both have a place in the market as users' needs are not ubiquitous and many will see value from differing, sometimes opposing, styles of infrastructure design,” he said.

But, as Facebook CEO Mark Zuckerberg told delegates when he took the stage, the company has saved “more than $1bn in the last three years” through it’s open source approach. That figure should serve as a stark warning to any datacentre infrastructure supplier that thinks it can rely on locking customers into solutions without ensuring it also delivers the efficiency they need at a price that is competitive against similarly efficient open source offerings.

As Stolarski observed after attending the summit: “The bottom line is that OCP members are energised and putting a tremendous amount of resources and effort into the project. If that groundswell of collaboration doesn’t have a positive impact on server design and datacentre efficiency, then I don’t know what could.”

Read more on Datacentre energy efficiency and green IT