Is it worth taking out a file on WekaIO? It seems to be disrupting the data industry

Some years ago Tarkan Maner of Nexsan likened the storage industry to a sort of digital mafiosi. They promise to offer protection, but they end up taking millions of off you through proprietary offers that can’t easily be refused. And once they are ensconced in the customer’s business, they don’t have to move fast for anybody.

To add insult to injury, storage vendors would make long self aggrandising corporate videos which proclaim themselves as philanthropists who are bringing peace to the world, levelling playing fields and democratising everything to within an inch of it life.

A new hero has emerged to stand up to these Poser Nostra, in Liran Zvibel, founder and CEO of WekaIO.

Zvibel pioneered new ways of moving mountains of complex data in a fraction of the time, by using artificial intelligence to constantly tinker away to get the best possible outcomes from the new GPU-filled servers, which are magnitude faster than traditional processing units.

The new systems need to effectively hold Petabytes of data, since today’s computer users are typically running projects that can need anything from a handful to hundreds of petabytes. At the same time they need to move gigabytes of data per second across the entire farm of GPU servers that are crunching all these big data files full of voice and video.

Zvibel pioneered techniques using the vehicle of Non volatile memory express (NVME) which, by comparison, makes data distribution by all flash arrays seem like traveling by horse and cart. By using object storage techniques, he invented a way to create unlimited namespaces. Other improvements in cleaning and tagging of data have made storage seem much simpler, while hiding the user from all the complexity running underneath the covers.

“In order to handle trillions of files per file system and billions of files per directory, we had to change the way storage works,” says Zvibel.

In order to crunch data on a grand scale, Zvibel created a system that can build up to thousands of controllers. The minimal latency NVMe devices and a network stack with a modest overhead created the possibility for new kinds of distributed algorithms.

Engineering a new file system is not an easy task, especially one that is based on a novel idea and genuine new data structures and algorithms. This is the first parallel file system to start from scratch in 20 years, which speaks volumes for the complexity of the problem.

Who is going to buy it though?

The system WekaIO is currently at version 3.1, which is usually the stage where  customers start to enjoy using it and not just those who enjoy raw technology. “It has gone through three generations, but it was ready for consumption after two,” says Zvibel

There is one impressive proof of concept customer at a huge Boston-based enterprise - presumably one that you service providers will be allowed to see. But not us. Yet.

Any other selling points for the UK channel to consider? “Even at an early stage we were able to show better results than the IBM Spectrum Scale system [the customer] had deployed for that project,” says Zvibel.

In summary, WekaIO is faster than the fastest SAN all-flash arrays you can buy today and bigger than any NAS you can find, says Zvibel.

The bad news is that customers take a long time to trust a new file system, especially when they were burned in the past by other newcomers that didn’t deliver as promised.

The good news is that HPE is now an OEM. That will be massively reassuring for the server huggers and the cloud cuddlers of enterprise IT.

WekaIO just needs some resellers to make them offers they can’t confuse.