Excelero and BeeGFS offer next-gen HPC based on NVMe flash

NVMe flash storage pioneer Excelero is making a tilt at the high-performance computing (HPC) market with parallel file system BeeGFS to offer very speedy, high-throughput, software-defined clusters.

Excelero is software-defined storage based around NVMe flash storage. Customers can deploy its NVMesh on multiple commodity drives to create a pool of NVMe storage that can be configured as one or a number of drives and which connect via NVMe-over-fabrics.

Ballpark figures for performance over HDD-equipped storage are 100x, according to Excelero CEO Lior Gal.

The new venture sees Excelero and BeeGFS, a parallel file system marketed by ThinkParQ, offered as a bundle aimed at HPC use cases.

Both products are software-based and customers can choose their own commodity hardware.

BeeGFS is a scale-out parallel file system developed at the Fraunhofer Centre for HPC in Germany. It was developed to overcome performance and scaling issues with Lustre and to be less costly than IBM’s GPFS (now Spectrum Scale).

The package provides the ability to build HPC clusters with extremely fast storage and a file system certified to work with it.

Use cases targeted are HPC workloads such as scientific data processing, for use as scratch data stores and burst buffers in those workloads, and including HPC processing with GPU processing.

According to Gal, this kind of hardware and software combination is the “next generation” for HPC, with an NVMe-equipped HPC cluster 100x faster than appliance-based approaches with spinning disk HDDs.

HPC used to mean mostly large batch processed calculation workloads done on mainframes and supercomputers, often in scientific work, weather prediction, oil and gas prospecting, and so on.

But the growing use of analytics in a wide variety of sectors has pushed HPC into the mainstream, where analytics are used alongside trading financial trading and commercial web activities.

The rise of AI and machine learning in data analytics, where large amounts of processing power need to be applied, has further boosted the recent rise of HPC, and it is this type of work that the Excelero/BeeGFS partnership aims to tap.