Scality object storage scales to 400k mailboxes for NZ provider

New Zealand-based email hosting and protection specialist SMX has deployed more than 2PB of Scality object storage, spurred by the need to support the roll-out of circa 400,000 mailboxes for a customer.

The move to Scality allowed SMX to move away from an existing Cassandra, VMware and IBM storage stack that would not have been able to scale to the new workload.

SMX is based in Auckland and specialises in hosted email with emphasis on email security against malware, viruses etc. It provides these services to SME, enterprise and service provider customers, and used to do so largely from its Cassandra and IBM storage infrastructure.

But, a contract with a service provider to deliver 400,000 mailboxes meant the existing setup would need to be upgraded.

Richard Gray, solutions architect at SMX, said: “Cassandra is a pretty good database but not good for bulk storage of data. It needs a 50% free space overhead, for one thing.”

“It is designed to run on Linux server clusters rather than sticking it all on a traditional storage array, so the problem was not with the IBM storage – more that Cassandra was not a good fit for scaling from 50,000 to 400,000 mailboxes.”

There were several constraints of the expected workload that meant object storage would be a good fit, according to Gray.

First, SMX knew the customer wanted all data held in New Zealand, especially as none of the big three cloud providers – AWS, Azure, GCP – have a physical presence there. Second, Gray’s team judged object storage to be a good fit for email, with lots of small objects and “no need for file system semantics on top”. Third, they wanted object storage on-premise

The choice came down to Scality or the open source Ceph, and the key decision point for SMX was that the mail platform planned for use – the open source Dovecot email server – was not judged to be a good fit with Ceph.

SMX eventually deployed a 10-node Scality cluster with capacity of 2.3PB and a 3x replication factor, all based on Cisco 3260 hardware. These take up to 36 drives each, most of which are spinning disk HDDs, but with flash drives for indexing.

The overriding benefit is that the new Scality-based infrastructure has allowed SMX to take on such a large new customer. Apart from that, Gray is full of praise for its simplicity and reliability.

“Accessing stored objects is very simple, with very simple addressing that is easy to understand,” he said. “In terms of performance, we’ve never had any issues or had it near its limits.”