ATA over Ethernet offers SAN storage on a budget

If you want block-access shared storage, you need some kind of SAN. Overwhelmingly, your choice has been limited to Fibre Channel (FC) or iSCSI, but in the background another technology, ATA over Ethernet, is staking a claim as a viable alternative.

ATA over Ethernet, or AoE, brings enterprise-level performance and lower cost compared with FC or iSCSI SAN hardware. It works analogously to iSCSI, which wraps SCSI commands in TCP packets, but at a lower network layer. Instead of using TCP as the transport mechanism, ATA over Ethernet encapsulates ATA block-level commands into Ethernet frames at network layer 2. Removing the SCSI command processing overhead lowers latency and processing requirements, making ATA over Ethernet arrays cheaper to make, buy and operate.

Coraid has claimed that its ATA over Ethernet arrays "offer up to a 5x to 8x price/performance advantage over legacy Fibre Channel and iSCSI solutions" and remove the need for the layer 3 multipathing and software when using IP networks.

ATA over Ethernet users have reported the technology is trivial to set up, while analyst firm ESG's lab validation tests have shown that acquisition costs are less than half those for an equivalent iSCSI SAN and less than 20 percent of those for a Fibre Channel SAN.

Operating at layer 2 also removes the possibility of routing, which works only at layer 3 and above. This limits scalability in a distributed network, although some users have argued on forums that this is not an issue for them. It also means the protocol is inherently more secure, as outside agents cannot connect to it directly.

While the name ATA over Ethernet suggests that ATA disks are the target for the protocol, Coraid connects a number of types of drive mechanisms in its array products. The company offers its EtherDrive range of SAN products in a number of sizes, from 2U to 4U, containing up to 108 TB per box, each of which can house SAS, SATA or SSD drives, accessed over either six 1 Gigabit Ethernet (GbE) or four 10 GbE ports.

ATA over Ethernet was open-sourced by Coraid in 2003, and it has been a native element of the Linux kernel since Version 2.6.11 in 2005. It has, however, seen little adoption by other vendors, so Coraid remains its sole commercial vendor.

The fact that ATA over Ethernet has only one vendor is a key reason why it has not been adopted on a widespread basis. Despite the lower costs of ATA over Ethernet, the safety factor involved in buying from well-known brand names has trumped its performance and cost benefits. However, as downward cost pressures, upward data growth rates and an uncertain global economy all seem set for the foreseeable future, ATA over Ethernet's cost advantage is likely to persuade more organisations, especially those in the public sector, to consider piloting this technology.

Palmer's College adopts ATA over Ethernet

For Dan Byne, network and systems developer at Palmer's College in Essex, the solution to his storage problem was obvious and hugely cheaper than the alternatives.

The sixth-form college's IT department, which serves more than 2,500 users' laptops and desktops, planned to migrate its servers from direct-attached to network-attached storage for a new virtualised infrastructure, partly paid for with a £115,000 central government loan for greener IT.

The college’s existing system was "a cluster of IBM storage and one SAN box clustered across a couple of servers delivering Windows home directories," Byne said.  The college was about to purchase an EMC iSCSI SAN when a neighbouring college suggested it consider Coraid's systems.

"We took a look, we were convinced and we signed up for it," Byne said, despite that the EMC reseller subsequently offered a significant price reduction.

The key deciding factor for Byne was the performance boost that Coraid's EtherDrive SRX systems offered. "Our tests showed that installing an OS is much quicker, and Windows Server 2008 startup time is 10 seconds, for example.

"FC SANs are comparable in speed and throughput—though our tests showed that in fact Coraid was faster—but there is not one manufacturer who will even get close to Coraid's price/performance advantage," Byne said. He said he also liked Coraid's simplicity and speed of setup as well as the fact that it can be managed using a command line interface.

"Now we run vSphere 4.1, and it all sits on Coraid appliances," Byne said. "We have two SRX3200 SANs. The virtual machines do everything—databases, SQL servers, file and print, and they host our intranet site. Everything is pretty much virtualised." The storage is mirrored for redundancy reasons over a 300-metre fibre link to the other side of the 1-acre campus.

"Coraid is so much simpler than iSCSI. It made much more sense to go with this," Byne said. "Coraid covered everything we were looking for with no single point of failure. We might have skipped it because a lot of people haven't heard of it, and lots of resellers go for the EMC route. IT managers will always be conservative on their storage, but with significant budget cuts in all areas of government, they need to have a look outside the big players in the storage field. They will be pleasantly surprised."