Cockroach Labs: The killer factor in the dead heat cloud zone
People don’t like the poor cockroach.
Although they’re widely despised, generally spurned and definitely a very unpleasant addition to a bathroom, shower, bed or couch once you’re on vacation somewhere tropical, the cockroach does have some redeeming qualities.
The cockroach is known for its ability to populate and colonise, be tough and resilient and for its ability to use the resources around it and flourish… even if those resources are the rim of a toilet seat.
As noted in this blog discussion, this is some of the thinking behind why Cockroach Labs is called Cockroach Labs i.e. it’s about an ability to be tough and functional.
The latest chirps from Cockroach Labs (Ed – do cockroaches chirrup and chirp?) is the 2022 Cloud Report from Cockroach Labs, published on June 14th 2022, which aims to evaluate the performance of AWS, Microsoft Azure and Google Cloud Platform for common OLTP (OnLine Transaction Processing) workloads.
Researchers at Cockroach Labs ran benchmarks on 56 different instance types and 107 discrete configurations to find that all three providers offered price-competitive offerings.
A statistical dead heat
Looking at the performance of the top cloud providers, AWS, Azure and Google Cloud were in a statistical dead heat when it comes to price and performance. All three providers had at least one instance type and storage combination in the $0.04-$0.05 reserved $/TPM range.
McClellan: More is more, we’ve scoped out a bigger field for the users.
Additionally, for the first time, AMD’s Milan processors outperformed Intel.
In past years, Cockroach Labs saw Intel lead the pack in overall performance with AMD competing on price-for-performance metrics. Now in its fourth year, both the overall performance leader and the price-for-performance leader were AMD-based instances.
According to Cockroach Labs, it expanded and improved its OLTP benchmarking this year, running more than 3,000 different OLTP iterations leveraging CockroachDB and adjusted its testing methodology to narrow the variation of results across runs.
“In this report, we aim to provide an unbiased picture of the performance users are paying for when they provision a specific configuration in one of the clouds,” said Keith McClellan, lead author on the report and director of partner solutions engineering at Cockroach Labs. “If I had to describe the report in a sentence I would say, more is more. This year we tested more instance types, node sizes and 3X as many runs as last year. Altogether, it adds up to more depth.”
Among its suggested findings, Cockroach Labs thinks that the ‘hidden’ costs of storage and data transfer can have a larger impact on total cost than the price of the instances themselves.
Cost killer in consistent clouds
It cautions users that the cost of running a workload is much more influenced by the cost of storage and networking than the cost of an instance, especially for consistent workloads.
“If there is one point to take away from this year’s report, especially if I were a CIO or CTO building a globally distributed application concerned about cost when picking a cloud provider(s), I would focus on the network transfer cost where I was planning to operate. Our findings really shine a light on each cloud’s total cost to operate,” said McClellan.
All said and done then, the one bug that we all seek to kill when we’re on holiday in Greece or Spain (other holiday destinations are available) is actually perceptive enough here to uncover what may perhaps be the killer factor in cloud instance operations right now.
