Spotify on benefits of ditching its datacentres and going ‘all-in’ on Google cloud

Spotify has opened up about how its ongoing push to go “all-in” on the Google Cloud Platform is progressing, and why adopting a multi-cloud strategy would not work for the firm.

The music streaming site revealed plans to close the private datacentres it runs its service from in February 2016 in preparation for moving its back-end infrastructure into to the Google public cloud.

Speaking to Computer Weekly at the Google Cloud Platform Next Conference in London, Nicholas Harteau, Spotify’s vice-president of infrastructure, said with “250 to 300” individual microservices to migrate, the process will take “quite some time” to complete.

“Managing a global datacentre footprint is something you had to do in past times, and now that it’s something you don’t have to do yourself, I don’t think there is a strong reason to do so,” he said.

These microservices collectively contribute ensuring the “business logic” that powers Spotify remains up and running.

“They vary quite widely [in function], from the service that holds all of the user accounts to another that annotates playlists with the appropriate metadata for a specific Samsung TV, for example,” said Harteau.

“Some of those services have storage components, while some are stateless, while others are for very specific parts of the product and others are quite broad. We have instances of those services spread around the globe wherever we have customers.”

The company needs to deliver these services from locations close to where its users are to ensure there is no lag in responsiveness when they use Spotify, which is why it makes sense to run that from the cloud, he said.

It is also one of the reasons why it will continue to rely on other third-party content delivery network providers to locally host content close to where its users are.

“When you click play on Nirvana or Justin Bieber, as your tastes allow, that is fetching content from some place on the internet and bringing it back to your phone,” said Harteau.

“For that purpose and that part of the service, we have the strategy that is to partner with the best providers of content delivery services in any given market, which include Amazon Cloud Front, Fastly, Akamai and others.”

Aside from that, Spotify has no interest in expanding its public cloud use by sourcing services from multiple providers, because of the unnecessary complexity this would bring, said Harteau.

“We consider focus our most precious resource, so we don’t want to have to figure out how to make our systems or Spotify work brilliantly across a bunch of different cloud suppliers. The big driver for us, when moving to the cloud, was basically to do fewer things,” he said.

This attitude has seen the firm extensively automate its existing datacentre estate, so its engineers can spend less time managing infrastructure and contribute more to improving the end product.

“Managing datacentre footprints in Europe is a complex task, and the myriad data privacy laws make it an interesting one. Trying to do that globally is also quite complicated,” he said.

“The more focus Spotify engineers can give to shipping a great product – and learning from our users about how the product is used and finding places to innovate – the better. Those are high value activities for us.”