Photobox gets snappy with mass-data migration to AWS cloud

Photobox Group has completed the migration of 9PB (petabytes) of image data to the Amazon Web Services (AWS) cloud five months ahead of schedule using the cloud giant’s Snowball appliances.

The online-only image printing company holds about 6.5 billion images, uploaded by customers who use its services to order physical copies of their digital photographs. The images were previously housed in two colocation datacentres in different parts of Europe.

But Photobox wanted a more durable setup that would also help it deliver a faster website experience to its customers and make it easier to manage the huge amounts of data it handles, so it decided it was time to move to the AWS public cloud.

Photobox Group’s chief technology officer (CTO), Richard Orme, tells Computer Weekly that the company’s on-premise setup meant its technology teams were spending too much time managing infrastructure and not enough on building projects that could support the organisation’s future growth.

“We were spending a lot of time as a technology function building infrastructure just to keep up with the pace of photos that we were ingesting, or then looking at compression technology used to shrink down the size of the photos without losing any content,” he says.

“So a lot of the effort we were putting in was not necessarily related to growth, it was just keeping the organisation going. It was clear that we needed to move our resources into developing new features for customers and developing new products.”

The company first thought that moving the data from its datacentres into the AWS S3 cloud storage service would take about 12 months using the cloud giant’s Snowball mass-data migration appliances.

Introduced in 2015, the Snowball appliance is touted by AWS as a way to eradicate the high data transmission costs that enterprises can sometimes incur when moving data to the cloud. The Snowball system sees the customer upload its data onto the appliance, which is then shipped, via courier, to an Amazon datacentre, where it is transferred directly to the cloud.

“Each Snowball has 100TB of capacity, and we used 90 of them,” says Orme. “It’s a staggering amount of data – we hold Europe’s largest consumer photo database.”

But when the project kicked off in January 2018, it ran into some teething problems. Photobox found the Snowball appliances were filling up too quickly, and it was taking longer than expected to transfer the data.

In response, Amazon accelerated the development of a feature it had in the works that would enable thousands of Photobox’s image files to be compressed down into a single file transfer.

Once the files reached Amazon’s datacentre, they were automatically decompressed and uploaded to its servers.

At the same time, the Photobox team was working on storage- and software-level improvements to the in-house system it had created to transfer the data from its infrastructure to the Snowball.

“They changed their product roadmap for us,” says Orme. “They accelerated some development work on the Snowball devices, which meant we were able to get the project not only back on track, but we were actually going faster than we thought we would.”

So much so that the project ended up taking just seven months to complete, which has helped Photobox to secure some “significant cost savings”, according to Orme.

“The key to finishing this project successfully was putting the right cross-functional team in place, including engineers and specialists in-house, working very closely with AWS,” he adds. “They tackled the early-day issues with such success that we ended up finishing the transfer in July – five months ahead of schedule.”