snvv - Fotolia
BFI deploys SpectraLogic tape in PB-scale digitisation project
British Film Institute got SpectraLogic tape and Black Pearl when it wanted multi-petabyte capacity and the ability to write tape archive of 10,000 movies into its workflows
The British Film Institute has deployed several petabytes of SpectraLogic tape storage with S3-capable Black Pearl boxes in a project to digitise 10,000 films in its archives.
The SpectraLogic deployment won out over hierarchical storage management (HSM) products because of its ability to fit with in-house-developed workflow applications, while savings made compared to the likely spend on HSM allowed it to buy two tape libraries instead of one.
The project – called Unlocking Film Heritage – required a new storage system that would integrate with workflow applications and accommodate predicted data growth.
After evaluating its options, the BFI deployed two SpectraLogic T950 tape libraries at opposite ends of its Hertfordshire conservation site, with the help of system integrator OvationData. Each library uses tapes in a different format, one with LTO-6 and the other with the IBM format TS1150 to allow for media diversity on top of geographical spacing.
Initially, it has rolled out 2PB in each library, but, for example, the IBM format library can scale to 20PB, with 8PB expected to be used in the next couple of years.
In front of those, it has deployed two SpectraLogic Black Pearls, which provide access to data on tape via S3 commands, with disk cache of 50TB as a staging area.
The BFI tendered for an archive solution through a process called Competitive Dialogue, where the potential customer sets out ambitions and frameworks for the project, rather than detailed technological requirements.
The year-long process resulted in a choice between HSM platforms and SpectraLogic’s Black Pearl as a way to access and manage data held on tape libraries.
HSM was rejected by BFI head of data Stephen McConnachie as “old school technology”.
“It’s a proprietary solution, with expensive licensing and a single avenue to get stuff in and out of the tape libraries,” he said.
“The Spectra model is a new paradigm. Black Pearl uses RESTful APIs to make storage and retrieval very simple from anywhere on the network, using S3.
Read more on archiving
- Have cloud archiving services reached a tipping point that makes them cheaper and easier to manage than in-house tape archives?
- Movie maker ditches industry-specific hardware and deploys Black Pearl, which gives NAS-like file access from tape.
“It standardises the storage framework with S3 puts and gets and has allowed us to develop our own integrations that can get stuff in and out of the tape repositories.
“Also, here in London you can almost literally throw a stone and hit an S3 developer. HSM needs a much less available skillset.”
The BFI has, for example, developed an automated ingest application in Python – for which there is a SpectraLogic SDK – that works with Black Pearl to process files with minimal human intervention.
McConnachie added: “We can ask it questions all day long, every day, and write our own integrations to it.”
He said he could not put a monetary figure on cost savings, but avoiding paying for an HSM system had allowed the BFI to buy two SpectraLogic tape libraries rather than one.
“The benefit of not buying an HSM system is huge,” he said. “It’s traditionally one of the key areas of major expense in this kind of project.”
