IP surveillance: The storage it needs, on-premise and in the cloud

Surveillance is changing

Monitoring and retention of video camera footage still forms a central core to the world of surveillance, and so a key requirement of storage in surveillance is the ability to handle sequential I/O well.  

Meanwhile, surveillance is increasingly being supplemented with AI  – or so-called vision intelligence – to bring actionable insight to camera-captured imagery.

So, where once security cameras would simply have been monitored by humans and retained as a record, movement of potential interest can be identified and alerts raised, for example. That’s of obvious utility to physical security where systems can be primed to recognise an intruder by size, shape, movement, and so on.

But AI has also been implemented in a wide range of other surveillance applications such as phasing traffic lights to improve vehicle flow, monitoring retail centre footfall to adjust product placement, and being able to recognise concerns over wellbeing – such as a fallen patient – in healthcare settings.

But no matter what the application and level of AI applied to it, sequential I/O will likely be the core storage performance characteristic required for surveillance systems.

However, not all apparently sequential I/O is as smooth as you think when it comes to video. Some video codecs, for example, comprise different types of frames that represent full images and differential changes that contain only changes portions. A hardly changing image would see little in the way of change, until a person walked across the camera’s view, which means I/O flow would be quite wave-like.

While the requirements of image capture are largely sequential, the application of analytical and machine learning functionality is likely to involve a whole load more randomness.

That’s because while ingest of an image is pretty sequential, reading and comparing patterns within images will make reference to potentially large amounts of existing patterns already saved. The AI side of things is potentially quite random in its I/O needs.

What storage media are suited to surveillance?

Spinning disk HDD

With sequential I/O forming the vast bulk of surveillance traffic, spinning disk HDDs are predominant in the field. Spinning disk is well-suited to sequential I/O as the mechanical nature of the drives means time savings are made when read heads don’t need to cycle in and out of different locations on disk platters.

Drives can be SATA, or the more performant and more expensive serial-attached SCSI (SAS), in terms of back-end connectivity. Both will work well with the main job of video ingest.

Solid state and NVMe

However, for the kind of heavy lifting that analytics requires, solid state storage where random access of storage cells is done electronically rather than mechanically doesn’t incur time penalties.

This kind of media means flash, or NVMe flash, perhaps with enhanced processing power from a GPU card will possibly be required.

That was the case with Hong Kong-based cloud service provider Vivavo, where it leveraged NVMe-based storage for facial recognition on street cameras.

Solid state is a lot more pricey than spinning disk, so it will often make sense to have a tier of flash where data being processed is held, while HDD is fine for bulk storage of video.

Tape

Tape is also a potentially useable bulk storage medium, and when tapes are not in use they consume no power, unlike HDDs. But access to data held on tape is a lot slower than other media. Automating that via use of LTFS – a way of layering a NAS-like file system on top of tape – would be one option. Using tape as another tier for bulk storage is another way to incorporate it.

Cloud storage

Finally, there is cloud storage, which in some senses may be an ideal medium for video surveillance footage and analytics, which can be leveraged in the form of as-a-service offerings in the cloud.

But there are drawbacks too, such as the bandwidth required to upload video that could multiply as the number of cameras increases. There are also charges for the retention of data in the cloud to be considered. Compliance may also be an issue if images are stored outside particular jurisdictions.

Surveillance and storage on-site

The easiest way to get IP surveillance is by buying a network video recorder (NVR) that comes with everything you need – cameras, servers, connecting hardware. These may be well suited to many smaller businesses, but may also be limited in terms of expansion, customisation and integration possible.

When it comes to HDDs for surveillance storage, any enterprise drive will work. But with retention of large amounts of imagery for long periods with potentially small amounts of access, higher capacities with less rigorous performance requirements fit the bill. That most probably means 7,200rpm SATA most of the time.

Some hard drive vendors, such as Seagate with its Skyhawk AI products, sell HDDs which they claim to be optimised for surveillance workloads. These are SATA drives, with an emphasis placed on MTBF (mean time between failures); 2 million hours in the case of the Seagate product.

Surveillance moves to the cloud

Increasingly, surveillance is moving to the cloud, in so-called video surveillance as a service, in which the customer just has cameras on-site plus the ability to connect to the internet.

With so-called vSaaS one big trade-off is ongoing data storage costs against the one-time outlay for an NVR product. Having said that, cloud costs are likely to be more predictable than spend on on-site storage capacity or dealing with failed components.

Issues could also arise if internet connectivity is broken and images are not captured. Hybrid systems have been developed that use some on-site storage to counter that eventuality.

A big plus for cloud-based surveillance is that the added intelligence of AI can be added as a service to existing provision.