photonetworkde - Fotolia
AIOps for storage: Potential huge benefits, but lots of lock-in
Applying artificial intelligence and machine learning to storage can help to predict bottlenecks, diagnose I/O issues and recommend upgrades, but suppliers’ products are largely limited to their own hardware
The growing complexity and diversity of IT is a challenge for CIOs and their teams. The demands of the datacentre, cloud infrastructure, containerisation, and security and compliance all put more pressure on IT management and operations.
At the same time, it is difficult to recruit and retain experienced staff. This has prompted IT departments and suppliers to look at ways to automate information services and infrastructure management, including for data storage.
One answer is to use machine learning (ML) and artificial intelligence (AI) to take up some of the workload. AIOps – or AI for IT operations – is one tool promoted by the industry to deal with complexity, to optimise systems and maximise uptime. It also plays an increasingly important role in storage management.
AIOps has developed rapidly in recent years because of improvements in ML and AI processing – including in the cloud – but also because IT systems are increasingly able to be monitored in real time.
But although everyone wants more visibility into their IT operations, the vast volumes of log data generated by modern hardware, on-premise and in the cloud, could overwhelm IT teams.
AIOps makes use of that data through analytics engines to predict peak workloads, bottlenecks, capacity limits and failures, and to flag the need for maintenance and upgrades.
For storage, AIOps promises to help firms with resource allocation, to make best use of available capacity and, potentially, move data between storage tiers and/or to the cloud.
AIOps promises to do this more quickly and perhaps with more accuracy than human system admins. It also allows organisations to scale up their digital operations without hiring more staff. It can be integrated with service management, dealing with users’ tickets, and DCIM (datacentre infrastructure management).
“The way I describe AIOps is an overarching technology that sits above the operational domain expert systems and can correlate data and feed that back as actions to the operational teams,” says Roy Illsley, chief analyst at Omdia. “That can be manual actions or it can be automated actions.”
What is (storage) AIOps, and what can it monitor?
Within IT management, AIOps sets out to make use of data generated by servers, networking equipment and storage arrays – but it aims to be more than a simple monitoring tool. By using AI, organisations can gain insights into system health, but also see how systems can be optimised.
This, according to GigaOm analyst Enrico Signoretti, is equivalent to putting “storage in autopilot mode”.
As suppliers added sensors to their systems, they could provide more up-to-date status information. They then tied this to predictive analytics and, later, automation.
“They use machine learning, and they feed their machine learning algorithms with all this information,” says Signoretti. “So now they have automation that makes better suggestions for what is happening in the system and what you should do when something is happening. All of this improves usability of the system.”
Read more on storage AIOps
- Podcast: How storage AIOps is revolutionising hybrid cloud ops. We talk to Tintri about storage and AIOps, which is the application of AI and machine learning-based software to monitor storage infrastructure to be able to detect patterns and predict failure, the need to upgrade, and so on.
- How to use AI in storage management. Organisations can use AI for storage management and AIOps to learn where to make improvements in their storage infrastructure. However, take note of the benefits and drawbacks.
Storage AIOps monitors standard measurements, such as utilisation, I/O activity and latency. By adding a degree of ML or AI, systems go beyond simply serving up raw data, and can flag unexpected events to a human analyst, predict when a component might fail or when a system needs more capacity.
Omdia’s Illsley describes this as “domain expertise”, and suppliers in other sectors, such as networking, also have their own AIOps capabilities.
The real power of AIOps for storage comes when combined with data from other parts of the IT system.
With storage, AI offers the prospect of spotting bottlenecks before they become issues, such as when data is moved to a lower-performance storage tier to make better use of capacity. The AI system then alerts IT managers and suggests changes, or even makes configuration changes by itself.
AIOps can also allocate workloads, including storage volumes, across different types of infrastructure. This comes into its own for firms that run hybrid architectures, where AIOps can manage the transition from on-premise to cloud, or between cloud tiers. But, to work, such systems need to integrate with cloud suppliers’ management application programming interfaces (APIs), and have an understanding of their pricing models.
Benefits and potential of AIOps
AIOps promises improved system usability, reliability and efficiency.
This comes principally from reduced maintenance downtime and reduced failures, and also from better allocation of compute and storage resources. This is especially useful for organisations that have a large number of separate systems, virtual machines (VMs), or that are starting to move towards containerisation for production.
AIOps becomes more useful as systems become more complex. Replacing dozens of servers with hundreds of VMs already places heavy loads on IT admins. Moving to potentially thousands of containers might not be possible at all without automation.
The greatest benefits of AIOps, however, come from integration across the whole of a firm’s IT estate.
This – for now, at least – is beyond the reach of most storage suppliers’ AI tools, which only work with their own products. The issue is whether they can talk to competitors’ systems, to compute and networking tools, or cloud management systems. As Omdia’s Illsely puts it, AIOps suppliers need to be the “neutral Switzerland”.
Storage suppliers will need to convince their competitors, and most or all of their customers, that they will work equally well with other suppliers’ tools. Alternatively, CIOs can look at hardware-independent monitoring tools, such as Splunk, Datadog or ServiceNow.
And, as GigaOm’s Signoretti points out, storage AIOps suppliers’ cloud capabilities are still limited – they need to develop cloud and hybrid capabilities. He expects these to grow, to allow firms to federate storage across suppliers, and according to policy. “We are heading to that type of scenario, but it will take some time,” he says.
Storage vendors with AIOps capabilities
Dell CloudIQ: Currently covers all Dell EMC storage and PowerEdge servers, as well as the firm’s hyperconverged systems and networking hardware.
HPE InfoSight: Supports ProLiant and Apollo servers and Alletra, Primera and Nimble storage, as well as HCI.
IBM Storage Insights Pro: Covers a wide range of systems including IBM’s own Flash storage, as well as Spectrum Scale and Cloud Object Storage. Also supports some from Dell EMC, Hitachi Vantara, NetApp and Pure Storage.
Infinidat InfiniVerse: Supports InfiniBox and InfiniBox SSA.
NetApp Active IQ: Covers OnTap, E-Series and StorageGRID storage, as well as integration with NetApp Cloud Manager.
Pure Storage Pure1: Pure1 manages all Pure arrays (FlashArray, FlashBlade and Portworx) storage. Pure1 Meta provides a full-stack analytics capability.
