It’s Time to Raise the Bar on Observability

Observability has come a long way in the last 30 years – from early service monitoring tools to today’s advanced platforms like Splunk, DataDog, Dynatrace, New Relic and others that ingest massive volumes of data across infrastructure and applications. However, for many IT teams, observability goals are still stuck in the world of monitoring individual components and putting out fires. The speed of business change demands that we aim higher.

The observability conversation needs to move from IT-centric monitoring to end-to-end service insights tied to real business KPIs. The technology now exists to cost-effectively store and analyse vast data sources in one place in almost real time. Combining this expansion of data with machine learning gives IT teams a valuable window into the health and performance of systems that underpin key business services.

This unified observability landscape is likely to become essential for modern IT for several reasons:

• It provides insight into root cause analysis by seeing connections and patterns across domains to prevent those lengthy war-room arguments over the source of problems. The ultimate goal should be to diagnose and solve issues preemptively before they impact users or business services.
• It needs to connect directly to business priorities. Modern observability platforms can give technical and business stakeholders shared visibility and language around what’s important, providing the opportunity to create business KPIs rather than technical metrics. This will minimise the potential for IT-Business misalignment.
• It will enable us to build informed what-if scenarios. Once we have full observability, the data has the potential to drive projection models of the impact of proposed changes to resources or configurations. This will become essential intelligence for good decision making in a world of rapidly changing business demands and extremely flexible IT resources, with variable cost, carbon footprint and response time impacts.

The strategic vision should be that observability data and insights become ubiquitous and consumable across the organisation in the future via AI and conversational interfaces. But cultural and process changes will be required to make this a reality. Stakeholders need training in asking good questions and setting expectations. Policies should focus on querying around specific business goals rather than open-ended exploration.

In summary, observability technology and best practices have hit an inflection point. Leading IT teams now have the capability to enable observability on a far more ambitious scale than before, achieving unified views across technical domains with direct business context. The benefits for automation, optimisation and planning are enormous. For businesses racing to adapt and compete, this new observability capability will soon become tomorrow’s table stakes. Now is the time for forward-thinking IT leaders to embrace this shift, and promote the benefits to the business.