Monitoring did not emerge yesterday. It has been part of the IT toolkit for decades — and in every era, it has done the same thing: kept pace with complexity.
When infrastructure was the primary challenge, we monitored infrastructure. When applications moved to the fore, we introduced APM tools. When cloud and microservices fundamentally exploded the complexity of system architectures, the concept of observability was born.
That evolution has not stopped.
Organizations are running more AI in production than ever before — chatbots, decision-support systems, automation pipelines, LLM-based workflows. AI is no longer a standalone project: it has become embedded in business operations.
This has created a new challenge — one that the next step in monitoring evolution is built to address: AI Observability.
The Stages of Evolution
To understand what AI Observability is and why it is needed, it is worth tracing the path that IT monitoring has followed over the past several decades.
- The Infrastructure Era
Monitoring started at the foundation. In the 1990s and early 2000s, monitoring asked a single question: is the server up? CPU, memory, network traffic. If those checked out, the IT team could sleep soundly. Systems were monolithic, failures were well-contained. A branch office went down, the local server was restarted — done. One alert meant one problem, and one team resolved it.
This model worked well as long as the IT environment remained simple and static.
- The Rise of Applications
Between 2005 and 2012, as digital services proliferated, applications became the primary channel for business operations. Application Performance Monitoring (APM) was born — no longer simply showing whether a server was alive, but how an application was performing: where it was slowing down, where a user journey was breaking.
This was the moment attention began to shift from technology toward experience.
- The Complexity Explosion and the Emergence of Observability
The next inflection point arrived between 2015 and 2020, when microservices, containerization, and cloud fundamentally changed how IT architectures operated. A single business process now passed through dozens — sometimes hundreds — of components. A failure no longer originated in a single system; it propagated as a cascade across the entire architecture.
The Amazon Prime Day outage of 2018 illustrated this clearly: overload of an internal system called Sable — a compute and storage service — triggered cascading failures across multiple services, from authentication to video playback. Tracing the fault took hours, while CNBC, citing internal documents, estimated revenue losses of approximately 1.2 million dollars per minute.
Observability was the answer: by correlating logs, metrics, and traces, it became possible to surface the causal chain behind technical failures.
- The Cloud and Real-Time Operations Era
The post-2020 period became the era of cloud and real-time operations. Hybrid and multi-cloud architectures became the norm, infrastructure changes by the minute, and load fluctuates constantly. Many organizations found that a sudden surge in digital traffic could no longer be interpreted by their existing tools — the tools signaled that something had become slower, but could not show where or why. According to the Dynatrace State of Observability 2024, 73 percent of organizations reported significant visibility blind spots in modern, distributed architectures. In this era, observability was no longer an optional tool — it became a prerequisite for keeping operations running.
The New Challenge: AI Is Running in Your Systems
The evolution did not stop in 2020. Today, a new layer is being built on top of the IT architectures we already know: artificial intelligence.
This is not a theoretical development. The Dynatrace State of Observability 2025 research — conducted with 842 CIOs, CTOs, and senior technology leaders — found that 100 percent of surveyed organizations are already using AI in some form within their business operations. Chatbots, decision-support systems, document analysis tools, LLM-based automations — AI has become embedded in daily workflows.
But this represents a qualitatively different challenge from a monitoring perspective.
Unlike traditional applications, an AI system operates on a probabilistic basis. It can produce different outputs for the same input — and those outputs can silently influence business processes. A customer service AI may provide inaccurate information. A credit-scoring model may produce a biased decision. In an automated workflow, an erroneous LLM output may advance to the next step without anyone noticing.
According to the Dynatrace AI Observability documentation, observing AI systems requires asking fundamentally different questions than those posed for traditional applications: What is the model’s response latency and accuracy? Is model drift occurring — a gradual shift in model behavior away from expected operation? How is token consumption evolving, and what are the associated costs? What happens when an AI agent invokes tools or communicates with other agents?
Just as the infrastructure monitoring of the 2000s could not handle the complexity of microservices, current observability tools do not provide a complete picture of what AI is doing inside our systems.
This is the point at which AI Observability becomes necessary.
What Is AI Observability?
Dynatrace defines AI Observability as: the collection, analysis, and correlation of telemetry data from AI systems, agents, and large language models (LLMs) in order to make visible, in real time, how these systems behave across every environment — including production.
That definition contains an important distinction.
Traditional observability monitors system state: is the service running, what is the response time, how many errors have been generated. AI Observability goes further — it also monitors what the AI is doing: what input it receives, what output it produces, how it reaches its decisions, what each response costs, and when it deviates from expected behavior.
Based on Dynatrace documentation, AI Observability provides visibility across four primary layers:
- User interactions: how people engage with the AI-powered service — what they ask, what they receive in response, and whether the outcome meets their needs.
- AI decision processes: how a prompt is constructed, what steps the AI takes before generating a response, and where it stalls or diverges from expected behavior.
- Agent interactions: when multiple AI agents work together — who is doing what, how they communicate with each other, and where errors can arise in autonomous workflows.
- Infrastructure: the cost of running AI — GPU utilization, memory consumption, network load.
Taken together, these layers provide the deepest currently available visibility into what AI is actually doing within our systems. That said, they do not encompass the full explainability of AI decisions. Why a model reached a specific conclusion cannot always be answered definitively. For agentic AI systems — where multiple AI agents operate jointly in a non-deterministic fashion — comprehensive traceability remains an evolving field (OpenTelemetry GenAI SIG, 2025).
What Do I Gain From Seeing What AI Does?
This is the question worth asking before anyone selects a platform, initiates a project, or plans a budget.
To understand the answer, it helps to first take stock of where we are today — and where we are heading.
Where Is the Average Enterprise Today?
Most organizations are currently running relatively straightforward AI solutions in production: customer service chatbots, document analysis tools, code generation, internal knowledge base search. According to Menlo Ventures’ 2025 research, only 16 percent of enterprise organizations have genuine AI agents in operation — where an LLM independently plans and executes steps. The majority are running simple, fixed-sequence processes built around a single model call.
That state is changing rapidly. Enterprise spending on generative AI grew from 11.5 billion dollars in 2024 to 37 billion dollars in 2025 — a 3.2x increase in a single year (Portkey Enterprise LLM Report 2026). According to McKinsey data, 78 percent of organizations are now applying AI in at least one business function, compared to 55 percent just two years ago.
Where Are We Heading in the Next Few Years?
Industry forecasts point in a clear direction: organizations are moving from simple copilot tools toward autonomous, multi-step AI agent systems. These agents do not merely respond — they make decisions independently, invoke tools, communicate with other agents, and carry business processes through to completion without human intervention. According to the Dynatrace State of Observability 2025, the expected ROI of agentic AI is 41 percent — among the highest figures across all AI application categories.
Where Can Problems Arise?
The more autonomous the AI, the higher the stakes — and the less visible the failure.
Three typical risk points:
- Model drift. AI model performance can shift over time: input data changes, and model behavior gradually drifts away from expected operation. This is not immediately visible — it occurs over weeks or months, and by the time the business impact becomes apparent, the root cause is difficult to trace.
- Cost explosion. The token-based pricing of LLM calls can be unpredictable. A poorly optimized prompt, an unexpected traffic spike, or an infinite agent loop can generate significant costs within minutes. Without visibility, these are only discovered after the fact.
- Agentic failure. When multiple AI agents work together, an erroneous decision can propagate in a cascade through the rest of the workflow. The human sees only the final result — not where something went wrong, or why.
How Can These Risks Be Mitigated?
AI Observability provides precisely the answers these risks demand. It delivers real-time visibility into model behavior, token consumption, agent decisions, and their business impact. It does not prevent failures — but it makes them visible before their consequences materialize.
Closing Thoughts
The history of monitoring contains a recurring pattern: every new technological era has created new visibility requirements. The infrastructure era gave rise to server monitoring. The era of digital applications gave rise to APM. The cloud and microservices era gave rise to observability.
The AI era now demands the same step. The difference is that the response must come faster. While previous transitions unfolded over years, the proliferation of AI-based systems is a matter of months. Those who see what their AI is doing can govern it. Those who do not can only hope it causes no business or reputational harm.
Organizations that begin building this visibility today will enter with an advantage in the competition — where the stakes are no longer whether to deploy AI, but whether it can be kept under control.
Sources
- Dynatrace – AI Observability documentation: docs.dynatrace.com
- Dynatrace – State of Observability 2025 (842 CIO/CTO global survey, Qualtrics): dynatrace.com
- Dynatrace – State of Observability 2024: dynatrace.com
- CNBC – Amazon Prime Day 2018 internal documents: cnbc.com
- Menlo Ventures – State of Generative AI in the Enterprise 2025: menlovc.com
- McKinsey – The State of AI 2025: mckinsey.com
- Portkey – Enterprise LLM Report 2026: portkey.ai
OpenTelemetry GenAI Semantic Conventions SIG, 2025: opentelemetry.io
