When it comes to troubleshooting and ensuring system health, Observability is all about one essential concept: connecting the dots. To identify and resolve problems effectively, we need to understand what happened, where it happened, and when it happened—all in context. This requires more than just collecting data; it demands that data be connected in meaningful ways.

The Evolution from Monitoring to Observability

Traditional monitoring systems were built on siloed data sources. Metrics, logs, and traces were treated as separate entities, stored in separate systems, and analyzed independently. While this approach served its purpose in the early days, it often left teams scrambling to piece together fragmented information during outages or performance bottlenecks.

Modern Observability changes this. It’s not just about collecting data; it’s about connecting data right at the source. Logs, metrics, and traces are no longer isolated pillars but integrated streams of information that provide a holistic view of the system. This shift allows teams to extract actionable insights on demand, making troubleshooting faster and more reliable.

Context is King: Why connected data matters

To understand the value of connected data, let’s break down the components of Observability:

1. Traces
   At the heart of modern Observability are traces, which represent end-to-end requests as they travel through distributed systems. A trace is made up of spans, and each span contains:
   • Duration: The time taken for a specific operation, often referred to as latency or response time.State: Whether the operation succeeded (OK) or failed (Error).Events: Key points within the operation, such as errors or retries.Context: Information about which service or process initiated the operation and what other services it interacted with. And we can add infrastructure information like host.name or pod.name.
   Traces naturally bring context into the picture. For example, if Service A calls Service B, and Service B encounters an error, the trace immediately shows the relationship between the two. This connected information makes it much easier to pinpoint the root cause of the issue compared to traditional methods, where logs and metrics would need to be manually correlated.
2. Logs and Metrics Reimagined
   Historically, logs were the first step in gathering Observability data. However, logs are inherently disconnected. You might find an error in a log file with a timestamp, but without additional context, you’d still have to cross-reference metrics or other logs to understand the full picture. Metrics, while useful, also have limitations. For example, they provide aggregate data like average response times or error rates, but they don’t show why those numbers are spiking. Metrics alone can mislead you if you don’t have the associated context. With traces and spans, the gap between logs and metrics disappears. Spans act as structured events—they include labels, timestamps, events, and execution contexts. This unified approach means you no longer need to collect metrics separately; you can derive them directly from trace data, creating metrics like latency, error rates, and traffic on the fly.

The need for new storage and thinking

This shift to connected, contextualized data requires a new way of thinking—not just about Observability but also about how we store and process data. Traditional relational databases or even time-series databases struggle to handle the scale and complexity of modern Observability data.

Instead, modern Observability platforms use columnar storage solutions like ClickHouse or custom event databases. These technologies allow for rapid queries and on-the-fly calculations, enabling teams to slice and dice data as needed without pre-aggregating metrics.

The Unknown Unknowns – finally solved

One of the biggest challenges with traditional monitoring has always been the need to define alerts and metrics before starting to monitor. You had to predict what information was relevant and which metrics would signal an issue—whether it was CPU usage hinting at resource exhaustion or memory consumption pointing to a potential bottleneck. But in modern distributed systems, the signals aren’t always obvious.

The rise of unknown unknowns—problems you don’t even know to look for—makes troubleshooting far more complex. This is where Observability shines. With the right data storage and UI, you no longer need to predefine metrics. Instead, you can extract the metrics you need, when you need them, directly from the data you’ve collected. This flexibility requires modern storage solutions, like columnar databases or custom event databases, which aren’t constrained by fixed schemas like traditional relational databases. The true power of Observability lies in this adaptability: the ability to extract meaningful insights from your data in real-time, even in the face of unpredictable, dynamic systems stitched together by ever-changing configuration files.

Beyond monitoring: the Observability mindset

Many people refer to modern observability as “Observability 2.0,” but in reality, it’s a natural evolution of the same goal: understanding system behavior. The difference lies in the approach.

Monitoring systems focus on predefined metrics and logs, requiring engineers to look at disparate buckets of data and manually stitch them together. Observability, on the other hand, starts with connected data that’s already enriched with context. This approach not only simplifies troubleshooting but also empowers teams to predict issues and optimize performance proactively.

The Bottom Line

Modern Observability isn’t just about better tools; it’s about changing how we think about data. By connecting logs, metrics, and traces into a unified system, we gain the ability to extract meaningful insights and solve problems faster. No more chasing disjointed clues or relying on incomplete data – the future of Observability is contextual, connected, and designed for efficiency.

Embracing this mindset means fewer blind spots, faster resolutions, and, ultimately, a more reliable system. It’s time to move beyond fragmented monitoring and embrace the power of observability.

You think differently? Let’s talk in the Observability Community – it is a complex topic without a single best solution.