A checkout that got slow
It is 11:20 and customers report that checkout takes eight seconds. Infrastructure dashboards look fine: CPU is at 40%, memory is flat and no pods have restarted. Without request-level data, the next hour goes to a conference bridge in which each team reports that its own component looks healthy.
With APM, a responder opens the checkout endpoint and sees that p95 latency jumped from 400 ms to 7.8 s at 11:04. A slow trace from that period shows the time breakdown: 30 ms in the web tier, 60 ms in the order service, then 7.4 s inside one call to the pricing service. Within that call, a single SQL statement takes 7.3 s. The pricing service's deployment went out at 11:02. The discussion moves from "whose fault is it?" to "what changed in that query?"
That is what APM is for. It connects a user-visible symptom to a specific place in the code or a dependency.
A working definition
Application performance monitoring (APM) is the practice, and the category of tools, for measuring the behaviour of applications in production by following requests through them. Most APM products collect:
- request rate, error rate and latency per service and endpoint
- distributed traces that show each request's path through services and dependencies
- errors and exceptions, grouped so that one bug shows up as one item
- a map of which services call which
- details from inside the runtime, such as method timing, garbage collection or thread pool state
- logs tied to the traces that produced them
Many also include real user monitoring in the browser, synthetic probes and infrastructure views, because the question "why is this slow?" often crosses into those areas.
How it collects data
Instrumentation
Something has to observe your code as it runs. There are three common ways:
-
Automatic instrumentation. An agent attaches to the runtime and wraps
well-known libraries such as HTTP servers, database drivers and message clients. The JVM
has
-javaagent; .NET has a profiling API; Python and Node.js can wrap modules at load time. You change no code. -
Manual instrumentation. Developers add spans around business operations
with an SDK. It takes more effort, but it is the only way to get names like
apply-discountand attributes like a customer tier. - Method-level capture. Some tools time individual methods in the runtime, either across the board or by rule. This shows where time goes inside a service, between the entry and exit points that automatic instrumentation sees.
Most real deployments mix the first two. OpenTelemetry has become the common vendor-neutral standard for this, so the instrumentation does not tie you to one backend. Our OpenTelemetry getting started guide walks through a first service.
Context propagation
For a trace to span several services, each one has to pass on a trace ID in outgoing calls.
The usual standard is the traceparent HTTP header defined by W3C Trace Context. If one hop drops it, the trace
breaks into two.
The backend
Telemetry goes to a backend that stores it and computes aggregates such as latency percentiles. The backend can be someone else's cloud service or a system on your own hardware, a choice we cover in on-premise vs SaaS APM.
What to look at first
A new APM user can get lost among the charts. For a given service, start with these questions:
- Is traffic normal? A latency improvement during a traffic drop may just mean users cannot reach you.
- What is the error rate, and is it one error or many? Grouped errors tell you if a spike comes from one new exception.
- What do p50, p95 and p99 look like? Averages hide trouble. If p50 is flat while p99 doubles, a subset of requests got slow, often those hitting one shard, a cold cache or a lock.
- Which dependency owns the time? A trace or dependency breakdown tells you whether time was spent in your code, the database or a downstream service.
- What changed? Line the spike up against deployments and config changes. Incident reviews very often point back to a recent change, so check for one early.
For a structured set of metrics to watch, read about golden signals, RED and USE.
Why percentiles beat averages: a short example
Imagine 100 requests. 95 take 100 ms and 5 take 4,000 ms.
- Average: (95 × 100 + 5 × 4,000) / 100 = 295 ms
- p50: 100 ms
- p95: 100 ms (the 95th request, when sorted, is still fast)
- p99: 4,000 ms
The average, 295 ms, describes nobody's actual experience. It says the service is somewhat slow, while the truth is that most users are fine and one in twenty is having a terrible time. Note also that a page that makes 20 backend calls gives the user many chances to hit the slow tail, so tail latency reaches more people than the percentage suggests.
APM vs related terms
| Term | What it focuses on | Relationship to APM |
|---|---|---|
| Observability | The ability to ask new questions about a system from its outputs | A broader idea; APM is one of the main ways to achieve it for applications |
| Infrastructure monitoring | Hosts, nodes, containers, disks, network | Tells you if the platform is healthy, not if requests are; the two should be linked |
| Log management | Collecting and searching log events | Provides detail that traces lack; traces tell you which logs to read |
| Distributed tracing | Following one request across services | The core data type in modern APM; see what is distributed tracing |
| Real user monitoring | Performance as experienced in the browser or app | Extends APM to the client, and shows what the backend cannot see |
| Synthetic monitoring | Scripted probes from known locations | Finds outages when no real users are active |
| AIOps | Applying statistics and machine learning to operations data | Builds on what APM collects; see what is AIOps |
What APM costs you
Monitoring is not free at runtime, and you should measure this rather than assume.
- Runtime overhead. Agents add CPU and memory per process and some latency on each request. Typically this is small, but it depends on the runtime, how many spans you create and how much method-level detail you ask for. Benchmark on your own service under realistic load.
- Startup time. Bytecode-instrumenting agents can slow JVM startup noticeably, which can matter for readiness probes and autoscaling.
- Telemetry volume. Traces can be large. Sampling exists for this reason, and tail-based sampling tries to retain the interesting traces.
- Sensitive data. URLs, SQL text and log messages can carry personal information. Review what is captured and mask it before it is stored. If your tool is a hosted service, that data also leaves your network.
- Attention. Poorly chosen alerts wear out responders. Good APM usage means fewer, better alerts, not more of them.
Choosing an APM tool: questions that matter
A demo always looks good, so bring your own hard cases:
- Does it cover your actual runtimes? Older application servers, native code and unusual frameworks are where coverage gaps appear.
- Where does the data live? If regulations or network design rule out sending telemetry outside, this question ends many evaluations at once.
- Is the instrumentation portable? OpenTelemetry support reduces future migration cost. Ask whether it is first-class or just an ingest endpoint.
- What privileges does the agent need? On OpenShift, ask which security context constraints it needs. See the restricted-v2 guide.
- What happens when the agent fails? Monitoring should not take your application down with it.
- How does it link traces with logs and changes? Test this with an actual past incident, not a canned demo.
- How is access controlled? Telemetry from one team may not be something another team should see. Ask about SSO, group mapping and audit logs.
When you may not need APM
A single monolith with a handful of users can often get by with good logs, a few metrics and a database slow query log. Batch jobs may need job-duration tracking more than request tracing. APM pays for itself when requests cross several services, when many teams share responsibility or when production problems regularly take longer to locate than they do to fix. If that does not describe you yet, start small, with OpenTelemetry on your busiest service, and let your needs grow from there.
How Wrentrix approaches APM
Wrentrix is an on-premise APM and AIOps platform that runs on OpenShift, Kubernetes or Linux, with no telemetry leaving your network. It is OpenTelemetry-native and accepts OTLP traces and logs. It provides a live service map, endpoint p50/p95/p99, grouped errors and logs linked to traces. Java and .NET services get method-level tracing without code changes through admission webhook hooks. You can browse sample data in the live demo.
Frequently asked questions
What does APM stand for?
APM stands for application performance monitoring, and is sometimes expanded as application performance management. Both refer to measuring how applications behave in production by following requests through them. Most tools today include traces, error tracking, service maps and logs linked to those traces.
What is the difference between APM and observability?
Observability describes how well you can understand a system's internal state from its outputs, including questions you did not plan for. APM is a practical category of tools focused on application requests, and it is one common way to achieve observability. Many products now use both words for overlapping capabilities.
What is the difference between APM and logging?
Logs are individual events written by your code, and they are good for detail about what happened. APM measures requests as a whole, showing timing and how calls relate across services. The two work best together: a trace shows you where to look, and logs linked to that trace explain what happened there.
Does APM slow down my application?
Usually a little, and the amount depends on the runtime, agent configuration and how much detail you capture. Heavy use of method-level timing or very high span counts cost more. The trustworthy answer comes from a load test on your own service, with and without the agent.
Can I do APM with open-source tools?
Yes. OpenTelemetry for instrumentation, plus an open-source tracing backend, metrics stack and dashboards, can cover the core needs. You take on the work of integrating and running them, and of keeping labels and links between signals consistent as services change.