Method-level tracing for Java, .NET, Python and Node.js

The gap between a span and a fix

OpenTelemetry auto-instrumentation is good at boundaries. It creates spans for inbound HTTP requests, outbound calls, database statements and messaging. That covers a lot. It also leaves a recognisable hole: a server span lasting 1.8 seconds, one database child of 40 ms, one HTTP child of 60 ms, and 1.7 seconds of nothing.

That missing time is in your own code, or in a library no one instrumented. It might be a serializer running over a large object, a lock, a regex with bad backtracking, a mapper making thousands of small allocations, or a cache miss path that nobody thought of as slow. To find it with spans alone, somebody has to guess the location, add a manual span, ship a release, wait for the problem to recur and guess again. In a regulated environment where a release has a change window, that loop can take weeks.

Profilers are the other classic answer. They are excellent for depth, but a profile is usually taken when you decide to take it, on a chosen instance, and it says little about how a method behaved at 14:05 last Tuesday compared with a normal Tuesday. Method-level tracing sits between the two: continuous, per-method timing on production traffic, tied to the same trace and endpoint data you already look at.

What Wrentrix supports, by runtime

The difference between the rows is real and worth planning around. On Java and .NET you get method visibility from the next restart onward without choosing what to watch. On Python and Node.js, rules determine what is covered, so someone needs to decide which parts of the application matter. If you want to see what that decision looks like in practice, the live demo has sample data, and we can go through rule setup for your codebase if you ask us.

How zero-code instrumentation reaches a process

No one edits a Dockerfile or a deployment manifest for this. The mechanism rests on a standard Kubernetes extension point.

When a pod is created, the API server can call a mutating admission webhook, which may change the pod spec before it is stored. Both major managed runtimes have a supported way to load extra code at startup. The JVM has the Java agent mechanism and reads JAVA_TOOL_OPTIONS from the environment. The .NET runtime offers startup hooks and a profiling API, both controlled by environment variables. An admission webhook is in the right position to arrange that for a pod without anybody changing the image. The admission webhooks guide explains this request flow for readers who have not worked with it.

Three consequences follow, and they are the things operators usually ask about first.

It takes effect on the next restart, and Wrentrix does not cause that restart. A webhook only sees pods at creation. A pod already running stays untouched until your next deployment, node drain or planned rollout. Wrentrix never restarts workloads. You can install the agent during the day and let instrumentation arrive with your normal release cycle.

It fails open. The webhook is configured so that if the agent is unavailable, pods start without instrumentation. In Kubernetes terms this is what failurePolicy: Ignore expresses. Your deployment does not depend on your monitoring being healthy, which is the correct priority. The price is that during an agent outage you can get pods without method data, and you will only notice if you look, so treat "instrumented pod count" as something to check after any agent maintenance.

It adds a small cost at pod start. The p95 admission webhook overhead is under 25 ms per pod start. That figure is for the admission step only. It says nothing about in-process cost, which depends on your application's call patterns. Test that on a representative workload in your own environment and compare before and after. A single overhead number, including one you might get from us, means little until it comes from your code.

Here is a generic way to confirm that a pod was mutated, using only standard tooling. Compare the environment and volumes in the live spec with what your manifest defines.

oc get pod payments-7c9f5d7b6-x2k4q -n prod -o jsonpath='{.spec.containers[0].env}'
oc get pod payments-7c9f5d7b6-x2k4q -n prod -o jsonpath='{.spec.volumes}'

Your manifest is unchanged in Git, so tools such as GitOps controllers normally do not report drift on the Deployment. The mutation lives on the pod.

Why 17-bucket histograms

Recording every call of every method would be a lot of data, and most of it would never be read. Wrentrix builds method baselines from 17-bucket histograms instead. Each method's durations are counted into 17 time ranges, and the distribution, not the individual calls, is what gets kept and compared.

The arithmetic explains the design. As an example, take a method called 5,000 times per minute. Raw recording means 5,000 values per minute for that method alone. A histogram means 17 counters, however busy the method is. A method called 50 million times per minute costs the same 17 counters. That is what makes method-level history affordable across a whole application, not only for a few spans you chose in advance.

A distribution also captures things an average cannot. Consider this simplified example of one method's durations in two periods:

Total calls are 10,000 in both columns. Only 1 call in 100 has moved, and the median has not changed at all, yet a whole new group has appeared above one second. If that were an average, it would climb by some amount that looks like mild noise. As a shape, it reads as "1% of calls now hit something slow": a lock, a timeout and retry, a connection wait. A bimodal shift often points at a different cause than a uniform slowdown, where everything moves up one bucket because of CPU pressure or GC.

There is an honest limit. A percentile estimated from a histogram cannot be more precise than its bucket boundaries, so a shift inside one bucket will not be visible. Method histograms are for spotting that behaviour changed and where. For microsecond precision on one code path, keep using a profiler.

Using it in an investigation

A typical flow, using an invented incident as the illustration:

1. An SLO alert fires on the POST /orders endpoint. The p99 has climbed while the p50 is unchanged.
2. You open a slow trace. The server span has a large block of unexplained self time. No downstream call accounts for it.
3. Method data for that endpoint shows InvoiceRenderer.render outside its baseline for this hour of the week, while the rest of the call tree looks ordinary.
4. Change correlation shows a ConfigMap update minutes before the shift, which enabled a new template.
5. You roll back the configuration, and watch the method return to its baseline.

The important step is number three. "Slower than usual for this hour" filters out the methods that are always slow. In any application some methods legitimately take a second, such as report generators or batch handlers, and ranking by absolute duration just lists them again every time. Baselines per method are what make the output a short list of suspects. The baselines page explains the hour-of-week model behind them.

Once you have a suspect, the language-specific work still has to be done. For a JVM that may mean checking GC logs, thread dumps and connection pools, as described in Java latency troubleshooting. On .NET the usual suspects are thread pool starvation, sync-over-async and allocation pressure, which the .NET guide covers. If a method is waiting on a connection and not doing work, read about connection pool exhaustion next.

When method-level tracing will not help much

Being clear about the edges saves time:

• Native and kernel time. Waits in native libraries, syscalls and I/O appear as time in a calling method. You learn where it was called from, not what happened underneath. Linux host metrics fill part of that in.
• Workloads that never restart. A long-lived pod outside your normal rollout cycle stays uninstrumented until it restarts. Plan the rollout for those, since Wrentrix will not force it.
• Other languages. Support is described for Java, .NET, Python and Node.js. Other runtimes can still send spans through any OpenTelemetry SDK, but method-level data for them is a question to ask us.
• Non-Kubernetes Java and .NET. The zero-code route described here is the admission webhook, which belongs to Kubernetes and OpenShift. If your JVMs run directly on Linux hosts, ask us about your setup before assuming equal coverage.

Frequently asked questions

Do I need to change application code to get method-level data?

For Java and .NET, no. Instrumentation is applied through an admission webhook when a pod starts, so neither the image nor the source changes. Python and Node.js are rule-based, so someone defines rules for what to trace, without editing the application itself.

Will installing the agent restart my pods?

No. Wrentrix never restarts workloads. A pod already running stays as it is, and it gains method-level tracing the next time it restarts for your own reasons, such as a deployment or node drain.

What happens if the Wrentrix agent is down when a pod starts?

The admission webhook fails open. The pod starts normally without instrumentation, so an agent outage does not block deployments. That pod will have no method-level data until it next restarts with the agent available.

What is the performance overhead of method-level tracing?

The measured figure we can give is the p95 admission webhook overhead, which is under 25 ms per pod start. Runtime overhead depends heavily on how your application behaves, so run a comparison on a representative workload before rolling out widely. Ask us if you want help designing that test.

Does method-level tracing remove the need for profiling?

No. Method-level tracing measures method durations into histograms and checks them against a learned baseline per method, linked to endpoints, so it tells you which method changed and when. Profiling a process with your runtime's own tools still answers the deeper question of what that method is doing with the CPU.