AIOps root-cause analysis on a model you host yourself

Forty alerts, three teams and one actual fault. The useful part of AIOps is reducing that to a short ranked list you can check, without sending your telemetry anywhere.

What the first 15 minutes look like without it

An invented example. At 11:07, a connection pool in the accounts service runs out of connections. Within three minutes:

  • accounts, transfers, statements and mobile-gateway all show raised latency.
  • Five SLO alerts are burning, and two synthetic checks are slow.
  • Error logs surge in four services, each blaming the one it calls.
  • Two pods restart after failing liveness checks, which makes things worse.

Everyone is looking at their own service. The mobile-gateway on-call engineer has the loudest alert, and she is the furthest from the cause. The first quarter of an hour goes to working out what is a cause and what is a symptom. Reducing MTTR discusses where this time goes in general.

Two separate steps: grouping, then explaining

Wrentrix separates two things that the term "AIOps" tends to blur.

The problem engine groups signals by topology into one incident. It knows from the live service map that mobile-gateway depends on transfers, which depends on accounts. Alerts on those services within the same period are unlikely to be independent, so they become a single problem with a single timeline. This relies on the structure of your system, and on the baselines that identify abnormal behaviour.

Automatic RCA then produces ranked candidates with a clickable evidence chain. For that problem, it proposes possible causes in order, and for each one shows the signals behind it. Each link goes back to the underlying data, whether that is a trace, a log pattern, a method baseline, or a config change.

For the example above, the output might read like this. It is written here for illustration and isn't a captured product output:

  1. **Connection pool exhaustion in accounts.** Evidence: the pool-wait method moved outside its baseline first, at 11:07; a new log pattern acquire took <*>ms; accounts sits upstream of all affected services on the dependency map.
  2. **ConfigMap change to accounts.** Evidence: edited at 10:52, 15 minutes before onset, in the same namespace; a pod restarted at 11:05 and would have picked up the new value.
  3. **Node pressure on worker-7.** Evidence: two restarted pods were on this node. Weak, since those restarts came after the onset.

Notice what makes this valuable. It does not claim certainty. Instead it gives an experienced engineer a place to begin, plus a fast way to reject a wrong suggestion: click the evidence, see whether it holds. The second candidate may turn out to be why the first happened, for instance if the edit reduced the pool size. A person works that out in two minutes by opening the diff, as opposed to the half hour it takes to find it unassisted. See also connection pool exhaustion.

Why the evidence chain matters more than the model

A language model produces plausible text. Sometimes plausible text is wrong, and a confident wrong root cause during an outage is worse than no answer. It sends people in the wrong direction under pressure.

So the design question for any RCA tool is what a human has to take on trust. With a clickable chain, the answer should be: nothing. Evaluate RCA tools, including this one, with a few practical tests:

  • Replay a past incident you understand well. Does the true cause rank near the top, and does the evidence shown actually support it?
  • Check that the links land on real data. A claim about a method's latency should open that method's data, not a paragraph.
  • Look at what it does on an incident with an obscure cause. A useful system should show its weak confidence, and not make up a story.
  • Give your engineers time to build their own judgment of the tool before relying on it. Keep a tally for the first few weeks: right, partly right, wrong.

We don't publish accuracy figures, as they'd depend on your environment, on your data quality and on the model you run. Test on your own incidents. The live demo shows the flow with sample data, without any connection to your systems.

Where the AI runs

The AI runs on infrastructure you control. Nothing goes to an external model, and no call to an outside AI service is involved. This also means it works in a fully air-gapped network. You can also turn AI off entirely.

If you are evaluating Wrentrix, write to us and we will go through what the AI features need in your environment.

One operational point applies to any model that reads telemetry: treat logs as untrusted input. Log lines and other telemetry can contain text written by outsiders, for instance a URL or a form field. A language model reading that can be influenced by it. This is the reason remediation deserves approval steps, and why a human should stay in the path for actions that really matter.

Asking questions in plain language

Natural-language queries let you ask something like "Top 3 services by traffic in the last 12 hours", which becomes a query plan. The plan is the part to look at. A query plan you can read means you can see how the question was interpreted, such as which metric was taken to mean "traffic" and which time range was used, before you repeat the answer to anyone.

This helps most with people who aren't using the tool every day: a duty manager who needs a figure during a bridge call, or an engineer from another team who doesn't know how the data is organised. It works in English and Turkish, as does the rest of the UI and assistant.

Sample questions to try against your data:

  • Which endpoints have the highest error rate in the last hour?
  • Which services had a deployment today?
  • What changed before the latency rise on the payments service?

Explain this screen

"Explain this screen" is available on any SLO, deploy or anomaly. It's most useful when you are looking at something outside your specialty. A database administrator looking at a burn-rate chart, or a new on-call engineer looking at an unfamiliar anomaly, gets a plain description of what it shows and what is unusual about it. That saves someone from asking a colleague at 03:00 what a graph means. It does not replace learning the system. Treat it as a helpful colleague who reads quickly and sometimes misreads.

When this isn't for you

  • If you have very few services and a team who already know them well, topology-based grouping has little to do, and a language model adds little.
  • If you have no spare hardware and no appetite for operating a model, turn AI off, which the product allows entirely. Ask us to go through what is available with it switched off.
  • If your telemetry is poor, with broken traces and no service naming, then fix that first. RCA can't find evidence that was never collected. Distributed tracing is the usual starting point.

For a vendor-neutral background on what this category can and can't do, read what AIOps is.

Frequently asked questions

Does Wrentrix send data to an external AI service?

No. The model runs on your own infrastructure, and no telemetry or prompt goes outside your network. It also runs fully air-gapped, with no external model involved.

Can I trust an automatically generated root cause?

Treat it as a ranked set of hypotheses, not a verdict. Each candidate comes with a clickable evidence chain, so an engineer can confirm or reject it by looking at the underlying data. Replaying past incidents you understand is a good way to learn how far to trust it in your environment.

Does the AI work in Turkish?

Yes. The UI and assistant are available in English and Turkish, and you can ask natural-language questions in either. Generated output quality in each language is worth checking on your own data as part of a trial.