What is AIOps?

The definition, stripped down

The term began as an industry analyst label for platforms that apply big data and machine learning to IT operations. Today people use it for almost anything with "AI" attached to an operations tool, so the word by itself tells you very little. A more helpful way to look at it is by the job being done.

AIOps work falls into five jobs, each with different techniques and different levels of maturity:

Let's go through them with concrete failures, since each job goes wrong in its own way.

Spotting abnormal behaviour

A fixed threshold of "alert over 800 ms" fails in two directions. Monday morning peak might legitimately reach 750 ms, while 500 ms at 3 a.m. Sunday, normally a 120 ms period, passes without notice even though something is clearly wrong.

Seasonal baselines fix this by learning what is normal for each period. A common approach buckets history by hour of week, giving 168 slots (7 × 24). Each slot learns a typical value and a spread, and a new sample is scored against its own slot.

Some details to check in any implementation:

• Is it per-endpoint, or just per-service? A slow new endpoint can hide inside a service's overall average.
• How does it treat new services and new endpoints? With no history, it should say so instead of guessing.
• Does an incident contaminate the baseline? If a week of bad behaviour becomes the new normal, it stops being flagged.
• Do holidays and campaigns break it? Expect noise around unusual traffic days.
• How does it treat low traffic? A p99 from ten requests isn't meaningful, and a scoring system ought to know this.

Anomaly is not the same as impact, either. A batch job can double a service's latency during a window that no customer sees. For paging, tie alerts to user impact through SLOs and burn rates, and treat anomalies as supporting evidence.

Reducing alert noise

When a database slows down, you might get alerts from the database, every service that calls it, each of their callers and the synthetic probe at the edge. That can easily be dozens of notifications for one problem.

Grouping them is where topology earns its keep. If the service map shows that A calls B and B calls the database, then simultaneous alerts on all three are likely a single problem. Without topology, a tool can only group by time window, and that merges two unrelated failures that happen to fire in the same minute.

The topology has to be real for this to work. A map generated from live traces stays current, whereas a CMDB maintained by hand tends to drift. Dependencies that traces do not capture, such as a shared storage array or a noisy neighbour on a node, can defeat it, so also link services to the infrastructure beneath them.

For a process-focused look at this problem, see how to fix alert fatigue.

Finding likely causes

Here is a worked example of how ranking works, with invented numbers. At 14:05, checkout p95 rises from 300 ms to 2.1 s. An engine looks for candidates:

There is no deep intelligence in this. It depends on ordering in time, position in the dependency graph and agreement between signals. That is also why it fails when data is missing. If the change that caused the problem is not recorded anywhere, such as a firewall rule or a database parameter set by hand, it cannot be ranked. The tool will then rank whatever it does see, potentially with confidence.

So look at whether a tool presents candidates with evidence you can click through, rather than a single declared "root cause". Real incidents often have more than one contributing factor, and a responder should be able to check the reasoning in seconds. See what-changed analysis for how to collect the change data this relies on.

Language models in operations

Language models brought a new kind of capability: reading messy context and writing a human explanation.

Good uses:

• turning "which services got slower after last night's release?" into a real query against your data
• summarising a long alert or incident timeline for someone joining the call
• explaining an unfamiliar screen or metric to a junior engineer
• drafting a postmortem from the recorded events, for a human to correct

Poor uses:

• computing percentiles or burn rates in the model's head, which should be done by ordinary code
• answering with no retrieved data, where it will fill the gap with plausible invention
• taking destructive actions on its own judgement

One design rule separates the usable from the risky: retrieved facts, then a model that explains them, then a way for a human to verify. When the model produces a query, you should be able to see that query and the result it returned.

A second consideration is that the model has to read your logs and traces. That raises a data question that earlier forms of AIOps did not. See data residency for what to ask.

Acting on findings

Automated remediation is old and proven when the failure and fix are well understood: restart a stuck consumer, clear a full temp directory, add capacity. The AI part is mostly in choosing when to propose the action. A safe progression:

1. Link each alert to a runbook, for a human to carry out.
2. Automate the runbook, with a human approval click.
3. Let it run unattended for a narrow set of well-understood, reversible, rate-limited actions.

Put a limit on how often automation can act. A restart loop that hides a memory leak until the whole fleet is affected is worse than a page. Runbook automation with Ansible AWX goes through this in detail.

Signs you are being sold a label

Ask these questions in any AIOps evaluation:

• Show me a wrong answer. Vendors who have run this in practice know how it fails and can tell you.
• What does it do when data is missing? "Insufficient evidence" is a healthy answer.
• Which data leaves my network, including prompts? Get it in writing.
• Can a responder see why? Avoid black boxes for anything that pages a person or changes production.
• Can I run it with AI turned off? Your incident process should not depend on one model.
• How much data does it need before it is useful, and how does it behave until then?

Run a test on your own past incidents. Note how many had the true cause in the top candidates and how often the tool sent people in the wrong direction. Without that, any claim about a percentage reduction in MTTR is just a marketing line.

What to fix before buying anything

AIOps builds on the quality of what you already have. It will not rescue:

• services with no consistent naming, so that the same app appears under three names
• missing traces, so that the dependency graph has holes
• no record of deployments or config changes
• alert rules that nobody owns

Fixing those helps your teams whether or not you add AI. In many organisations, this hygiene and a set of good SLOs deliver more improvement than any model.

How Wrentrix does this

Wrentrix learns hour-of-week baselines from its own history for endpoints and methods, and its problem engine groups signals by topology into one incident. Automatic RCA gives ranked candidates with a clickable evidence chain. Natural-language questions become a query plan. The language model runs inside your network, no data goes to an external model and you can turn AI off entirely. Remediation runs through webhooks or Ansible AWX job templates, automatically or with approval.

Frequently asked questions

What is AIOps in simple terms?

AIOps uses statistics, machine learning and language models on operations data such as metrics, logs and traces. The goal is to find problems sooner, reduce alert noise, suggest likely causes and sometimes trigger a fix. Humans still own the decisions that carry risk.

What is the difference between AIOps and observability?

Observability is about having data rich enough to understand your systems and ask new questions of them. AIOps is the analysis layer that works on such data to automate parts of detection and diagnosis. Without good telemetry, AIOps has little to work with.

Will AIOps replace SREs?

No. It can take over repetitive parts of the work, such as gathering context and grouping alerts, but people still design systems, weigh risky decisions and handle new failure modes. It tends to shift effort rather than remove the role.

Can AIOps run without sending data to the cloud?

Yes, if the platform and its models run on your own infrastructure. Baselines and topology analysis need no external service, and language models can be hosted locally, with some quality trade-offs. Ask each vendor precisely what data leaves the network, including prompts and licence checks.

How do I measure if AIOps is working?

Replay real incidents and check how often the true cause was in the top candidates, and how often it misled responders. Then track alert volume per incident and time to the first correct hypothesis over a few months. Be careful with MTTR alone, because it varies a lot with incident mix.