Kubernetes and OpenShift monitoring under restricted-v2

The install, step by step

Most platform teams want to know first what they are being asked to apply to their cluster, so here is the sequence.

1. In the Wrentrix UI, add a cluster.
2. Download the ready-made YAML. The token is already inside it.
3. Read the file. It is yours to review, and your security team may want to do so too.
4. Apply it.

oc apply -f wrentrix-k8s-agent.yaml

Nodes and pods appear in about two minutes. Java and .NET workloads gain method-level tracing on their next restart. Wrentrix itself never restarts them.

The platform can be installed with a Helm chart, an OpenShift operator or Docker Compose, for example:

helm install wrentrix ./wrentrix.tgz -n wrentrix -f values.yaml

Because the downloaded agent YAML contains a token, treat it as a credential. Don't commit it to a shared Git repository in plain form. If you use GitOps, follow the same process you use for any other secret.

What running under restricted-v2 means

Monitoring agents have a long history of asking for a lot: privileged containers, host paths, host networking, a custom SCC. In many regulated OpenShift environments, that request starts a review that can outlast the evaluation itself.

Wrentrix runs under the default restricted-v2 SCC, with least-privilege pods, no root and no extra SCCs. Under restricted-v2, OpenShift assigns a UID from the namespace's allocated range, requires that privilege escalation is disallowed, drops all capabilities (allowing only NET_BIND_SERVICE to be added), and requires a seccomp profile of RuntimeDefault or similar. You can verify the admission result yourself using a standard annotation:

oc get pods -n wrentrix -o custom-columns='POD:.metadata.name,SCC:.metadata.annotations.openshift\.io/scc'

If that column shows restricted-v2 for the pods you care about, the claim is verified on your own cluster and you don't need to take anyone's word for it. The restricted-v2 guide explains what the SCC permits and why images that assume a fixed UID break under it.

Fail-open admission

The agent includes an admission webhook that handles method-level instrumentation. Anything in the pod-creation path deserves suspicion, since a badly configured webhook with failurePolicy: Fail can prevent every pod in a cluster from starting, including the ones that would fix it.

The Wrentrix webhook fails open: if the agent is unavailable, pods start without instrumentation. The p95 overhead of the webhook is under 25 ms per pod start. To inspect it directly:

oc get mutatingwebhookconfigurations
oc get mutatingwebhookconfiguration <name> -o yaml

Check failurePolicy, timeoutSeconds, and the namespaceSelector. For a background on these fields, including why cluster-wide webhooks should not intercept control plane namespaces, read the admission webhooks guide.

What you see once it is running

Wrentrix shows workloads, pods, events and HPA state, along with resource usage at min, average, p95 and peak. OpenShift already ships a cluster monitoring stack as a core platform component, and nothing here requires you to remove it. The built-in stack watches the health of the platform. Wrentrix connects the platform view with what applications are doing: requests, traces, methods, logs. For how the built-in pieces fit together and what they do not cover, see the OpenShift monitoring guide.

Events matter more than they get credit for. Kubernetes events such as OOMKilling, BackOff, FailedScheduling and Unhealthy are often the only record of why a pod restarted, and by default they are kept for a short time in the cluster (the API server's --event-ttl defaults to one hour). Once they expire, oc describe tells you nothing about what happened last night.

Right-sizing with four numbers, and why they matter

Requests that are too high waste capacity. Requests that are too low cause scheduling pressure and contention. Limits that are too low lead to CPU throttling and OOM kills. Teams usually set these once, by guesswork, and don't revisit them.

The usage figures in Wrentrix, min/avg/p95/peak, tell you different things, and the distance between them is what you should look at.

CPU and memory need different thinking. CPU is compressible: going above a limit causes throttling, not failure. Memory is not: going above a limit gets the container OOM-killed. So p95 can be a fair input for a CPU request, whereas memory limits should respect the peak plus a margin. The peak must also come from a period that includes your heavy work, such as month-end or batch windows. Wrentrix offers right-sizing hints from this data. They are hints: the decision still has to consider growth and what you know about the application. The right-sizing guide goes into a method for that.

The HPA trap

CPU-based HPA targets are percentages of the request, so changing requests changes the scaling behaviour. The Kubernetes formula is desiredReplicas = ceil(currentReplicas × currentMetric ÷ targetMetric).

Worked example: 4 replicas, each with a 1000m request and using 300m on average, with a target of 60%. Current utilization is 30%, so the HPA wants ceil(4 × 30 ÷ 60) = 2 replicas. Now lower the request to 500m with the same usage. Utilization becomes 60%, and the HPA computes ceil(4 × 60 ÷ 60) = 4. That is a different stable state at the same load, and the bill you thought you were reducing may not fall as far as you expected. Review HPA state and resource requests as one change.

Change correlation: what changed before it broke?

Many production incidents begin with a change. Finding that change is harder than it should be, as it may be someone else's. Wrentrix lines up changes to spec, ConfigMap, Secret and rollout restarts against latency and error spikes.

Each of those types is slippery in a different way:

• Spec changes create a new ReplicaSet for a Deployment, so they are visible in rollout history.
• ConfigMap edits do not restart pods. A mounted file updates after a delay, whereas an environment variable sourced from a ConfigMap changes only upon the next pod start. So the change and its effect may be hours or days apart, and the effect may appear when something unrelated restarts the pod.
• Secret changes behave the same way, and a rotated database password that takes effect only upon restart is a classic source of incidents that "started at random".
• Rollout restarts (oc rollout restart deployment/x) set the kubectl.kubernetes.io/restartedAt annotation on the pod template. The image and the config did not change, so a Git diff shows nothing, but every pod is new.

An invented example of the payoff: p99 on checkout climbs at 02:14. There was no deployment that night. The timeline shows a ConfigMap edit at 18:30 the previous day, and a node drain at 02:12 that recreated two pods, the first to read the new value. Those two pods are the slow ones. Without a shared timeline, this takes hours and a good memory. The change correlation guide describes how to build this practice even without tooling. Correlation is evidence, not proof. Unrelated changes happen near incidents too, so verify before rolling back.

When to look elsewhere

• If you only need cluster health for a small environment, the monitoring built into OpenShift may be sufficient on its own.
• If your concern is control-plane internals, such as etcd latency or API server request throttling, the platform's own dashboards remain the most direct place to look.
• Docker Compose runs on one host, so it cannot give you the multi-replica layout in the architecture diagram, which shows UI, API and analysis at 3+ replicas, with PostgreSQL for storage. If you want that layout, install on OpenShift or Kubernetes.
• We don't publish hardware sizing for arbitrary cluster counts. Bring your node and pod numbers, and ask us.

For Linux machines outside the cluster, see Linux server monitoring.

Frequently asked questions

Does Wrentrix need a custom SCC or privileged pods on OpenShift?

No. Wrentrix runs under the default restricted-v2 SCC, with no root and no extra SCCs. You can check this on your cluster by reading the openshift.io/scc annotation on the pods.

Will installing the agent restart my workloads?

No. Wrentrix never restarts workloads. Nodes and pods show up in about two minutes after you apply the YAML, while method-level tracing for Java and .NET begins as each pod next restarts for its own reasons.

What happens to new pods if the agent or webhook is down?

The webhook fails open, so pods start without instrumentation and deployments are not blocked. They simply lack method-level data until they next restart with the agent available. Check this behaviour in your own test cluster by scaling the agent down and creating a pod.

Can Wrentrix run next to OpenShift's built-in monitoring?

Yes, and it has to, because the built-in stack is a core part of OpenShift. That stack watches platform health, and Wrentrix adds application traces, method timings, logs, right-sizing hints and change correlation on top of it.

Can I see which change caused a latency spike?

Wrentrix lines up spec, ConfigMap, Secret and rollout restart changes against latency and error spikes on a timeline. That shows which changes happened just before a spike. Whether one of them is the true cause is something you confirm, since timing alone is not proof.