Kubernetes Right-Sizing: Requests and Limits

Q: How do I know if a pod is being CPU throttled?

Compare throttled periods with total periods from container_cpu_cfs_throttled_periods_total and container_cpu_cfs_periods_total. A sustained high ratio on a service with unexplained tail latency is a strong clue. Average CPU can look moderate during heavy throttling, so don't rely on that alone.

What requests and limits really do

They look like a pair, but they act at different times and on different resources.

Setting	CPU	Memory
Request	Used for scheduling; sets CPU weight when the node is busy	Used for scheduling; influences eviction ranking under node pressure
Limit	Enforced by CFS quota; the container is throttled when it hits the limit	Enforced by the kernel; the container is OOM killed when it hits the limit
Too low	Latency, slow startup	Crashes, restarts
Too high	Wasted capacity, since the node's capacity is reserved	Wasted capacity

Two points cause most confusion:

CPU is compressible and memory isn't. Running out of CPU makes you slow. Running out of memory makes you dead.
The scheduler uses requests, not actual usage. A node can be "full" while its CPUs are nearly idle, because pods have reserved capacity they don't use.

Your QoS class follows from those settings. Guaranteed means requests equal limits for all containers, on both CPU and memory. BestEffort means none are set, and Burstable is everything else. Under node pressure, kubelet evicts BestEffort and over-request Burstable pods first.

Step 1: Collect the right data

You need at least two weeks of usage, including a month-end, a release and your busiest period, if you can. Measure each container, not each pod. Useful sources are:

CPU usage per container     rate of container_cpu_usage_seconds_total
Memory per container        container_memory_working_set_bytes
CPU throttling              container_cpu_cfs_throttled_periods_total
                            / container_cpu_cfs_periods_total
Requests and limits         kube_pod_container_resource_requests
                            kube_pod_container_resource_limits
OOM kills                   kube_pod_container_status_last_terminated_reason

Take these metric names from the kubelet and kube-state-metrics as they're exposed in your cluster. Use the working set for memory, because that's close to what the OOM killer goes by, whereas container_memory_usage_bytes includes reclaimable page cache and overstates your needs.

For each container, summarise usage as min, average, p95 and peak. The relationship between them matters more than any single number.

Step 2: Read the shape of the workload

The same average can call for very different settings. Here are three invented workloads, all averaging 200m CPU:

Workload	Avg	p95	Peak	Reading	Request	CPU limit
Steady API	200m	260m	340m	Tight range	260m to 300m	2× request, or none
Spiky API	200m	700m	1800m	Bursts matter	700m	High, or none
Nightly batch	200m	150m	3200m	Quiet most of the time, then heavy	Separate CronJob sizing	Depends on deadline

Averages would size all three the same, and two of them would suffer. Peak would oversize them all. p95 is a reasonable anchor for CPU requests on a latency-sensitive service. Use the peak for memory.

Step 3: Apply the rules by resource

CPU requests

Set the request near p95 for services where latency matters, and closer to the average for background work. Remember that replica count matters here. If one of three replicas fails, the survivors take 50% more traffic, so check whether your p95 still fits. Spare capacity for failover is not waste.

CPU limits: the throttling trap

CPU limits are enforced in periods, 100 ms by default. A limit of 500m gives the container 50 ms of CPU time in each 100 ms period, across all its threads.

Here's a worked example. A service with a 500m limit has a thread pool of 8 threads handling a burst, all runnable:

CPU time consumed per wall-clock millisecond: 8 ms
Quota: 50 ms
Time until quota is gone: 50 / 8 = 6.25 ms
Remaining time in the period, stopped: 100 − 6.25 = 93.75 ms

Every thread stops for nearly 94 ms, with the monitoring graph saying average CPU is well under the limit. A request that needed 20 ms of CPU across several slices can end up taking several hundred ms. The usual symptom is p99 latency that's far above p50, with nothing wrong in the application or its dependencies.

This is why many teams set no CPU limit on latency-sensitive services, relying on requests for fair sharing when a node is busy. Others keep limits to stop runaway tenants or to get predictable performance, or because a Guaranteed QoS class is needed. Whichever you choose, watch the throttled-period ratio. As a rough starting point, sustained throttling above a few percent on a latency-sensitive service deserves a look.

Runtimes also read the limit to size themselves: JVMs set their default GC and pool thread counts according to the container's CPU limit. Removing or raising a limit can change that behaviour, so set thread counts explicitly and test.

Memory requests and limits

Set the memory request based on the peak working set, plus headroom. Many teams set the limit equal to the request. That way, the pod doesn't rely on capacity that may not be there when it's needed, and it gets killed for its own excess rather than becoming an eviction target due to a neighbour.

Allow for growth over time. A slow leak looks fine in a two-week sample if pods restart weekly. Look at the working set across a pod's full lifetime. If it keeps going up until a restart, that's not a sizing problem.

Runtime-managed memory

Language runtimes hold memory by design, so the working set may not reflect true need.

JVM. The heap is only part of the picture: metaspace, thread stacks, code cache, direct buffers and GC structures all add to it. If you use -XX:MaxRAMPercentage=75 with a 2 GiB limit, the heap can reach 1.5 GiB, leaving 512 MiB for everything else. Shrinking the limit by looking at heap use alone is a common cause of OOM kills. See Java latency troubleshooting.
.NET. Server GC can hold plenty of memory when it's available. The runtime respects container limits, and settings such as DOTNET_GCHeapHardLimit exist for tighter control.
Node.js. Its heap limits can be set with --max-old-space-size, and off-heap buffers count too.
Go. GOMEMLIMIT gives the runtime a soft limit, usefully set below the container limit.

If a runtime expands to whatever it's given, measured usage doesn't tell you the minimum. To find that, reduce gradually under load, and observe GC time and latency.

Step 4: Check HPA before changing anything

If a workload scales by CPU with a HorizontalPodAutoscaler, utilization is calculated as a percentage of the request. Changing the request changes when it scales. For example:

Current state: request 1000m, target 70%, each pod using 350m. Utilization is 35%, and the HPA stays at its minimum.
After "right-sizing" to a 400m request: 350m / 400m = 87.5%. That's above 70%.
The desired replica count becomes ceil(current × 87.5 / 70). With 4 replicas, that's ceil(5) = 5.

So the total reserved capacity falls from 4,000m to 2,000m, which is good, but the replica count moves. Lower the target or adjust the minimum replicas when you make such changes, then watch for flapping. Also, avoid having both HPA and VPA act on the same metric for the same workload, since they'll work against each other.

Step 5: Get a second opinion from VPA

The Vertical Pod Autoscaler is an add-on project, and on OpenShift it's available as an operator. In Off mode, it only produces recommendations, with nothing changed:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: order-api
  namespace: payments
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: order-api
  updatePolicy:
    updateMode: "Off"

kubectl describe vpa order-api -n payments

The output lists target, lower-bound and upper-bound values for each container. Compare them with your own analysis. Don't accept them blindly: VPA learns from past behaviour, so it may not see a peak that hasn't happened yet. Newer Kubernetes versions also offer in-place pod resizing, which reached beta in 1.33. Check your version's documentation, as it means fewer restarts, though runtimes that size themselves at startup may not benefit.

Step 6: Roll out safely

Change one workload at a time, with a plan that can be reversed.

Start with the largest gap between request and p95. The first handful of workloads often accounts for much of the reclaimable capacity.
Avoid critical services until you've practised on lower-risk ones.
Reduce in steps: for instance, halfway towards the target. Observe through a full business cycle. Note that lowering requests does restart pods, so use your normal rollout process.
Watch p95 and p99 latency, error rate, throttling, OOM kills and restarts. Line up the change with these graphs. If you need help, see what-changed correlation.
Record the change and reason in version control, or wherever your manifests live.
Review quarterly. Traffic grows and code changes.

After rightsizing, you still have to reduce the cluster size, or nothing is saved. A cluster autoscaler or a planned node reduction realises the gain.

Governance helpers

LimitRange sets defaults and minimums per namespace, so a pod without any settings doesn't become BestEffort.
ResourceQuota caps a namespace's total requests, which gives teams an incentive to right-size and protects the cluster.

When not to right-size

Leave headroom deliberately for: workloads with a hard latency target, those that have strict failover needs, seasonal systems whose peak isn't in your sample and services with slow startup that can't scale out in time. A low-utilization number can be a sound business decision, as long as someone made it on purpose.

How Wrentrix helps

Wrentrix shows resource usage as min/avg/p95/peak for each workload on Kubernetes and OpenShift, along with right-sizing hints and HPA and event views. Change correlation lines up rollouts against latency and error spikes, so you can check that a resize didn't hurt anything. See the Kubernetes and OpenShift monitoring page. Applying a hint is your decision, through your normal release process.

Frequently asked questions

Should I set CPU limits in Kubernetes?

It's debated, and the answer depends on the workload. Without limits, latency-sensitive services avoid throttling and can use idle CPU, with requests providing fairness under contention. Keep limits where you need predictable performance, strict tenant isolation or a Guaranteed QoS class, and watch the throttling ratio.

What is the difference between requests and limits in Kubernetes?

A request is what the scheduler reserves for a container and is used for placement and CPU sharing. A limit is the maximum the container may use, enforced by throttling for CPU and an OOM kill for memory. A container can use more than its request as long as the node has spare capacity and it stays within its limit.

How do I know if a pod is being CPU throttled?

Compare throttled periods with total periods from container_cpu_cfs_throttled_periods_total and container_cpu_cfs_periods_total. A sustained high ratio on a service with unexplained tail latency is a strong clue. Average CPU can look moderate during heavy throttling, so don't rely on that alone.

How much headroom should I leave on memory limits?

There's no universal figure, as it depends on how much the peak varies and how much a restart costs. Start from the highest working set across a full business cycle and add a margin that your team is comfortable with. Reduce it gradually while watching for OOM kills, and take care with runtimes that hold on to memory.

Can I use VPA and HPA together?

Yes, but not on the same metric for the same workload, or they will fight each other. A common approach is HPA on CPU with VPA in Off mode, giving recommendations only. Another is to use HPA on a custom metric, leaving CPU and memory to VPA.