Kubernetes CPU Throttling: CFS Quota, cgroup v2 and the Limits-vs-Requests Latency Trap (2026)

Your dashboard says the pod averages 35 percent of its CPU limit, yet p99 latency has doubled and the service owner is certain the cluster is “slow”. Both facts can be true at once. Kubernetes CPU throttling is enforced in windows of 100 milliseconds, not in the one-minute averages that most graphs show, and a container with a limit can be frozen for most of a window after it spends its budget in a few milliseconds of parallel work. Nothing in the average hints at it. The stall shows up as tail latency, missed health checks, and garbage collection pauses that look inexplicably long.

This matters more in 2026 than it did when the problem was first widely documented. Nodes now routinely have 64 to 192 cores, so a runtime that sizes its thread pools to the node, but is billed against a two-CPU quota, can exhaust that quota in a couple of milliseconds. Meanwhile cgroup v2 has become the default substrate, in-place pod resize is stable, and the old argument over whether to set CPU limits at all has hardened into two camps. You will leave with the mechanism, the arithmetic, the detection queries, the kernel history, and a decision framework you can defend in a design review.

What this covers: how requests and limits map to cgroup files, the CFS bandwidth algorithm with worked numbers, why multi-threaded runtimes trip it, the metrics and commands that expose it, kernel fixes and CPU burst, Guaranteed QoS and the static CPU manager, the case for and against limits, runtime tuning for Go, JVM and Node.js, HPA and in-place resize interactions, and a debugging workflow.

Context and Background

Kubernetes models CPU as a compressible resource. If a container wants more CPU than it is entitled to, the kernel does not kill it as it would for memory; it delays it. That makes CPU feel safe to over-commit, and it is the reason defaults, templates, and admission policies have historically stamped a CPU limit onto every container. The limit looks like a seatbelt. In practice it is a rate limiter applied by the Linux scheduler, and rate limiters have burst behaviour that average-based dashboards cannot see.

The Kubernetes documentation on resource management describes the contract in two sentences. A request tells the scheduler how much to reserve when placing the pod and, per the docs, the kubelet reserves at least the request amount for the container. A limit means the kubelet enforces that the container is not allowed to use more than that. The mechanism behind the second sentence is the Completely Fair Scheduler (CFS) bandwidth controller, a feature merged into Linux long before containers were mainstream to let administrators cap groups of tasks.

Two things changed the landscape. First, cgroup v2 replaced the v1 file layout with a single unified hierarchy, renaming cpu.shares to cpu.weight and merging cpu.cfs_quota_us plus cpu.cfs_period_us into one file, cpu.max. Kubernetes has supported cgroup v2 for several releases, and the project published a notice in January 2026 about a revised conversion from v1 CPU shares to v2 weight because the original linear mapping produced very low weights. Second, the community accumulated a decade of post-mortems about latency caused by limits that were comfortably above average usage.

The incumbents in this space are not tools but conventions. Many platform teams copy a Helm chart default of limits.cpu: 500m onto every service. Many vendor guides recommend “requests equal limits” for the Guaranteed quality-of-service (QoS) class. Many SRE teams, after one bad incident, ban limits entirely. All three positions are defensible in some regimes and harmful in others, which is why this article spends as much time on the mechanism as on the recommendation.

If you are sizing pods first and tuning second, read in-place pod resize and rightsizing alongside this piece. Rightsizing sets the numbers; throttling analysis tells you whether those numbers behave the way you think when load arrives in bursts.

How Requests and Limits Become cgroup Files

In short: the CPU request becomes a relative weight (cpu.weight on cgroup v2, cpu.shares on v1) that only matters when CPUs are contended, while the CPU limit becomes a hard bandwidth cap, cpu.max, expressed as a quota of CPU-microseconds allowed per period, with a default 100 ms period. Requests shape sharing; limits cause throttling.

Figure 1: How a pod’s CPU request and limit travel through the kubelet and the OCI runtime into cgroup v2 files that the kernel scheduler enforces.

The figure traces the path. The kubelet computes values from the pod spec and hands them to the container runtime over the Container Runtime Interface (CRI). The low-level runtime, runc or crun, writes them into the container’s cgroup directory. Only after that does the kernel take over, and from then on Kubernetes is not involved in any scheduling decision. The kubelet does not “throttle” anything; it configures and then watches.

Requests: weight, not a guarantee of wall-clock time

On cgroup v1 the request in millicores maps to cpu.shares using the formula shares = milliCPU * 1024 / 1000. A 1000m request becomes 1024 shares, and 100m becomes 102, as the Kubernetes project’s own January 2026 write-up on the conversion notes. On cgroup v2 the kernel exposes cpu.weight instead, with a default of 100 and an allowed range of 1 to 10000 per the kernel cgroup v2 documentation.

The runtime has to translate shares into weight. The long-standing OCI mapping is linear across the two ranges, which for a 1 CPU request gives a weight of 39, compared with the default of 100. The Kubernetes enhancement discussion that led to the new formula cites exactly that figure as the problem: Kubernetes workloads at modest requests ended up with very low weight relative to non-Kubernetes processes such as system daemons, which sit at 100. I am deliberately not quoting the replacement formula here, because it is still rolling through runc, crun and kubelet releases; check the release notes of your runtime and Kubernetes version before assuming either mapping.

The critical conceptual point is that weight is proportional, and only operative when runnable threads outnumber available cores. If a node has idle cores, a container with a tiny request can use all of them. If the node is saturated, weights divide the CPU: two containers with weights 100 and 300 split the contended capacity 25 to 75. The request is therefore a priority under contention, not a reservation of cores and not a latency promise.

Limits: quota per period

The limit is converted to a quota. For a limit of 2 (2000m) and the default 100 ms period, the kubelet and runtime write 200000 100000 to cpu.max. The kernel documentation defines the format as $MAX $PERIOD with the default max 100000, meaning “the group may consume up to $MAX in each $PERIOD duration”, and max means no limit. The 100 ms default is also what the CFS bandwidth documentation lists for cpu.cfs_period_us in v1; the same page gives a minimum quota or period of 1 ms and an upper bound of 1 second for the period.

Notice what the quota is measured in: CPU time, summed across all cores. A two-CPU limit is 200 ms of CPU time per 100 ms of wall-clock time. Whether that is spent as two threads running continuously for the whole period or sixteen threads running 12.5 ms each is invisible to the quota. That indifference is the root of the problem.

Kubernetes also has a kubelet feature, cpuCFSQuota, which can disable quota enforcement node-wide, and a related knob to change the period; both are node-level decisions that bypass per-pod limits. They are rarely the right first step, but they exist and appear in the “remove limits” debate later.

QoS classes and where CPU fits

QoS classes are derived from requests and limits: Guaranteed when every container has memory and CPU requests equal to limits, Burstable when at least one has a request or limit but the pod is not Guaranteed, and BestEffort when none are set. CPU QoS does not decide eviction (memory pressure does that), but it does change cgroup placement: the kubelet puts pods into per-QoS parent cgroups, and the parent’s weight, derived from the sum of requests, affects how each class competes. BestEffort pods get a minimal weight, which means that under contention they starve first.

The two cgroup hierarchies you must keep apart

A frequent source of confusion is that the throttling counters from the kernel are non-hierarchical. The cgroup v2 documentation states that the five CFS bandwidth stats (nr_periods, nr_throttled, throttled_usec, nr_bursts, burst_usec) count only throttling from that cgroup’s own bandwidth limit, not from ancestors. In Kubernetes, the container cgroup typically carries its own cpu.max, and the pod-level cgroup may carry one too, computed as the sum of container limits when all containers have limits. A container can be throttled by the pod-level cap and show zero throttling on its own counters. When the container-level numbers look clean but latency is awful, read the pod cgroup’s cpu.stat as well.

The CFS Bandwidth Algorithm With Worked Numbers

The scheduling picture changes the moment you appreciate that quota is distributed to CPUs in slices, and that threads on different CPUs withdraw from one shared pool. The kernel’s CFS bandwidth control documentation describes it: unassigned quota is tracked globally and refreshed to the quota value at each period boundary; as threads consume bandwidth it is transferred to per-CPU run queues on demand, in slices; once all quota is assigned, further requests cause threads to be throttled until the next period. The slice size comes from sched_cfs_bandwidth_slice_us, which defaults to 5 ms.

Figure 2: Per-CPU run queues withdraw 5 ms slices from the global pool; when the pool is empty, every thread in the cgroup is throttled until the next period boundary.

Arithmetic example one: the parallel burst

Take an illustrative service with a limit of 1 CPU, so a quota of 100 ms per 100 ms period. It averages 0.4 CPU over a minute, which looks healthy. Now suppose a garbage collection cycle or a request fan-out wakes eight worker threads at the start of a period on eight different cores, and each wants 20 ms of CPU.

Total demand is 160 ms of CPU, but the budget is 100 ms. The eight threads run in parallel, so after 12.5 ms of wall-clock time they have collectively consumed 100 ms of quota. At that instant, the pool is empty and every thread in the cgroup is throttled. The remaining 87.5 ms of the period are dead time. The threads need another 7.5 ms each, and they resume only when the next period refills the pool. A task that would have completed in 20 ms of wall time on an unlimited container instead takes about 100 + 7.5 = 107.5 ms. That is roughly a 5x slowdown, and the minute-level average utilization of this container is still low.

The throttle ratio for that single period is 1 of 1, even though the container was running less than an eighth of the time. Over a minute (600 periods at 100 ms) the container might be throttled in 90 of them, producing a ratio of 15 percent while using only 0.4 CPU on average. This is exactly why the throttled-period ratio is more informative than CPU utilization.

Arithmetic example two: the slice lottery

The slice mechanism adds a second, subtler effect. With a 5 ms slice, a limit of 200 ms per period is 40 slices. A thread that wakes on a mostly idle CPU asks for a slice, runs 100 microseconds, blocks on I/O, and the leftover slice sits on that CPU’s run queue. The kernel docs say that when all threads on a CPU become unrunnable, all but 1 ms of the slice may return to the global pool, with the 1 ms retained as a compile-time minimum, min_cfs_rq_runtime. Across 64 cores with light, jittery traffic, that retained slack can reach tens of milliseconds, which is quota that is stranded on the wrong CPU when a different CPU needs it.

This is the same phenomenon that, in older kernels, was compounded by slice expiry: unused local slices were discarded at the period boundary, so a highly threaded, mostly idle application could be throttled in a large share of periods while using a small fraction of its quota. We return to that kernel history shortly because it determines which of your nodes are affected.

Why multi-threaded runtimes burn quota fastest

Quota is consumed by the sum across threads, but a runtime sets its parallelism from what it sees. If a process sees 64 cores and starts 64 worker threads, a burst of work will run in 1/64 of the wall-clock time and consume the whole quota almost immediately. The relationship is easy to state: the time to exhaust the quota within a period is roughly quota / active_threads. For a 2 CPU limit (200 ms) and 32 busy threads, that is 6.25 ms. After that the container sits idle for 93.75 ms.

Three types of burst cause this repeatedly. Stop-the-world garbage collection and parallel GC phases, which deliberately use all the cores they think they have. Request fan-out into thread pools, such as a web server that dispatches CPU-heavy JSON serialization across a pool. And startup, where class loading, JIT compilation and cache warm-up run in parallel across all visible cores and stack against the quota exactly when readiness probes are ticking.

Reading the counters the kernel gives you

On cgroup v2 cpu.stat contains usage_usec, user_usec, system_usec always, and when the controller is enabled nr_periods, nr_throttled, throttled_usec, nr_bursts and burst_usec. In v1 the equivalents are nr_periods, nr_throttled and throttled_time (nanoseconds). The throttled percentage over a window is delta(nr_throttled) / delta(nr_periods). The mean stall per throttled period is delta(throttled_usec) / delta(nr_throttled). In the first worked example it would be 87,500 microseconds. A value near the full period means threads are frozen almost the entire time they could be running.

The exact cadvisor and Prometheus names appear in the next section, but the principle belongs here: the kernel counts periods and throttled periods, so the best ratio is by count of periods, not by CPU seconds. A container using 40 percent of its limit on average can sit in the 20 to 60 percent throttled-period range if its load is bursty.

Detecting Throttling: Metrics, Pressure and Commands

You cannot fix what your dashboards cannot see. The signals fall into three layers: kernel counters exported by the kubelet’s embedded cAdvisor, pressure stall information (PSI), and direct inspection on the node.

The Prometheus metrics

The cAdvisor counters most teams use are container_cpu_cfs_periods_total (enforcement intervals elapsed), container_cpu_cfs_throttled_periods_total (intervals in which the container was throttled), and container_cpu_cfs_throttled_seconds_total (total time spent throttled). They map directly to nr_periods, nr_throttled and the throttled-time counter in cpu.stat. Some cAdvisor versions on cgroup v2 hosts export the same series, but verify on your cluster that all three exist, because historic gaps in v2 support have appeared in specific kubelet releases.

The headline query is the throttled-period ratio:

sum by (namespace, pod, container) (
  rate(container_cpu_cfs_throttled_periods_total{container!=""}[5m])
)
/
sum by (namespace, pod, container) (
  rate(container_cpu_cfs_periods_total{container!=""}[5m])
)

A companion query estimates the average stall per throttled period, which separates “throttled often but briefly” from “throttled rarely but disastrously”:

sum by (namespace, pod, container) (rate(container_cpu_cfs_throttled_seconds_total{container!=""}[5m]))
/
sum by (namespace, pod, container) (rate(container_cpu_cfs_throttled_periods_total{container!=""}[5m]))

And a utilization query to hold alongside them, because the contrast between low usage and high throttling is the diagnostic signature:

sum by (namespace, pod, container) (rate(container_cpu_usage_seconds_total{container!=""}[5m]))
/
sum by (namespace, pod, container) (kube_pod_container_resource_limits{resource="cpu"})

There is no universal threshold. As a reasoning aid rather than a standard, treat sustained ratios above 25 percent on latency-sensitive services as worth investigating, and any ratio above a few percent on a service with a tight p99 objective as a candidate cause when latency regresses. Batch jobs can run at 80 percent throttled and be perfectly healthy, because they care about throughput and a stall costs nothing they can see.

Why the five-minute rate hides things

Prometheus rate() over five minutes averages 3,000 periods. A service that is throttled hard for four seconds out of every minute shows a ratio near 6 percent, which may sit below your alert threshold even as every request arriving in those four seconds misses its deadline. If you suspect this pattern, query a one-minute rate, or better, sample cpu.stat locally at a one-second interval during a load test and plot the deltas. Latency histograms from the application are the other half of the puzzle: align throttled-seconds spikes with latency spikes by timestamp.

Pressure stall information

PSI, exposed in cgroup v2 as cpu.pressure, reports the share of time that tasks were stalled waiting for CPU. The file is a nested-keyed file, per the kernel docs, with the full format defined in the PSI documentation (Documentation/accounting/psi.rst). Typically it carries a some line with avg10, avg60, avg300 and a cumulative total in microseconds; whether a full line is present for CPU depends on your kernel version, so read the file before building a dashboard on it.

PSI complements the CFS counters rather than replacing them. Throttling is quota exhaustion within a cgroup; PSI also captures plain contention, where the node is out of CPU even though no quota was exceeded. When cpu.pressure is high and throttling is low, you have a node that is oversubscribed (requests too low, too many neighbors), not a limit set too tight. When throttling is high and PSI at the node level is quiet, the cure is the container’s own configuration. Kubernetes exposes PSI metrics through the kubelet on supported versions and feature configurations; confirm availability in your distribution before relying on it.

On-node inspection

On a cgroup v2 node you can read the files directly. First find the container’s cgroup path from the pod’s UID and container ID, then:

# Locate the cgroup (systemd driver example; path varies by runtime and driver)
CG=/sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod<POD_UID>.slice/cri-containerd-<CONTAINER_ID>.scope

cat $CG/cpu.max          # e.g. "200000 100000" = 2 CPUs of quota per 100 ms
cat $CG/cpu.weight       # relative share derived from the request
cat $CG/cpu.stat         # nr_periods nr_throttled throttled_usec ...
cat $CG/cpu.pressure     # PSI for this cgroup

# Watch deltas once per second
while sleep 1; do
  awk '/nr_periods|nr_throttled|throttled_usec/ {printf "%s=%s ", $1, $2} END {print ""}' $CG/cpu.stat
done

On a cgroup v1 node, the files are cpu.cfs_quota_us, cpu.cfs_period_us, cpu.shares and cpu.stat (with throttled_time in nanoseconds), under /sys/fs/cgroup/cpu,cpuacct/kubepods/.... From inside a container, reading /sys/fs/cgroup/cpu.max and /sys/fs/cgroup/cpu.stat gives the container’s own view, and is the quickest way to prove to a skeptical developer that the quota is the culprit.

If you need to see which threads are being stalled, perf sched and eBPF scheduler tools can show run queue latency per thread, though they need privileged access on the node. For most investigations, the three Prometheus queries plus a latency histogram are enough to decide.

Kernel History: What Was Fixed and What Still Bites

Throttling complaints in Kubernetes surfaced widely from around 2018. The popular narrative, that CFS quota was “buggy”, is partly true and worth stating precisely, because it determines whether upgrading your nodes’ kernel changes anything.

The slice expiry bug

The key patch is titled “sched/fair: Fix low cpu usage with high throttling by removing expiration of cpu-local slices”, authored by Dave Chiluk and committed by Peter Zijlstra in August 2019 per the commit record. The commit message describes the symptom: highly threaded, non-CPU-bound applications under a cfs quota hit a high percentage of throttled periods while not using their allocated quota, typical of user-interactive workloads in Kubernetes or Mesos on multi-core machines.

The root cause was that each CPU’s run queue received a bandwidth slice and, if it was not fully used within the period, the slice expired at the period boundary and the quota was lost. The author traced the underlying behavior to an earlier commit introduced in v3.16-rc1 in 2014, having gone unnoticed for nearly five years, and cited Kubernetes issue 67577. The fix removed the runtime expiration mechanisms so unused slices carry past the period boundary. The cost, stated openly in the message, is that a group can slightly exceed its quota in some periods, bounded by the leftover on each per-CPU run queue (typically up to 1 ms per CPU through min_cfs_rq_runtime). The commit reports an almost 30x improvement in an artificial test with a 10 ms per 100 ms quota on an 80-CPU machine, and that test is published as fibtest. I report that number as the patch author’s artificial benchmark, not as a prediction for your service.

The commit also records an earlier fix, 512ac999d275, for the non-expiration of per-CPU slices, which the later patch built on. Stable-kernel backports reached the 4.19 series and other long-term branches, but distribution kernels vary. Check your vendor’s changelog, not just the version number, because a node image labeled with a base kernel version may or may not include the backport. If you run a modern managed node image, you almost certainly have the fix; if you run an old custom AMI or an on-prem fleet pinned to a 4.x kernel, verify.

What the fix did not change

The fix removed a class of spurious throttling. It did not remove throttling for legitimately parallel bursts like the one in example one. A container that burns its whole quota in 12.5 ms is not suffering a bug; it is being rate limited exactly as designed. Many incident reports blamed “the kernel bug” for what was in fact a thread-count-versus-quota mismatch, and upgrading the kernel left them unchanged.

CPU burst

The kernel later gained a mechanism designed for the opposite situation: a container whose average is well under its limit but which needs short surges. The CFS burst feature is documented in the bandwidth control page: cpu.cfs_burst_us sets the maximum accumulated unused run-time, 0 disables accumulation, and positive values must be no larger than the quota. In cgroup v2 it appears as cpu.max.burst with a default of 0 and a range from 0 to the max value. The documentation’s own example uses a 50 ms period, a 20 ms quota (40 percent of one CPU) and a 10 ms burst, an extra 20 percent of one CPU that can be spent after periods of underuse. The page describes the idea as borrowing against future underrun with bounded interference.

The burst controller landed in the Linux 5.x series in 2021 (reported as 5.14; verify against your kernel), and OCI runtime-spec and runc support followed. Kubernetes does not expose burst as a first-class pod field in the stable API that I could verify, so using it today generally means a node-level agent, an NRI plugin or a custom runtime hook that writes the file. Treat it as a node-team tool, not an application-team switch. The statistics nr_bursts and burst_usec tell you whether it is being used.

Burst helps the “light average, spiky demand” profile in example one, but it has a ceiling by design. In the 8-thread example, a burst allowance of 60 ms would let the 160 ms demand complete after 10 ms more of the period instead of in the next one, but it will do nothing for a service whose sustained demand exceeds the limit.

Guaranteed QoS, the Static CPU Manager and Exclusive Cores

If throttling comes from sharing cores through a quota, the cleanest cure is to stop sharing: give the container whole cores it owns. The kubelet’s CPU Manager does this with the static policy.

What the static policy requires

According to the CPU management policies documentation (v1.31 edition; confirm for your version), the default policy none applies no affinity beyond what the OS scheduler does, and CFS quota enforces limits for Guaranteed and Burstable pods. The static policy gives exclusive CPUs to containers in Guaranteed pods with integer CPU requests. Fractional requests, such as 1.5, do not qualify and run in the shared pool, as do Burstable and BestEffort pods. The docs state that for exclusive containers CFS quota does not bound them; their cpuset size equals their integer CPU limit, so usage is limited by the cpuset itself.

The shared pool begins with all node CPUs minus reserved ones, and the static policy requires a reservation greater than zero via --kube-reserved, --system-reserved or --reserved-cpus so the shared pool is never empty. Policy options include full-pcpus-only, which allocates whole physical cores and rejects pods that cannot be satisfied with SMTAlignmentError; per the same page, that option is beta and on by default in that version, while others like distribute-cpus-across-numa and align-by-socket were alpha. Changing policy on a live node requires draining it, stopping the kubelet, deleting /var/lib/kubelet/cpu_manager_state, changing the setting and restarting; skipping the state file deletion causes the kubelet to crash-loop.

A pod that qualifies

apiVersion: v1
kind: Pod
metadata:
  name: latency-critical-api
spec:
  containers:
  - name: api
    image: registry.example.com/api:1.0
    resources:
      requests:
        cpu: "4"
        memory: 8Gi
      limits:
        cpu: "4"        # integer and equal to request -> Guaranteed + exclusive cores
        memory: 8Gi

That container gets four dedicated CPUs, removed from the shared pool and placed in its cpuset. Its throttling counters will remain near zero because the cpuset, not a time budget, caps its parallelism. Its worst-case behaviour changes from “stalled for the rest of the period” to “runs at most four threads at a time”, which is a much more predictable failure mode.

The costs of exclusivity

Exclusive cores are not free. Capacity fragmentation is the first cost: a node with 32 CPUs, 2 reserved, can host seven 4-CPU exclusive pods and then the remaining 2 CPUs serve everything else. Idle exclusive cores cannot be borrowed by neighbours, so average utilization drops, which in a cost-driven cluster is a real price. The second cost is operational: the policy change procedure above is a drain-and-restart. The third is that the benefit applies only to integer-CPU Guaranteed pods; a sidecar with a fractional request in the same pod disqualifies that sidecar from exclusivity, and the pod’s other containers can still qualify per container, but the shared-pool sidecar remains under quota rules. Plan sidecar sizing deliberately.

For GPU nodes, where host CPU threads feed accelerators, the same logic matters in a different guise: data loader threads throttled by a limit starve the GPU. See Karpenter versus Cluster Autoscaler for GPU nodes for how node shapes interact with this, and the discussion of checkpointing on interruptible capacity in spot GPU training and fault tolerance, where a throttled data pipeline can quietly lower the effective cost-efficiency of expensive hardware.

Limits Versus Requests: Both Sides of the Argument

The debate over CPU limits is the most visible consequence of the mechanism above. It helps to separate the technical claims from the cultural ones.

The case for removing CPU limits

The argument runs in five steps. Requests already guarantee proportional share under contention, so a pod will not starve its neighbours when the node is full. Limits only matter when the node has spare CPU, and in that case throttling a container wastes idle cores. A limit imposes a latency penalty in bursts that the average cannot reveal. Setting the limit high enough to avoid throttling makes it meaningless as protection. Therefore, set accurate requests, leave limits unset, and use node-level monitoring for saturation.

This works well when the cluster is single-tenant or trusted, when requests are honest, and when node-level headroom is managed. Its proponents also point out that removing limits exposes true demand: you see the 12-core burst your service wanted instead of a flat line at the limit, which feeds better rightsizing.

The case for keeping CPU limits

Limits are the only mechanism that makes a runaway loop, an infinite retry storm or a compromised workload unable to consume a whole node’s CPU. With no limit, a burst can be bounded only by cpu weight, which gives it a proportional share, not a ceiling; the node’s other tenants lose throughput even if they are not starved entirely. Limits give predictable performance for benchmarking and capacity tests, because behaviour does not depend on how busy the neighbours are. In multi-tenant platforms, limits are part of a fairness contract and enforced by LimitRange and ResourceQuota. Some cost models also need them: bin-packing and chargeback assume a cap.

There is a second, subtle argument. Without limits, a service tuned in a quiet staging cluster will behave differently on a busy production node, because it gets less than the spare capacity it benchmarked with. Limits make that performance a stable contract in both places. If the limit is the same as the request, the pod is Guaranteed, and the performance difference between environments narrows.

The position this article defends

My view, which is opinion informed by the mechanism and not a measured result: the correct question is not “limits or no limits” but “what thread count does this process run, and what does it do in bursts?” Three rules follow. If the runtime’s parallelism is aligned to the limit (GOMAXPROCS equals the CPU limit, JVM active processor count equals the limit), then limits are low-risk, because the process will not try to use more CPU in parallel than the quota allows. If the workload is latency-sensitive, bursty and the cluster is trusted, drop limits and keep honest requests. If the workload is latency-critical with steady demand, use Guaranteed QoS with integer CPUs and the static policy, which achieves isolation without time-slicing. Everything else, especially multi-tenant, keeps limits, but sized by a measured p99 of burst consumption rather than a copy-pasted constant.

Figure 3: A decision path for choosing between requests only, requests equal to limits, and exclusive cores, based on latency sensitivity and tenancy.

The tree compresses the argument: tenancy and latency profile decide the policy, and exclusivity is reserved for the steady, latency-critical corner.

Aligning Runtime Parallelism With the Quota

Whichever policy you choose, the single highest-leverage fix is to make the runtime’s idea of “how many CPUs do I have” match the quota. Most throttling in practice is a mismatch between thread count and limit, not a limit that is too small for the work.

Go: GOMAXPROCS

Historically the Go runtime set GOMAXPROCS to the number of logical CPUs, ignoring cgroup quotas, so a Go service with a 2 CPU limit on a 64-core node ran 64 scheduler threads and burned its quota in about 3 ms of busy time. The workaround was Uber’s automaxprocs library, imported for its side effect. Since Go 1.25 the runtime itself does the job: if the process’s cgroup has a CPU bandwidth limit below the logical CPU count, GOMAXPROCS defaults to that lower limit. The release notes stress that the runtime ignores CPU requests and considers only limits, and that it periodically re-reads limits and the logical CPU count, which matters for in-place resize. Setting the environment variable or calling runtime.GOMAXPROCS disables both behaviours, and the containermaxprocs=0 and updatemaxprocs=0 GODEBUG settings disable them individually. runtime.SetDefaultGOMAXPROCS restores the default.

Two details deserve attention. A fractional limit such as 1500m cannot be an integer number of Ps, so the runtime rounds; check the release notes for the rounding rule in your version, and remember that a rounded-up value still exceeds a fractional quota in bursts. And if you removed limits, the runtime sees no bandwidth limit and uses all logical CPUs, which is what you want in that regime, but means a burst can spread across the entire node.

On Go versions before 1.25, the guidance is: import go.uber.org/automaxprocs, or set GOMAXPROCS through the downward API:

env:
- name: GOMAXPROCS
  valueFrom:
    resourceFieldRef:
      resource: limits.cpu     # integer rounding applies; divisor defaults to 1

JVM: container awareness and GC threads

The HotSpot JVM has been container-aware since JDK 10 with backports to 8u191, and cgroup v2 support arrived in later releases; verify the exact minor versions for your vendor build. The JVM derives Runtime.availableProcessors() from the cgroup, and from that sizes the default GC thread count, the common fork-join pool and the JIT compiler threads. A fractional limit is rounded up, so a 1500m limit yields 2 processors. Use -XX:ActiveProcessorCount=N to override explicitly, and -XX:ParallelGCThreads and -XX:ConcGCThreads when you want GC parallelism below the processor count.

The JVM-specific throttling trap is startup. With a small limit, JIT compiler threads, class loading and GC all contend for 100 ms windows, so a service can fail readiness probes while it is merely compiling. Mitigations include a higher temporary limit during startup (in-place resize makes this practical, see below), CDS or AOT features, and startup probes with generous thresholds instead of tight readiness probes.

Node.js and Python

Node.js has no cgroup-aware thread count for its libuv pool, which defaults to four threads regardless of limits, controlled by UV_THREADPOOL_SIZE. Worker threads, cluster forks and native addons are chosen by the application. os.availableParallelism() in recent Node versions reflects the process’s allowed CPUs, but whether it honours cgroup quotas depends on the version; test it inside a container. For Python, os.cpu_count() reports node CPUs and ignores the quota, which causes multiprocessing.Pool() defaults and Gunicorn worker heuristics to oversubscribe. Compute worker counts from /sys/fs/cgroup/cpu.max in your entrypoint or set them explicitly through configuration.

A small helper for any language

#!/bin/sh
# Print the effective CPU limit (rounded up) from cgroup v2, fall back to nproc
if [ -r /sys/fs/cgroup/cpu.max ]; then
  read quota period < /sys/fs/cgroup/cpu.max
  if [ "$quota" != "max" ]; then
    echo $(( (quota + period - 1) / period ))
    exit 0
  fi
fi
nproc

Use the output to set worker counts in an entrypoint script, and log it at startup, so that a mismatch is visible in the first line of the container log instead of discovered during an incident.

Interactions: HPA, In-Place Resize and Observability Pipelines

Horizontal Pod Autoscaler (HPA)

The HPA computes utilization as usage divided by the request, not the limit. This produces two traps. A throttled container cannot rise above its limit, so if the limit is close to the request, measured utilization saturates near 100 percent and the HPA scales out, sometimes correctly, sometimes only because a bursty pattern briefly pinned the ratio. The opposite trap arises when you remove limits: a container can use several times its request, and the HPA sees utilization of 300 percent and scales aggressively, which may be right, but it also means your replicas are measured against a number (the request) that is no longer a cap. Honest requests become more important, not less, when limits go away.

Throttling also delays the signal. A pod stalled 80 percent of a period reports lower CPU usage than it demanded, so the HPA under-reacts. If the real symptom is latency, scale on a latency or queue-depth metric through custom or external metrics, and keep CPU as a secondary signal. The same reasoning appears in agent and LLM workloads, where traces are the better trigger; see OpenTelemetry GenAI semantic conventions for observability for how to attach latency and span data that lets you correlate stalls with infrastructure counters.

In-place pod resize

In-place pod resize reached stable in Kubernetes 1.35 per the project’s December 2025 announcement, after alpha in 1.27 and beta in 1.33. Before it, changing a container’s CPU meant recreating the pod. For throttling, resize matters in three ways. You can raise a limit on a running pod to stop an active incident without a restart. You can give a JVM a larger limit during startup and shrink it after warm-up. And you can test a hypothesis cheaply: raise the limit by 2x, watch the throttle ratio, and see whether latency follows. Details of resizePolicy, the resize subresource and QoS constraints are in the official task page; do not assume changing the QoS class is possible through a resize, and check how your runtime’s GOMAXPROCS or JVM processor count reacts to the new cgroup value (Go 1.25 re-reads it; many JVM configurations do not).

If you use an automated rightsizer, give it throttling data as input as well as usage data. A recommender that sets limits to p95 usage will set them below burst demand by construction, creating the trap it was meant to avoid. The rightsizing patterns in the in-place resize article cover this feedback loop.

Sidecars and service meshes

Sidecars share the pod’s CPU budget in their own cgroups, each with its own limit. A proxy sidecar with limits.cpu: 200m on a busy service becomes the throttled component, and the symptom appears as latency added by the mesh, not as a CPU problem. Check per-container throttle ratios, not per-pod aggregates. Pod-level resources, a newer Kubernetes capability, let you set a shared budget across containers; check the current status for your version before relying on it.

A Debugging Workflow You Can Run in an Hour

Figure 4: A decision workflow from a latency alert to a verified fix, branching on throttle ratio, pressure signals and thread count versus limit.

The workflow in Figure 4 has a deliberate ordering: confirm throttling exists before touching configuration, find out why before changing the limit, and re-measure afterward.

Step 1: Confirm the symptom. Run the throttle-ratio query for the suspect container over the incident window. If the ratio is under a percent or two, throttling is probably not the cause; look at garbage collection logs, lock contention, noisy I/O or network. If the ratio spikes in step with latency, proceed.

Step 2: Separate throttling from contention. Read node-level and container-level cpu.pressure. High node pressure with low throttling means the node is oversubscribed: lower the number of pods per node, raise requests, or add nodes. Low pressure with high throttling means the container’s own quota is binding.

Step 3: Compare threads to limit. Inside the container, count runnable threads at peak (ps -eLo pid,tid,stat,comm | awk '$3 ~ /R/' | wc -l during a burst, or use top -H). Compare to the CPU limit. If peak runnable threads are a multiple of the limit, align runtime parallelism first.

Step 4: Characterize the load. Bursty or sustained? Compute usage / limit over one minute: if it is low (below 50 percent) but the throttle ratio is high, the load is bursty and the options are to raise or remove the limit, or enable burst. If usage is near the limit, the limit is genuinely too small: raise requests and limits together, or scale out.

Step 5: Test with a controlled change. Use in-place resize or a canary deployment with a limit at 2x and compare p99 and throttled seconds against the control. A change that removes throttling but does not reduce latency means throttling was a symptom, not the cause.

Step 6: Record the decision. Write down the thread count, limit, observed ratio and the measured effect in the service’s runbook. Without this, the next engineer re-learns it during the next incident.

A load-test harness

To reproduce throttling in a test cluster, run a deliberately parallel spinner under a one-CPU limit and watch the counters:

apiVersion: v1
kind: Pod
metadata:
  name: throttle-lab
spec:
  containers:
  - name: spin
    image: busybox:1.36
    command: ["/bin/sh", "-c"]
    args:
    - |
      # 8 short bursts of parallel work each second, then idle
      while true; do
        for i in 1 2 3 4 5 6 7 8; do
          (timeout 0.02 sh -c 'while :; do :; done') &
        done
        wait
        sleep 0.9
      done
    resources:
      requests: { cpu: "500m" }
      limits:   { cpu: "1" }

Here eight busy loops for 20 ms of wall time each consume up to 160 ms of CPU, against a 100 ms quota. Average usage stays near 0.16 CPU, well under the 1 CPU limit, yet the container is throttled at the start of each second. Compare nr_throttled growth with container_cpu_usage_seconds_total; this is the cleanest demonstration to give a team that insists average CPU proves the limit is fine. (The loop timings are illustrative; shell process start-up overhead will shift the exact numbers.)

Trade-offs, Gotchas, and What Goes Wrong

Raising the limit may not help. If the runtime scales threads to the limit-blind node size, raising the limit from 2 to 4 CPUs only doubles the quota; a 64-thread burst still burns it in 6 ms instead of 3 ms. Fix parallelism first, then size.

Removing limits moves risk, it doesn’t remove it. Without limits, node CPU saturation becomes the failure mode. One pod’s runaway loop reduces everyone’s share toward their weights, system daemons at weight 100 can compete with pods whose requests convert to low weight, and kubelet or runtime heartbeats can slow, leading to NotReady nodes. Reserve CPU for system daemons through system-reserved and kube-reserved, and alert on node-level PSI.

Throttling can cascade through retries. A throttled service times out, clients retry, and retries add load to the same stalled service, increasing bursts. Circuit breakers and retry budgets are part of the fix.

Probes can kill healthy pods. A liveness probe with a one-second timeout sampled during an 87 ms stall is usually fine, but during repeated throttled periods a few consecutive failures restart the pod, and the restart itself is a startup burst that is throttled again. Use startup probes, and keep liveness probes cheap and lenient.

Hierarchical effects. Pod-level and container-level cgroups both carry quota. The non-hierarchical counters mean you can miss the real culprit, as noted earlier. Also, cgroup v1 and v2 differ in how sub-cgroups sum shares; a mixed fleet during migration can show different behaviour for the same manifest, especially because of the share-to-weight conversion change that is rolling through runtimes.

Fractional limits and rounding. A limit of 1500m yields a runtime of 1 or 2 threads depending on rounding, and a mismatch between the runtime’s integer view and the quota’s fractional budget leaves either unused quota or early throttling.

Kernel assumptions. Fixes depend on the kernel build. Confirm the slice-expiry fix is in your node image, and treat CPU burst as a kernel-and-runtime feature that must be verified, not assumed.

Anti-patterns to avoid: a cluster-wide default limit of a few hundred millicores; setting limits at p95 of average usage; using CPU utilization alone for HPA on bursty services; and treating requests == limits as a cure-all without considering the parallelism mismatch.

Practical Recommendations

Start with measurement, because the right setting depends on the shape of the load and you have the data to learn it. Export the three cAdvisor counters, build the throttle-ratio and stall-per-period panels, and review them per container weekly for latency-critical services. Add node-level PSI alerts so that removing a limit never leaves the node unobserved.

Then fix parallelism before capacity: use Go 1.25 or automaxprocs, set the JVM processor count explicitly where fractional limits make rounding ambiguous, and compute worker counts in Python and Node from the cgroup. After that, choose a regime per workload class. Steady, latency-critical, integer-CPU services belong on Guaranteed QoS with the static policy where the node pool can afford fragmentation. Bursty, latency-sensitive services on trusted clusters are good candidates for request-only. Batch and multi-tenant workloads keep limits, with LimitRange guardrails.

Keep the decision reversible. In-place resize and canary comparisons let you test a change on a few replicas, and a written runbook preserves the reasoning.

[ ] Export container_cpu_cfs_throttled_periods_total, container_cpu_cfs_periods_total and container_cpu_cfs_throttled_seconds_total.
[ ] Alert on throttle ratio for latency-sensitive services, with a one-minute view in the dashboard.
[ ] Verify node kernels include the slice-expiry fix and know whether CPU burst is available.
[ ] Align GOMAXPROCS, JVM processors and worker pools with the limit.
[ ] Use honest requests, since HPA and weights depend on them.
[ ] Reserve CPU for system daemons before removing limits.
[ ] Document each service’s regime and the evidence for it.

Frequently Asked Questions

Why is my container throttled when CPU usage is low?

Throttling is decided per 100 ms period on summed CPU time across all threads, not on average utilization. A container with many threads can spend its entire quota in a few milliseconds of wall-clock time and then sit frozen until the next period, while the one-minute average stays low. Compare the throttled-period ratio with utilization; a low-usage, high-throttle pair is the signature of bursty parallel work under a tight limit.

Should I remove CPU limits in Kubernetes?

It depends on tenancy and workload shape. On a trusted, single-team cluster with honest requests and node-level monitoring, removing limits from latency-sensitive services avoids needless throttling. On multi-tenant clusters, limits protect neighbours and make performance predictable. A middle path is Guaranteed QoS with integer CPUs and the static CPU manager policy, which gives isolation without time-slicing. Whatever you choose, align runtime thread counts with the CPU you actually have.

What does cpu.max mean in cgroup v2?

cpu.max holds two values, a maximum and a period, both in microseconds, with the kernel default of max 100000. The group may consume up to the maximum in each period. A Kubernetes limit of 2 CPUs becomes 200000 100000. The word max in the first field means no limit. Kubernetes derives this from limits.cpu, while cpu.weight is derived from the request.

How do I measure Kubernetes CPU throttling?

Use cAdvisor metrics: divide the rate of container_cpu_cfs_throttled_periods_total by the rate of container_cpu_cfs_periods_total to get the throttled-period ratio, and use container_cpu_cfs_throttled_seconds_total for stall time. On the node, read cpu.stat and cpu.pressure in the container’s cgroup. Use short windows such as one minute, because five-minute rates hide short severe stalls.

Does the CPU request affect throttling?

Not directly. The request becomes cpu.weight (or cpu.shares on v1), which sets proportional sharing when the node is contended, and it is also what the scheduler reserves and what HPA divides by. Throttling comes from the limit’s quota. The request matters indirectly: requests set equal to limits yield Guaranteed QoS, and with integer values under the static policy they provide exclusive cores that avoid quota-based throttling.

Did the Linux kernel fix CFS throttling?

It fixed one real bug and added one tool. A 2019 patch removed the expiry of per-CPU quota slices, which caused spurious throttling in highly threaded, low-usage applications, per the commit message. CPU burst, available in newer kernels, lets groups bank unused quota. Neither removes throttling for legitimately parallel bursts that exceed the quota within a period; that behaviour is by design, so thread counts and limits still need to match.