On this page
The HorizontalPodAutoscaler is usually the first autoscaler a Kubernetes team turns on and the last one it understands. A CPU target of 70 per cent is easy to write; knowing why the deployment took four minutes to scale down, or why it added eight pods at once and then sat saturated while they started, needs the algorithm. Since Kubernetes 1.23 the stable API is autoscaling/v2, and its behavior field gives direct control over how fast the HPA moves in each direction.
This article explains what the controller does on each sync, as of Kubernetes 1.29 (March 2024), which metrics to scale on, how behavior changes the outcome, and how HPA fits with the other autoscalers. The sample code replays a load trace through the same decision steps, so you can see the effect of a configuration before applying it.
What happens on each sync

The HPA controller runs inside kube-controller-manager and evaluates every HPA on a fixed interval, 15 seconds by default. For each metric it computes desiredReplicas = ceil(currentReplicas × currentMetricValue ÷ desiredMetricValue). If the ratio is within the tolerance of 1.0 (0.1 by default), it does nothing. With several metrics it takes the highest result, so a CPU metric and a queue metric can be combined safely.
Utilization targets are percentages of the pod’s resource request. A container without a CPU request gives the HPA nothing to measure against, and a request set far below real usage makes every pod look overloaded. Pods that are not yet ready, or have no metrics yet, are treated conservatively: when scaling up, not-ready pods count as using none of the target, which damps the scale-up rather than overshooting.
The raw recommendation then passes through stabilisation and rate limits. The controller keeps the recommendations from the stabilisation window; for scale-down it uses the highest of them, so a brief dip does not remove pods. Rate-limit policies cap how many pods may be added or removed per period. If you set no behavior, scale-up has no window and may add 100 per cent or 4 pods per 15 seconds, whichever is more; scale-down waits 300 seconds and may then remove up to 100 per cent per 15 seconds.
%%{init: {'theme': 'base', 'themeVariables': {'signalColor': '#0073BB', 'signalTextColor': '#0F172A', 'actorBkg': '#FFF3E0', 'actorBorder': '#FF9900', 'actorTextColor': '#232F3E', 'noteBkgColor': '#FFF8E7', 'noteBorderColor': '#FF9900', 'noteTextColor': '#232F3E', 'lineColor': '#232F3E'}}}%%
sequenceDiagram
participant H as HPA controller
participant M as Metrics API
participant A as API server
participant C as Node autoscaler
loop Every 15 seconds
H->>M: Read pod CPU usage
M-->>H: Usage per pod
Note over H: Ratio, tolerance,<br/>window, policies, min/max
H->>A: Update replicas on /scale
end
A-->>C: New pods are Pending
C->>A: Add nodes, pods scheduled
Note over A: Pods start, pass readiness,<br/>then receive trafficThe sequence shows why scaling is slower than the 15-second sync suggests. New pods need a node, an image pull, application start-up and a passing readiness probe before they help. If the node autoscaler has to add a node, that is minutes, not seconds. The target utilisation is the buffer that covers this gap.
Choosing the metric
CPU utilisation works for services whose cost grows with request volume and where CPU is the bottleneck. Memory is rarely a good scaling signal: many runtimes do not release memory when load falls, so the HPA scales up and never back down. From Kubernetes 1.27 a ContainerResource metric (beta, on by default) can target one container’s usage, which avoids a sidecar’s CPU skewing the average.
For services limited by something other than CPU, scale on what users feel or what is queued: requests per second per pod, in-flight requests or queue depth through the custom or external metrics APIs. These need an adapter such as the Prometheus Adapter, or KEDA, which runs its own metrics server and manages an HPA for you.
Running the Vertical Pod Autoscaler in update mode and an HPA on CPU or memory for the same workload makes them react to each other: VPA raises the request, utilisation falls, HPA removes pods, utilisation rises. Use VPA in recommendation mode to set requests, or scale the HPA on a custom metric.
Tuning behavior
This HPA keeps one replica per Availability Zone, targets 65 per cent CPU to leave room while new pods start, scales up as fast as the default, and scales down slowly: it waits 10 minutes, then removes the smaller of 20 per cent or 2 pods per minute.
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: orders-api
namespace: orders
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: orders-api
minReplicas: 3 # one per Availability Zone
maxReplicas: 40 # a ceiling you can afford and the database can take
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 65 # leaves headroom while new pods start
behavior:
scaleUp:
stabilizationWindowSeconds: 0
selectPolicy: Max
policies:
- type: Percent
value: 100
periodSeconds: 15
- type: Pods
value: 4
periodSeconds: 15
scaleDown:
stabilizationWindowSeconds: 600 # ride out dips shorter than 10 minutes
selectPolicy: Min
policies:
- type: Percent
value: 20
periodSeconds: 60
- type: Pods
value: 2
periodSeconds: 60Whether the slow scale-down is worth it depends on traffic. The sample replays a trace through the HPA steps and reports replica-minutes (a proxy for cost), syncs spent saturated, and the number of changes:
"""Replay a load trace through the HorizontalPodAutoscaler algorithm.
Implements the autoscaling/v2 decision steps the kube-controller-manager
applies on every sync (15 s by default): the replica ratio with a 0.1
tolerance, the stabilisation window, scaleUp/scaleDown rate-limit policies
with selectPolicy, and min/max replicas. Use it to see how a `behavior`
block reacts to a recorded traffic pattern before you apply it to a cluster.
Simplifications (documented so results are read correctly):
- one Resource metric with a Utilization target;
- utilisation = load / (ready replicas x capacity per pod at 100% of request);
- new pods serve traffic after `startup_seconds`; until then they are not
counted (the real controller treats not-yet-ready pods more conservatively).
"""
from __future__ import annotations
import math
from collections import deque
from dataclasses import dataclass, field
import yaml
SYNC_SECONDS = 15
TOLERANCE = 0.1
DEFAULT_UP = {"stabilizationWindowSeconds": 0, "selectPolicy": "Max",
"policies": [{"type": "Percent", "value": 100, "periodSeconds": 15},
{"type": "Pods", "value": 4, "periodSeconds": 15}]}
DEFAULT_DOWN = {"stabilizationWindowSeconds": 300, "selectPolicy": "Max",
"policies": [{"type": "Percent", "value": 100, "periodSeconds": 15}]}
@dataclass
class Spec:
min_replicas: int
max_replicas: int
target_utilization: int
scale_up: dict = field(default_factory=lambda: dict(DEFAULT_UP))
scale_down: dict = field(default_factory=lambda: dict(DEFAULT_DOWN))
@classmethod
def from_manifest(cls, text: str) -> "Spec":
doc = yaml.safe_load(text)
if doc.get("apiVersion") != "autoscaling/v2" or doc.get("kind") != "HorizontalPodAutoscaler":
raise ValueError("expected an autoscaling/v2 HorizontalPodAutoscaler")
spec = doc["spec"]
metrics = spec.get("metrics", [])
if len(metrics) != 1 or metrics[0].get("type") != "Resource" \
or metrics[0]["resource"]["target"].get("type") != "Utilization":
raise ValueError("the simulator supports exactly one Resource metric with a Utilization target")
behavior = spec.get("behavior") or {}
up = {**DEFAULT_UP, **(behavior.get("scaleUp") or {})}
down = {**DEFAULT_DOWN, **(behavior.get("scaleDown") or {})}
return cls(spec.get("minReplicas", 1), spec["maxReplicas"],
metrics[0]["resource"]["target"]["averageUtilization"], up, down)
def raw_recommendation(current: int, utilization: float, target: int) -> int:
ratio = utilization / target
if abs(ratio - 1.0) <= TOLERANCE:
return current
return math.ceil(current * ratio)
def _period_change(events: deque, now: int, period: int, direction: int) -> int:
return sum(n for t, n in events if now - t < period and (n > 0) == (direction > 0))
def scale_up_limit(current: int, rules: dict, events: deque, now: int) -> int:
limits = []
for p in rules["policies"]:
start = current - _period_change(events, now, p["periodSeconds"], +1)
if p["type"] == "Pods":
limits.append(start + p["value"])
else:
limits.append(math.ceil(start * (1 + p["value"] / 100)))
if rules.get("selectPolicy") == "Disabled":
return current
return min(limits) if rules.get("selectPolicy") == "Min" else max(limits)
def scale_down_limit(current: int, rules: dict, events: deque, now: int) -> int:
limits = []
for p in rules["policies"]:
start = current - _period_change(events, now, p["periodSeconds"], -1) # removed pods are negative
if p["type"] == "Pods":
limits.append(start - p["value"])
else:
limits.append(math.floor(start * (1 - p["value"] / 100)))
if rules.get("selectPolicy") == "Disabled":
return current
# Max = allow the biggest change = the lowest floor
return max(limits) if rules.get("selectPolicy") == "Min" else min(limits)
def simulate(spec: Spec, load: list[float], capacity_per_pod: float,
start_replicas: int | None = None, startup_seconds: int = 0) -> list[dict]:
"""Replay `load` (one value per sync period) and return one row per sync."""
replicas = start_replicas or spec.min_replicas
pending: deque = deque() # (ready_at, count)
recs: deque = deque() # (time, recommendation)
events: deque = deque() # (time, +added / -removed)
rows = []
for i, demand in enumerate(load):
now = i * SYNC_SECONDS
while pending and pending[0][0] <= now:
pending.popleft()
not_ready = sum(c for _, c in pending)
ready = max(1, replicas - not_ready)
util = 100.0 * demand / (ready * capacity_per_pod)
rec = raw_recommendation(ready, util, spec.target_utilization) + not_ready
recs.append((now, rec))
window = max(spec.scale_up["stabilizationWindowSeconds"], spec.scale_down["stabilizationWindowSeconds"])
while recs and now - recs[0][0] > window:
recs.popleft()
up_window = [r for t, r in recs if now - t <= spec.scale_up["stabilizationWindowSeconds"]]
down_window = [r for t, r in recs if now - t <= spec.scale_down["stabilizationWindowSeconds"]]
up_rec, down_rec = min(up_window), max(down_window)
desired = up_rec if replicas < up_rec else down_rec if replicas > down_rec else replicas
if desired > replicas:
desired = min(desired, scale_up_limit(replicas, spec.scale_up, events, now))
elif desired < replicas:
desired = max(desired, scale_down_limit(replicas, spec.scale_down, events, now))
desired = max(spec.min_replicas, min(spec.max_replicas, desired))
if desired != replicas:
events.append((now, desired - replicas))
if desired > replicas and startup_seconds:
pending.append((now + startup_seconds, desired - replicas))
if desired < replicas:
# Remove not-ready pods first, as a ReplicaSet scale-down prefers them
drop = replicas - desired
while drop and pending:
t, c = pending.pop()
take = min(c, drop)
if c - take:
pending.append((t, c - take))
drop -= take
rows.append({"t": now, "load": demand, "ready": ready, "utilization": round(util, 1),
"recommendation": rec, "replicas": desired})
replicas = desired
while events and now - events[0][0] > 1800:
events.popleft()
return rows
def summarise(rows: list[dict], target: int) -> dict:
over = sum(1 for r in rows if r["utilization"] > 100)
return {
"peak_replicas": max(r["replicas"] for r in rows),
"replica_minutes": round(sum(r["replicas"] for r in rows) * SYNC_SECONDS / 60, 1),
"syncs_over_100pct": over,
"syncs_over_target": sum(1 for r in rows if r["utilization"] > target),
"changes": sum(1 for a, b in zip(rows, rows[1:]) if a["replicas"] != b["replicas"]),
}On two synthetic traces the trade-off shows clearly. With a two-minute dip between spikes and 45-second pod start-up, the default already keeps its pods through the dip (its 300-second window is longer), so the tuned version costs about 50 per cent more replica-minutes for no benefit. With a seven-minute dip and two-minute start-up, the default scales down and then spends 17 syncs saturated on the second spike; the tuned version spends 8. Run your own traces through it before choosing.
| Trace | Behavior | Replica-minutes | Syncs saturated | Syncs over target |
|---|---|---|---|---|
| 2-min dip, 45 s start-up | Default | 792 | 4 | 8 |
| 2-min dip, 45 s start-up | hpa.yaml | 1,207 | 4 | 8 |
| 7-min dip, 120 s start-up | Default | 963 | 17 | 26 |
| 7-min dip, 120 s start-up | hpa.yaml | 1,450 | 8 | 16 |
A short or zero scale-up window with generous policies limits saturation; the cost is a few extra pods for a few minutes. Lengthen the scale-down window only when traces show load returning soon after it falls, or when pods take long to start.
How HPA fits with the other autoscalers

HPA adds pods; a node autoscaler (Cluster Autoscaler, or Karpenter on AWS) adds nodes when those pods cannot be scheduled. They need each other, and both depend on accurate resource requests. KEDA builds on HPA rather than replacing it: it adds scaling to and from zero and event sources such as queues and streams. VPA sets requests, which is the input HPA utilisation targets depend on.
Set minReplicas to at least one pod per zone for services with availability targets, and pair the HPA with a PodDisruptionBudget so voluntary disruptions do not take replicas below what the service needs. Set maxReplicas to what downstream dependencies (databases, licences, third-party rate limits) can take, and alert when the HPA sits at its maximum.
hpa.yaml was validated against the Kubernetes 1.29 OpenAPI schema with kubernetes-validate==1.29.1 in strict mode. hpa_sim.py was tested with Python 3.13, pyyaml==6.0.1 and pytest==8.0.2: fourteen tests cover tolerance, default and custom rate limits, selectPolicy Min, Max and Disabled, the 300-second default window and a spike with start-up delay. The table comes from the sample’s example.py on synthetic traces; the simulator simplifies not-ready pod handling and has no metric lag, so it shows the shape of the behaviour rather than exact replica counts. Nothing was run on a cluster.
Key takeaways
- The HPA recalculates every 15 seconds: ratio with a 10 per cent tolerance, stabilisation, rate limits, then min and max.
- Utilisation is measured against requests; set them accurately or the HPA scales on noise.
- Scale on CPU where CPU is the bottleneck; otherwise use requests, in-flight work or queue depth through custom or external metrics.
- The default behavior scales up quickly and waits five minutes to scale down; change it only when your traces show why.
- HPA needs a node autoscaler underneath it, and must not share a CPU or memory signal with VPA in update mode.
What did tuning behavior change for your services? Share your settings and traffic shape in the comments.
The complete sample for this article is on GitHub: nithinmohantk/blog-samples-working-repo/hpa-behavior-sim. Clone it with git clone https://github.com/nithinmohantk/blog-samples-working-repo.git and open the hpa-behavior-sim folder. Its README says how to run it and what has been verified.
Glossary
- HPA
- HorizontalPodAutoscaler: changes a workload’s replica count based on metrics.
- Sync period
- How often the HPA controller evaluates each HPA; 15 seconds by default.
- Tolerance
- The band around the target (10 per cent by default) within which the HPA does not scale.
- Stabilisation window
- The period of past recommendations the HPA considers before scaling, to avoid flapping.
- Scaling policy
- A limit on how many pods (or what percentage) may be added or removed per period.
- VPA
- Vertical Pod Autoscaler: recommends or applies CPU and memory requests.
- KEDA
- Kubernetes Event-driven Autoscaling: scales on event sources and manages an HPA.
References & Further Reading
- → Horizontal Pod Autoscaling— Algorithm details, tolerance, readiness, behavior
- → Sample code for this article— HPA behavior simulator and example manifest
Discover more from C4: Container, Code, Cloud & Context
Subscribe to get the latest posts sent to your email.