Kubernetes / Intermediate: rollout, policy, and evidence / Lesson 9
Resources and scheduling feasibility
Connect requests, limits, node feasibility, and placement policy without confusing declared capacity with observed utilization.
On this page
Lesson goals
- Explain the model, assumptions, and boundaries of resources and scheduling feasibility.
- Interpret both examples and distinguish declared intent from established evidence.
- Diagnose the listed failures and justify the worked recovery decisions without executing cluster operations.
Requests and limits answer different questions
A request contributes to scheduling and resource allocation decisions; a limit constrains runtime usage according to the resource and implementation. CPU throttling differs from memory exhaustion, and neither is diagnosed by a single utilization percentage. The first fragment declares container CPU and memory in explicit units. 500m CPU is half a CPU unit; 256Mi is a binary memory quantity. Defaults and admission can alter omitted fields, so inspect the admitted specification. ResourceQuota and LimitRange apply namespace policy but do not manufacture physical capacity or make a namespace a complete security boundary.
resources:
requests:
cpu: 500m
memory: 256Mi
limits:
cpu: "1"
memory: 512Mi
The fragment belongs inside a container specification. Values illustrate units rather than measured recommendations; the request and limit do not prove available node capacity.
Filter feasibility before ranking nodes
Scheduling first requires a node that satisfies hard constraints, then chooses among feasible options according to scheduling policy. Requests, taints, required affinity, volume topology, ports, and node conditions can all matter. A Pod can be Pending while average cluster CPU is low because no single eligible node fits its request or storage zone. The second simulation checks CPU fit only for synthetic nodes and states that limitation. Real scheduler effective requests also consider init behavior, overhead, and additional resources; do not use the simplified sum as a production placement calculator.
nodes = {"a": 700, "b": 400}
request_millicpu = 500
print([name for name, free in nodes.items() if free >= request_millicpu])
print("CPU-only synthetic feasibility")
['a']
CPU-only synthetic feasibility
Only a has enough synthetic unallocated requested CPU. Memory, taints, affinity, volumes, overhead, and actual utilization are intentionally not modeled.
Understand affinity and tolerations
Node selectors and required node affinity restrict placement; preferred affinity expresses a preference rather than a guarantee. Taints repel Pods unless matching tolerations permit consideration. A toleration does not force placement on the tainted node and does not override resource shortage or all other constraints. Pod affinity and anti-affinity can improve locality or separation but can also make a rollout impossible. Use topology spread rules when their documented semantics fit the failure-domain requirement. Review labels used for security-sensitive placement because a label controlled by an untrusted node may not establish the promised isolation.
Read scheduling and runtime failures separately
FailedScheduling evidence points toward feasibility, while a running container later throttled, evicted, or terminated requires runtime investigation. Exit code 137 can reflect a kill and is not by itself proof of an out-of-memory event. Compare termination reason, node pressure, workload usage, limits, and relevant events. Raising every limit can increase contention and does not fix a memory leak. Reducing requests to make a Pod fit can overcommit the node without reducing real demand. Use representative approved measurements and review the consequences for neighbors and rollout headroom.
Plan constraints as a single contract
Capacity must cover normal demand, failures, rollout surge, and scaling delays within cost and dependency limits. Before adding stricter placement, verify that enough eligible capacity exists in the required zones. Before relaxing it, understand whether the constraint protects data, availability, or trust. Document requests, runtime limits, scheduling rules, storage, and ownership together. No scheduler, node, stress tool, or load generator is run here. YAML and arithmetic checks provide local teaching evidence only; validating real capacity requires an authorized isolated experiment and platform-specific observation, not a universal tuning value copied from a lesson.
Common errors and fixes
Low average CPU is assumed to guarantee placement
- Why it happens
- Per-node requests and hard constraints were ignored.
- How to fix it
- Inspect exact FailedScheduling constraints and eligible nodes.
- Prevent it next time
- Track feasible capacity by workload and failure domain.
A toleration is expected to force placement
- Why it happens
- Permission to consider a tainted node was mistaken for selection.
- How to fix it
- Review affinity, selectors, requests, and remaining filters.
- Prevent it next time
- Document the separate purpose of each rule.
Exit 137 is labeled OOM without evidence
- Why it happens
- An exit code replaced termination and node context.
- How to fix it
- Correlate reason, events, pressure, and usage.
- Prevent it next time
- Preserve first-failure runtime evidence.
Check your understanding
1. Two eligible nodes each have 400m unallocated requested CPU. Can one 500m Pod fit by adding the capacities?
Show worked solution
No. A Pod is placed on one node, so neither has enough CPU in this simplified example. Real feasibility also includes other constraints. Adding aggregate capacity does not create a node large enough for an indivisible request.
2. Why is lowering a request without measuring demand risky?
Show worked solution
It can make scheduling succeed while leaving actual demand unchanged, increasing contention or later failure. Identify whether the request was excessive or the workload genuinely needs more capacity. Preserve reliability and neighbor protection rather than optimizing only the Pending count.
Key takeaways
- A request contributes to scheduling and resource allocation decisions; a limit constrains runtime usage according to the resource and implementation.
- Node selectors and required node affinity restrict placement; preferred affinity expresses a preference rather than a guarantee.
- Capacity must cover normal demand, failures, rollout surge, and scaling delays within cost and dependency limits.