After PodGroup Pods have passed the scheduling cycle, the binding cycle
asynchronously invokes the Reserve and PreBind phases of the
DynamicResources plugin. When a shared ResourceClaim is first allocated,
other Pods in the group may have yet to reach the Reserve phase. If an
in-flight allocation is recorded in Reserve for a claim that is already
allocated, later removing that in-flight allocation results in
`Restore`-ing the claim in the assume cache to an old version that may
be unallocated. If the restored claim is unallocated, then the scheduler
continuously falls into a loop:
- Claim in AssumeCache is unallocated
- Pending allocation is recorded
- API GET in PreBind yields the allocated claim
- "claim got allocated elsewhere" error
This change cuts off that cycle by verifying that claims are actually
unallocated in the Reserve phase before recording an in-flight
allocation.
Unschedulable -> Wait
UnschedulableAndUnresolvable -> Unschedulable
In the future, we can return UnschedulableAndUnresolvable if there's not
enough pods in the podgroup in total.
When a Pod fails to asynchronously bind, a race occurs between the patch
to add the `PodScheduled=False` condition and a subsequent binding
attempt. If binding succeeds and adds `PodScheduled=True` before the
`PodScheduled=False` condition is set, then the last update persists
where the Pod is scheduled with a `spec.nodeName` but still has the
`PodScheduled=False` condition.
This change adds the `metadata.resourceVersion` of the Pod to patches so
these conflicts can be detected and handled appropriately.
In the DRA scheduler plugin, `SignalClaimPendingAllocation` and
`MaybeRemoveClaimPendingAllocation` coordinate in-flight allocations for
claims shared between Pods when scheduling a PodGroup. `Signal` is
called in Reserve to store an in-flight allocation and mark a claim as
being shared. `MaybeRemove` is called in Unreserve to release a Pod's
share of the claim and remove the in-flight allocation if no more Pods
are sharing the claim. `MaybeRemove` is also called at the end of
PreBind so successful binding cycles remove the in-flight allocation in
favor of the allocation stored in the API.
When a Pod transiently fails in PreBind, `MaybeRemove` is invoked twice:
first in PreBind, and again in Unreserve. Since shares are tracked with
a counter, these double-counted releases can cause the in-flight claim
to be removed from the cache prematurely and poison the AssumeCache with
a claim that has both an updated resourceVersion and no allocation,
halting progress.
This change refactors the `MaybeRemove` call in PreBind to only fire
when PreBind has finished successfully. When Pods sharing a claim are
binding concurrently, each one will either:
- Update the claim's allocation successfully and force-remove the
in-flight allocation, or
- Fail to bind and wait until Unreserve to call `MaybeRemove`
The DRAListTypeAttributes feature gate was enabled but not passed
through AllocatorFeatures(), so the scheduler always selected the
incubating allocator which doesn't support list-type attribute
intersection matching (KEP-5491). The experimental allocator has
the intersection logic but was never selected.
