upstream/kubernetes
1
0
Fork 0
mirror of https://github.com/kubernetes/kubernetes.git synced 2026-04-15 14:26:33 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
kubernetes/pkg/controller/nodelifecycle/node_lifecycle_controller_test.go

3904 lines
130 KiB
Go
Raw Permalink Normal View History

Rename minioncontroller to nodecontroller
2014-12-19 04:27:01 -05:00
/*
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
Copyright 2017 The Kubernetes Authors.
Rename minioncontroller to nodecontroller
2014-12-19 04:27:01 -05:00
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
package nodelifecycle
Rename minioncontroller to nodecontroller
2014-12-19 04:27:01 -05:00
import (
generated: run refactor
2020-02-07 21:16:47 -05:00
"context"
adding fakeGetPodsAssignedToNode
2019-09-27 11:28:58 -04:00
"fmt"
unit tests: Skip flaky tests on Windows Some of the unit tests are currently flaky on Windows. This commit skips them until they are resolved.
2023-02-21 07:48:02 -05:00
goruntime "runtime"
Move orphaned Pod deletion logic to PodGC
2016-09-12 10:47:17 -04:00
"strings"
Rename minioncontroller to nodecontroller
2014-12-19 04:27:01 -05:00
"testing"
"time"
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
"github.com/google/go-cmp/cmp"
Swtich nodelifecyclecontroller to coordination/v1
2019-10-16 03:11:03 -04:00
coordv1 "k8s.io/api/coordination/v1"
Node lifecycle controller should use a label for excluding nodes The current mechanism for excluding "master" nodes based on node names is fragile and should be fixed by using a label exclusion similar to service load balancers. The legacy code path is preserved behind a defaulted-on gate and will be removed in the future.
2019-07-16 22:24:21 -04:00
v1 "k8s.io/api/core/v1"
pkg/api: move Semantic equality to k8s.io/apimachinery/pkg/api/equality
2017-01-25 08:39:54 -05:00
apiequality "k8s.io/apimachinery/pkg/api/equality"
pkg/api/resource: move to apimachinery
2017-01-25 08:13:07 -05:00
"k8s.io/apimachinery/pkg/api/resource"
start the apimachinery repo
2017-01-11 09:09:48 -05:00
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
adding fakeGetPodsAssignedToNode
2019-09-27 11:28:58 -04:00
"k8s.io/apimachinery/pkg/fields"
"k8s.io/apimachinery/pkg/labels"
adding test for retrying MarkPodsNotReady
2019-10-30 09:46:30 -04:00
"k8s.io/apimachinery/pkg/runtime"
Scripted migration from clientset_generated to client-go.
2017-06-23 16:56:37 -04:00
"k8s.io/client-go/informers"
update deployment, daemonset, replicaset, statefulset to apps/v1
2018-08-15 22:06:39 -04:00
appsinformers "k8s.io/client-go/informers/apps/v1"
Swtich nodelifecyclecontroller to coordination/v1
2019-10-16 03:11:03 -04:00
coordinformers "k8s.io/client-go/informers/coordination/v1"
Scripted migration from clientset_generated to client-go.
2017-06-23 16:56:37 -04:00
coreinformers "k8s.io/client-go/informers/core/v1"
clientset "k8s.io/client-go/kubernetes"
"k8s.io/client-go/kubernetes/fake"
move testing/core to client-go
2017-01-25 15:07:10 -05:00
testcore "k8s.io/client-go/testing"
Move pkg/kubelet/apis to k8s.io/kubelet/pkg/apis
2021-02-09 13:47:42 -05:00
kubeletapis "k8s.io/kubelet/pkg/apis"
switch node controller to shared informers
2016-09-23 12:01:58 -04:00
"k8s.io/kubernetes/pkg/controller"
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
"k8s.io/kubernetes/pkg/controller/nodelifecycle/scheduler"
Moved node/testutil to upper dir.
2017-08-04 19:34:28 -04:00
"k8s.io/kubernetes/pkg/controller/testutil"
move pkg/util/node to component-helpers/node/util (#105347) Signed-off-by: Neha Lohia <nehapithadiya444@gmail.com>
2021-11-12 10:52:27 -05:00
controllerutil "k8s.io/kubernetes/pkg/controller/util/node"
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
"k8s.io/kubernetes/pkg/util/node"
Refactoring taints to reduce sprawl
2017-07-06 09:13:13 -04:00
taintutils "k8s.io/kubernetes/pkg/util/taints"
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
"k8s.io/kubernetes/test/utils/ktesting"
chore: depr. pointer pkg replacement for pkg/controller
2025-07-07 07:22:36 -04:00
"k8s.io/utils/ptr"
Rename minioncontroller to nodecontroller
2014-12-19 04:27:01 -05:00
)
Move NodeController constants to flags
2015-03-31 07:17:12 -04:00
const (
Update Node Monitor Grace Period default duration to 50s Update description Improve flag comment Update Test case value to be 50s by default Update Description Run make update Minor description fix
2024-07-23 06:26:30 -04:00
testNodeMonitorGracePeriod = 50 * time.Second
Change default Kubelet NodeStatusUpdateFrequency from 2s to 10s.
2015-04-07 15:36:09 -04:00
testNodeStartupGracePeriod = 60 * time.Second
Move NodeController constants to flags
2015-03-31 07:17:12 -04:00
testNodeMonitorPeriod = 5 * time.Second
NodeLifecycleController: Remove race condition Patch request does not support RV by default, we need to include them explicitly and patching lists actually overwrites whole field. It means that there is a race condition, in which we can overwrite changes to taints that happened between GET and PATCH requests.
2022-10-18 08:01:15 -04:00
testRateLimiterQPS = float32(100000)
Expose flags for new NodeEviction logic in NodeController
2016-08-05 08:50:19 -04:00
testLargeClusterThreshold = 20
Fix const nameing in node/metrics
2017-06-05 04:36:09 -04:00
testUnhealthyThreshold = float32(0.55)
Move NodeController constants to flags
2015-03-31 07:17:12 -04:00
)
Replace hand-written informers with generated ones Replace existing uses of hand-written informers with generated ones. Follow-up commits will switch the use of one-off informers to shared informers.
2017-02-06 13:35:50 -05:00
func alwaysReady() bool { return true }
using pod pointers in node lifecycle controller
2019-10-11 09:41:34 -04:00
func fakeGetPodsAssignedToNode(c *fake.Clientset) func(string) ([]*v1.Pod, error) {
return func(nodeName string) ([]*v1.Pod, error) {
adding fakeGetPodsAssignedToNode
2019-09-27 11:28:58 -04:00
selector := fields.SelectorFromSet(fields.Set{"spec.nodeName": nodeName})
generated: run refactor
2020-02-07 21:16:47 -05:00
pods, err := c.CoreV1().Pods(v1.NamespaceAll).List(context.TODO(), metav1.ListOptions{
adding fakeGetPodsAssignedToNode
2019-09-27 11:28:58 -04:00
FieldSelector: selector.String(),
LabelSelector: labels.Everything().String(),
})
if err != nil {
return nil, fmt.Errorf("failed to get Pods assigned to node %v", nodeName)
}
using pod pointers in node lifecycle controller
2019-10-11 09:41:34 -04:00
rPods := make([]*v1.Pod, len(pods.Items))
for i := range pods.Items {
rPods[i] = &pods.Items[i]
}
return rPods, nil
adding fakeGetPodsAssignedToNode
2019-09-27 11:28:58 -04:00
}
}
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
type nodeLifecycleController struct {
(ALPHA GCP FEATURE) Add IPAM controller IPAM controller unifies handling of node pod CIDR range allocation. It is intended to supersede the logic that is currently in range_allocator and cloud_cidr_allocator. Note: for this change, the other allocators still exist and are the default. It supports two modes: * CIDR range allocations done within the cluster that are then propagated out to the cloud provider. * Cloud provider managed IPAM that is then reflected into the cluster.
2017-08-08 19:25:20 -04:00
*Controller
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
leaseInformer coordinformers.LeaseInformer
Replace hand-written informers with generated ones Replace existing uses of hand-written informers with generated ones. Follow-up commits will switch the use of one-off informers to shared informers.
2017-02-06 13:35:50 -05:00
nodeInformer coreinformers.NodeInformer
update deployment, daemonset, replicaset, statefulset to apps/v1
2018-08-15 22:06:39 -04:00
daemonSetInformer appsinformers.DaemonSetInformer
node-lifecycle-controller: improve processPod test-coverage
2024-10-09 06:26:38 -04:00
podInformer coreinformers.PodInformer
Replace hand-written informers with generated ones Replace existing uses of hand-written informers with generated ones. Follow-up commits will switch the use of one-off informers to shared informers.
2017-02-06 13:35:50 -05:00
}
Swtich nodelifecyclecontroller to coordination/v1
2019-10-16 03:11:03 -04:00
func createNodeLease(nodeName string, renewTime metav1.MicroTime) *coordv1.Lease {
return &coordv1.Lease{
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
ObjectMeta: metav1.ObjectMeta{
Name: nodeName,
Namespace: v1.NamespaceNodeLease,
},
Swtich nodelifecyclecontroller to coordination/v1
2019-10-16 03:11:03 -04:00
Spec: coordv1.LeaseSpec{
chore: depr. pointer pkg replacement for pkg/controller
2025-07-07 07:22:36 -04:00
HolderIdentity: ptr.To(nodeName),
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
RenewTime: &renewTime,
},
}
}
Swtich nodelifecyclecontroller to coordination/v1
2019-10-16 03:11:03 -04:00
func (nc *nodeLifecycleController) syncLeaseStore(lease *coordv1.Lease) error {
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
if lease == nil {
return nil
}
newElems := make([]interface{}, 0, 1)
newElems = append(newElems, lease)
return nc.leaseInformer.Informer().GetStore().Replace(newElems, "newRV")
}
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
func (nc *nodeLifecycleController) syncNodeStore(fakeNodeHandler *testutil.FakeNodeHandler) error {
some manual fixes
2019-12-17 03:01:53 -05:00
nodes, err := fakeNodeHandler.List(context.TODO(), metav1.ListOptions{})
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
if err != nil {
return err
}
newElems := make([]interface{}, 0, len(nodes.Items))
for i := range nodes.Items {
newElems = append(newElems, &nodes.Items[i])
}
return nc.nodeInformer.Informer().GetStore().Replace(newElems, "newRV")
}
node-lifecycle-controller: improve processPod test-coverage
2024-10-09 06:26:38 -04:00
func (nc *nodeLifecycleController) syncPodStore(pod *v1.Pod) error {
if pod == nil {
return nil
}
newElems := make([]interface{}, 0, 1)
newElems = append(newElems, pod)
return nc.podInformer.Informer().GetStore().Replace(newElems, "newRV")
}
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
func newNodeLifecycleControllerFromClient(
Wire contexts to Core controllers
2021-04-22 14:27:59 -04:00
ctx context.Context,
switch node controller to shared informers
2016-09-23 12:01:58 -04:00
kubeClient clientset.Interface,
evictionLimiterQPS float32,
secondaryEvictionLimiterQPS float32,
largeClusterThreshold int32,
unhealthyZoneThreshold float32,
nodeMonitorGracePeriod time.Duration,
nodeStartupGracePeriod time.Duration,
nodeMonitorPeriod time.Duration,
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
) (*nodeLifecycleController, error) {
switch node controller to shared informers
2016-09-23 12:01:58 -04:00
react to informer updates
2017-02-08 16:18:21 -05:00
factory := informers.NewSharedInformerFactory(kubeClient, controller.NoResyncPeriodFunc())
switch node controller to shared informers
2016-09-23 12:01:58 -04:00
Swtich nodelifecyclecontroller to coordination/v1
2019-10-16 03:11:03 -04:00
leaseInformer := factory.Coordination().V1().Leases()
Replace hand-written informers with generated ones Replace existing uses of hand-written informers with generated ones. Follow-up commits will switch the use of one-off informers to shared informers.
2017-02-06 13:35:50 -05:00
nodeInformer := factory.Core().V1().Nodes()
update deployment, daemonset, replicaset, statefulset to apps/v1
2018-08-15 22:06:39 -04:00
daemonSetInformer := factory.Apps().V1().DaemonSets()
node-lifecycle-controller: improve processPod test-coverage
2024-10-09 06:26:38 -04:00
podInformer := factory.Core().V1().Pods()
Replace hand-written informers with generated ones Replace existing uses of hand-written informers with generated ones. Follow-up commits will switch the use of one-off informers to shared informers.
2017-02-06 13:35:50 -05:00
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
nc, err := NewNodeLifecycleController(
Wire contexts to Core controllers
2021-04-22 14:27:59 -04:00
ctx,
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
leaseInformer,
node-lifecycle-controller: improve processPod test-coverage
2024-10-09 06:26:38 -04:00
podInformer,
Replace hand-written informers with generated ones Replace existing uses of hand-written informers with generated ones. Follow-up commits will switch the use of one-off informers to shared informers.
2017-02-06 13:35:50 -05:00
nodeInformer,
daemonSetInformer,
kubeClient,
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
nodeMonitorPeriod,
nodeStartupGracePeriod,
nodeMonitorGracePeriod,
Replace hand-written informers with generated ones Replace existing uses of hand-written informers with generated ones. Follow-up commits will switch the use of one-off informers to shared informers.
2017-02-06 13:35:50 -05:00
evictionLimiterQPS,
secondaryEvictionLimiterQPS,
largeClusterThreshold,
unhealthyZoneThreshold,
)
switch node controller to shared informers
2016-09-23 12:01:58 -04:00
if err != nil {
return nil, err
}
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
nc.leaseInformerSynced = alwaysReady
Replace hand-written informers with generated ones Replace existing uses of hand-written informers with generated ones. Follow-up commits will switch the use of one-off informers to shared informers.
2017-02-06 13:35:50 -05:00
nc.podInformerSynced = alwaysReady
nc.nodeInformerSynced = alwaysReady
nc.daemonSetInformerSynced = alwaysReady
node-lifecycle-controller: improve processPod test-coverage
2024-10-09 06:26:38 -04:00
return &nodeLifecycleController{nc, leaseInformer, nodeInformer, daemonSetInformer, podInformer}, nil
NodeController listing nodes from cache instead of cache in apiserver
2016-12-19 05:15:39 -05:00
}
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
func TestMonitorNodeHealth(t *testing.T) {
refactor: generated
2016-12-03 13:57:26 -05:00
fakeNow := metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC)
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
timeToPass := 60 * time.Minute
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
healthyNodeNewStatus := v1.NodeStatus{
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.NewTime(fakeNow.Add(timeToPass)),
LastTransitionTime: fakeNow,
},
},
}
unhealthyNodeNewStatus := v1.NodeStatus{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Conditions: []v1.NodeCondition{
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Type: v1.NodeReady,
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
Status: v1.ConditionUnknown,
// Node status was updated by nodecontroller timeToPass ago
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
},
},
}
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
tests := map[string]struct {
nodeList []*v1.Node
updatedNodeStatuses []v1.NodeStatus
expectedInitialStates map[string]ZoneState
expectedFollowingStates map[string]ZoneState
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
}{
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
"No Disruption: Node created recently without failure domain labels (happens only at cluster startup)": {
nodeList: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
Split NodeController rate limiters between zones
2016-07-12 08:29:46 -04:00
},
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
},
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
},
updatedNodeStatuses: []v1.NodeStatus{
healthyNodeNewStatus,
},
expectedInitialStates: map[string]ZoneState{
"": stateNormal,
},
expectedFollowingStates: map[string]ZoneState{
"": stateNormal,
},
},
"No Disruption: Initially both zones down, one comes back": {
nodeList: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
},
},
},
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
{
ObjectMeta: metav1.ObjectMeta{
Name: "node1",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone2",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone2",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
Added pod evictors for new zone.
2017-06-23 03:38:05 -04:00
},
},
},
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
updatedNodeStatuses: []v1.NodeStatus{
unhealthyNodeNewStatus,
healthyNodeNewStatus,
},
expectedInitialStates: map[string]ZoneState{
testutil.CreateZoneID("region1", "zone1"): stateFullDisruption,
testutil.CreateZoneID("region1", "zone2"): stateFullDisruption,
},
expectedFollowingStates: map[string]ZoneState{
testutil.CreateZoneID("region1", "zone1"): stateFullDisruption,
testutil.CreateZoneID("region1", "zone2"): stateNormal,
},
Added pod evictors for new zone.
2017-06-23 03:38:05 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
"Partial Disruption: Nodes created recently without status conditions (happens only at cluster startup)": {
nodeList: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
},
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
{
ObjectMeta: metav1.ObjectMeta{
Name: "node1",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
},
{
ObjectMeta: metav1.ObjectMeta{
Name: "node2",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
},
},
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
{
ObjectMeta: metav1.ObjectMeta{
Name: "node3",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
},
Add timestamp cache to NodeController
2015-03-31 11:15:39 -04:00
},
},
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
updatedNodeStatuses: []v1.NodeStatus{
unhealthyNodeNewStatus,
unhealthyNodeNewStatus,
unhealthyNodeNewStatus,
healthyNodeNewStatus,
},
expectedInitialStates: map[string]ZoneState{
// we've not received any status for the nodes yet
// so the controller assumes the zones is fully disrupted
testutil.CreateZoneID("region1", "zone1"): stateFullDisruption,
},
expectedFollowingStates: map[string]ZoneState{
testutil.CreateZoneID("region1", "zone1"): statePartialDisruption,
},
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
"Partial Disruption: one Node failed leading to the number of healthy Nodes to exceed the configured threshold": {
nodeList: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
When a node becomes unreachable, do not evict DaemonSet-managed pods. Part of the graduation requirement for DaemonSet spelled out in #15310.
2016-01-08 16:38:02 -05:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
When a node becomes unreachable, do not evict DaemonSet-managed pods. Part of the graduation requirement for DaemonSet spelled out in #15310.
2016-01-08 16:38:02 -05:00
},
},
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
{
ObjectMeta: metav1.ObjectMeta{
Name: "node1",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
},
},
},
When a node becomes unreachable, do not evict DaemonSet-managed pods. Part of the graduation requirement for DaemonSet spelled out in #15310.
2016-01-08 16:38:02 -05:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
{
ObjectMeta: metav1.ObjectMeta{
Name: "node2",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
When a node becomes unreachable, do not evict DaemonSet-managed pods. Part of the graduation requirement for DaemonSet spelled out in #15310.
2016-01-08 16:38:02 -05:00
{
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
When a node becomes unreachable, do not evict DaemonSet-managed pods. Part of the graduation requirement for DaemonSet spelled out in #15310.
2016-01-08 16:38:02 -05:00
},
},
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
When a node becomes unreachable, do not evict DaemonSet-managed pods. Part of the graduation requirement for DaemonSet spelled out in #15310.
2016-01-08 16:38:02 -05:00
{
refactor: use metav1.ObjectMeta in other types
2017-01-16 22:38:19 -05:00
ObjectMeta: metav1.ObjectMeta{
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
Name: "node3",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
},
When a node becomes unreachable, do not evict DaemonSet-managed pods. Part of the graduation requirement for DaemonSet spelled out in #15310.
2016-01-08 16:38:02 -05:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
When a node becomes unreachable, do not evict DaemonSet-managed pods. Part of the graduation requirement for DaemonSet spelled out in #15310.
2016-01-08 16:38:02 -05:00
},
},
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
{
ObjectMeta: metav1.ObjectMeta{
Name: "node4",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
When a node becomes unreachable, do not evict DaemonSet-managed pods. Part of the graduation requirement for DaemonSet spelled out in #15310.
2016-01-08 16:38:02 -05:00
},
},
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
updatedNodeStatuses: []v1.NodeStatus{
unhealthyNodeNewStatus,
unhealthyNodeNewStatus,
unhealthyNodeNewStatus,
healthyNodeNewStatus,
healthyNodeNewStatus,
},
expectedInitialStates: map[string]ZoneState{
testutil.CreateZoneID("region1", "zone1"): stateNormal,
},
expectedFollowingStates: map[string]ZoneState{
testutil.CreateZoneID("region1", "zone1"): statePartialDisruption,
},
When a node becomes unreachable, do not evict DaemonSet-managed pods. Part of the graduation requirement for DaemonSet spelled out in #15310.
2016-01-08 16:38:02 -05:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
"Full Disruption: the zone has less than 2 Nodes down, the last healthy Node has failed": {
nodeList: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
},
},
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
{
ObjectMeta: metav1.ObjectMeta{
Name: "node1",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
},
},
},
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
{
ObjectMeta: metav1.ObjectMeta{
Name: "node2",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
Add timestamp cache to NodeController
2015-03-31 11:15:39 -04:00
},
},
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
updatedNodeStatuses: []v1.NodeStatus{
unhealthyNodeNewStatus,
unhealthyNodeNewStatus,
unhealthyNodeNewStatus,
},
expectedInitialStates: map[string]ZoneState{
// if a zone has a number of unhealthy nodes less or equal to 2
// the controller will consider it normal regardless on
// the ration of healthy vs unhealthy nodes
testutil.CreateZoneID("region1", "zone1"): stateNormal,
},
expectedFollowingStates: map[string]ZoneState{
testutil.CreateZoneID("region1", "zone1"): stateFullDisruption,
},
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
"Full Disruption: all the Nodes in one zone are down": {
nodeList: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
},
},
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
{
ObjectMeta: metav1.ObjectMeta{
Name: "node1",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone2",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone2",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
Split NodeController rate limiters between zones
2016-07-12 08:29:46 -04:00
},
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
},
},
updatedNodeStatuses: []v1.NodeStatus{
unhealthyNodeNewStatus,
healthyNodeNewStatus,
},
expectedInitialStates: map[string]ZoneState{
testutil.CreateZoneID("region1", "zone1"): stateFullDisruption,
testutil.CreateZoneID("region1", "zone2"): stateNormal,
},
expectedFollowingStates: map[string]ZoneState{
testutil.CreateZoneID("region1", "zone1"): stateFullDisruption,
testutil.CreateZoneID("region1", "zone2"): stateNormal,
},
},
"Full Disruption: all the Nodes in both the zones are down": {
nodeList: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
},
},
},
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
{
ObjectMeta: metav1.ObjectMeta{
Name: "node1",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region2",
v1.LabelTopologyZone: "zone2",
v1.LabelFailureDomainBetaRegion: "region2",
v1.LabelFailureDomainBetaZone: "zone2",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
Add timestamp cache to NodeController
2015-03-31 11:15:39 -04:00
},
},
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
updatedNodeStatuses: []v1.NodeStatus{
unhealthyNodeNewStatus,
unhealthyNodeNewStatus,
},
expectedInitialStates: map[string]ZoneState{
testutil.CreateZoneID("region1", "zone1"): stateFullDisruption,
testutil.CreateZoneID("region2", "zone2"): stateFullDisruption,
},
expectedFollowingStates: map[string]ZoneState{
testutil.CreateZoneID("region1", "zone1"): stateFullDisruption,
testutil.CreateZoneID("region2", "zone2"): stateFullDisruption,
},
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
"Full Disruption: Ready condition removed from the Node": {
nodeList: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
Delete all pods based on condition transition time.
2015-03-20 13:35:41 -04:00
},
},
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
},
updatedNodeStatuses: []v1.NodeStatus{
{
Conditions: []v1.NodeCondition{},
},
},
expectedInitialStates: map[string]ZoneState{
testutil.CreateZoneID("region1", "zone1"): stateNormal,
},
expectedFollowingStates: map[string]ZoneState{
testutil.CreateZoneID("region1", "zone1"): stateFullDisruption,
},
},
"Full Disruption: the only available Node has the node.kubernetes.io/exclude-disruption label": {
nodeList: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
},
},
},
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
{
ObjectMeta: metav1.ObjectMeta{
Name: "node-master",
CreationTimestamp: fakeNow,
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
labelNodeDisruptionExclusion: "",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
},
},
},
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
updatedNodeStatuses: []v1.NodeStatus{
unhealthyNodeNewStatus,
healthyNodeNewStatus,
},
expectedInitialStates: map[string]ZoneState{
testutil.CreateZoneID("region1", "zone1"): stateFullDisruption,
},
expectedFollowingStates: map[string]ZoneState{
testutil.CreateZoneID("region1", "zone1"): stateFullDisruption,
},
NodeController doesn't evict Pods if no Nodes are Ready
2016-05-16 05:20:23 -04:00
},
Change eviction logic in NodeController and make it Zone-aware
2016-07-13 10:57:22 -04:00
}
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
for testName, tt := range tests {
t.Run(testName, func(t *testing.T) {
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx := ktesting.Init(t)
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
fakeNodeHandler := &testutil.FakeNodeHandler{
Existing: tt.nodeList,
Clientset: fake.NewSimpleClientset(),
}
nodeController, _ := newNodeLifecycleControllerFromClient(
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx,
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
testNodeMonitorPeriod)
nodeController.recorder = testutil.NewFakeRecorder()
nodeController.enterPartialDisruptionFunc = func(nodeNum int) float32 {
return testRateLimiterQPS
}
nodeController.enterFullDisruptionFunc = func(nodeNum int) float32 {
return testRateLimiterQPS
NodeController sets NodeTaints instead of deleting Pods
2017-02-06 07:58:48 -05:00
}
Change eviction logic in NodeController and make it Zone-aware
2016-07-13 10:57:22 -04:00
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
syncAndDiffZoneState := func(wanted map[string]ZoneState) {
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
t.Errorf("unexpected error: %v", err)
}
if diff := cmp.Diff(wanted, nodeController.zoneStates); diff != "" {
t.Errorf("unexpected zone state (-want +got):\n%s", diff)
}
Change eviction logic in NodeController and make it Zone-aware
2016-07-13 10:57:22 -04:00
}
node-lifecycle-controller: improve monitorNodeHealth test-coverage (#116687) * node-lifecycle-controller: refactor monitorNodeHealth tests to improve test-coverage * address PR review comments * dedupe test logic
2023-04-12 10:02:33 -04:00
// initial zone state
nodeController.now = func() metav1.Time { return fakeNow }
syncAndDiffZoneState(tt.expectedInitialStates)
// following zone state
nodeController.now = func() metav1.Time { return metav1.Time{Time: fakeNow.Add(timeToPass)} }
for i := range tt.updatedNodeStatuses {
fakeNodeHandler.Existing[i].Status = tt.updatedNodeStatuses[i]
}
syncAndDiffZoneState(tt.expectedFollowingStates)
})
Change eviction logic in NodeController and make it Zone-aware
2016-07-13 10:57:22 -04:00
}
}
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
func TestPodStatusChange(t *testing.T) {
refactor: generated
2016-12-03 13:57:26 -05:00
fakeNow := metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC)
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
// Because of the logic that prevents NC from evicting anything when all Nodes are NotReady
// we need second healthy node in tests. Because of how the tests are written we need to update
// the status of this Node.
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
healthyNodeNewStatus := v1.NodeStatus{
Conditions: []v1.NodeCondition{
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Type: v1.NodeReady,
Status: v1.ConditionTrue,
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
// Node status has just been updated, and is NotReady for 10min.
refactor: generated
2016-12-03 13:57:26 -05:00
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 9, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
},
},
}
// Node created long time ago, node controller posted Unknown for a long period of time.
table := []struct {
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
fakeNodeHandler *testutil.FakeNodeHandler
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
timeToPass time.Duration
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
newNodeStatus v1.NodeStatus
secondNodeNewStatus v1.NodeStatus
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
expectedPodUpdate bool
expectedReason string
description string
}{
{
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
fakeNodeHandler: &testutil.FakeNodeHandler{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Existing: []*v1.Node{
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
{
refactor: use metav1.ObjectMeta in other types
2017-01-16 22:38:19 -05:00
ObjectMeta: metav1.ObjectMeta{
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
Name: "node0",
refactor: generated
2016-12-03 13:57:26 -05:00
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
Labels: map[string]string{
Use topology labels instead of old beta names (#96033) * Rename const for topology.../zone * Rename const for topology.../region * Rename const for failure-domain.../zone * Rename const for failure-domain.../region * Restore old names for compat
2020-11-05 23:26:50 -05:00
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
},
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
refactor: generated
2016-12-03 13:57:26 -05:00
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
},
},
},
},
{
refactor: use metav1.ObjectMeta in other types
2017-01-16 22:38:19 -05:00
ObjectMeta: metav1.ObjectMeta{
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
Name: "node1",
refactor: generated
2016-12-03 13:57:26 -05:00
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
Labels: map[string]string{
Use topology labels instead of old beta names (#96033) * Rename const for topology.../zone * Rename const for topology.../region * Rename const for failure-domain.../zone * Rename const for failure-domain.../region * Restore old names for compat
2020-11-05 23:26:50 -05:00
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
},
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Type: v1.NodeReady,
Status: v1.ConditionTrue,
refactor: generated
2016-12-03 13:57:26 -05:00
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
},
},
},
},
},
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
},
timeToPass: 60 * time.Minute,
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
newNodeStatus: v1.NodeStatus{
Conditions: []v1.NodeCondition{
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
// Node status was updated by nodecontroller 1hr ago
refactor: generated
2016-12-03 13:57:26 -05:00
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
},
},
},
secondNodeNewStatus: healthyNodeNewStatus,
expectedPodUpdate: true,
expectedReason: node.NodeUnreachablePodReason,
description: "Node created long time ago, node controller posted Unknown for a " +
"long period of time, the pod status must include reason for termination.",
},
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx := ktesting.Init(t)
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
for _, item := range table {
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
nodeController, _ := newNodeLifecycleControllerFromClient(
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx,
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
item.fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
testNodeMonitorPeriod,
Remove CLI flag enable-taint-manager Signed-off-by: kerthcet <kerthcet@gmail.com>
2022-09-14 05:04:08 -04:00
)
refactor: generated
2016-12-03 13:57:26 -05:00
nodeController.now = func() metav1.Time { return fakeNow }
Apply davidopps comments to TaintController PR
2017-02-13 05:48:34 -05:00
nodeController.recorder = testutil.NewFakeRecorder()
adding fakeGetPodsAssignedToNode
2019-09-27 11:28:58 -04:00
nodeController.getPodsAssignedToNode = fakeGetPodsAssignedToNode(item.fakeNodeHandler.Clientset)
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
if err := nodeController.syncNodeStore(item.fakeNodeHandler); err != nil {
NodeController listing nodes from cache instead of cache in apiserver
2016-12-19 05:15:39 -05:00
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
t.Errorf("unexpected error: %v", err)
}
if item.timeToPass > 0 {
refactor: generated
2016-12-03 13:57:26 -05:00
nodeController.now = func() metav1.Time { return metav1.Time{Time: fakeNow.Add(item.timeToPass)} }
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
item.fakeNodeHandler.Existing[0].Status = item.newNodeStatus
item.fakeNodeHandler.Existing[1].Status = item.secondNodeNewStatus
}
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
if err := nodeController.syncNodeStore(item.fakeNodeHandler); err != nil {
NodeController listing nodes from cache instead of cache in apiserver
2016-12-19 05:15:39 -05:00
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
t.Errorf("unexpected error: %v", err)
}
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
zones := testutil.GetZones(item.fakeNodeHandler)
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
for _, zone := range zones {
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
nodeController.zoneNoExecuteTainter[zone].Try(tCtx.Logger(), func(value scheduler.TimedValue) (bool, time.Duration) {
(ALPHA GCP FEATURE) Add IPAM controller IPAM controller unifies handling of node pod CIDR range allocation. It is intended to supersede the logic that is currently in range_allocator and cloud_cidr_allocator. Note: for this change, the other allocators still exist and are the default. It supports two modes: * CIDR range allocations done within the cluster that are then propagated out to the cloud provider. * Cloud provider managed IPAM that is then reflected into the cluster.
2017-08-08 19:25:20 -04:00
nodeUID, _ := value.UID.(string)
adding pods lister
2019-08-01 12:59:22 -04:00
pods, err := nodeController.getPodsAssignedToNode(value.Value)
adding pods to DeletePods parameters
2019-09-30 07:55:19 -04:00
if err != nil {
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
_, err = controllerutil.DeletePods(tCtx, item.fakeNodeHandler, pods, nodeController.recorder, value.Value, nodeUID, nodeController.daemonSetStore)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
return true, 0
})
}
podReasonUpdate := false
for _, action := range item.fakeNodeHandler.Actions() {
if action.GetVerb() == "update" && action.GetResource().Resource == "pods" {
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
updateReason := action.(testcore.UpdateActionImpl).GetObject().(*v1.Pod).Status.Reason
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
podReasonUpdate = true
if updateReason != item.expectedReason {
t.Errorf("expected pod status reason: %+v, got %+v for %+v", item.expectedReason, updateReason, item.description)
}
}
}
if podReasonUpdate != item.expectedPodUpdate {
adding pods lister
2019-08-01 12:59:22 -04:00
t.Errorf("expected pod update: %+v, got %+v for %+v", item.expectedPodUpdate, podReasonUpdate, item.description)
Added unit test for adding reason with termination.
2016-11-02 15:56:19 -04:00
}
}
}
Rename node status to node health in NodeLifecycleController Since we are going to treat both node status and node lease as node heartbeat/health signals, this PR makes the renmae changes, so that the follow-up PRs are easier to review.
2018-10-01 14:32:56 -04:00
func TestMonitorNodeHealthUpdateStatus(t *testing.T) {
refactor: generated
2016-12-03 13:57:26 -05:00
fakeNow := metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC)
Sync node status from node controller to master.
2015-01-16 17:28:20 -05:00
table := []struct {
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
fakeNodeHandler *testutil.FakeNodeHandler
timeToPass time.Duration
newNodeStatus v1.NodeStatus
expectedRequestCount int
expectedNodes []*v1.Node
expectedPodStatusUpdate bool
Sync node status from node controller to master.
2015-01-16 17:28:20 -05:00
}{
Use fake time in nodecontroller unittest; rename receiver 's' to 'nc'
2015-03-22 21:10:35 -04:00
// Node created long time ago, without status:
// Expect Unknown status posted from node controller.
Sync node status from node controller to master.
2015-01-16 17:28:20 -05:00
{
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
fakeNodeHandler: &testutil.FakeNodeHandler{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Existing: []*v1.Node{
Node controller monitor node status
2015-03-11 23:00:52 -04:00
{
refactor: use metav1.ObjectMeta in other types
2017-01-16 22:38:19 -05:00
ObjectMeta: metav1.ObjectMeta{
Node controller monitor node status
2015-03-11 23:00:52 -04:00
Name: "node0",
refactor: generated
2016-12-03 13:57:26 -05:00
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Node controller monitor node status
2015-03-11 23:00:52 -04:00
},
},
},
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
Move hostip to sync status
2015-01-30 15:50:47 -05:00
},
Node controller monitor node status
2015-03-11 23:00:52 -04:00
expectedRequestCount: 2, // List+Update
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
expectedNodes: []*v1.Node{
Sync node status from node controller to master.
2015-01-16 17:28:20 -05:00
{
refactor: use metav1.ObjectMeta in other types
2017-01-16 22:38:19 -05:00
ObjectMeta: metav1.ObjectMeta{
Node controller monitor node status
2015-03-11 23:00:52 -04:00
Name: "node0",
refactor: generated
2016-12-03 13:57:26 -05:00
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Node controller monitor node status
2015-03-11 23:00:52 -04:00
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
Populate node statu transition timestamp and reason
2015-02-04 16:56:59 -05:00
{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
Fix reason reporting in controller
2015-09-11 06:08:09 -04:00
Reason: "NodeStatusNeverUpdated",
Use node out of disk condition in the scheduler while scheduling pods. Set the out of disk node condition to unknown in the node controller if the kubelet does not report its node condition in a long time. Update node controller unit tests. Implement a node condition predicate function that checks if a given node satisfies the conditions defined by the predicate and if it does, use that node for scheduling pods. The predicate function takes both NodeReady and NodeOutOfDisk into consideration to determine if a node is fit for scheduling pods. The predicate is then passed to the node lister in the scheduler factory so that the node lister can run the predicate function on the nodes when schedling pods thereby omitting nodes that does not satisfy the predicate. Also update listers test.
2015-10-22 15:47:43 -04:00
Message: "Kubelet never posted node status.",
refactor: generated
2016-12-03 13:57:26 -05:00
LastHeartbeatTime: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Use node out of disk condition in the scheduler while scheduling pods. Set the out of disk node condition to unknown in the node controller if the kubelet does not report its node condition in a long time. Update node controller unit tests. Implement a node condition predicate function that checks if a given node satisfies the conditions defined by the predicate and if it does, use that node for scheduling pods. The predicate function takes both NodeReady and NodeOutOfDisk into consideration to determine if a node is fit for scheduling pods. The predicate is then passed to the node lister in the scheduler factory so that the node lister can run the predicate function on the nodes when schedling pods thereby omitting nodes that does not satisfy the predicate. Also update listers test.
2015-10-22 15:47:43 -04:00
LastTransitionTime: fakeNow,
},
set all node conditions to Unknown when node is unreachable
2016-11-10 13:09:27 -05:00
{
Type: v1.NodeMemoryPressure,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
LastTransitionTime: fakeNow,
},
{
Type: v1.NodeDiskPressure,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
LastTransitionTime: fakeNow,
},
NodePIDPressure condition should set to unknown when node lost connnection with contorl
2018-09-13 20:50:05 -04:00
{
Type: v1.NodePIDPressure,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
LastTransitionTime: fakeNow,
},
Populate node statu transition timestamp and reason
2015-02-04 16:56:59 -05:00
},
Node controller monitor node status
2015-03-11 23:00:52 -04:00
},
},
},
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
expectedPodStatusUpdate: false, // Pod was never scheduled
Node controller monitor node status
2015-03-11 23:00:52 -04:00
},
Use fake time in nodecontroller unittest; rename receiver 's' to 'nc'
2015-03-22 21:10:35 -04:00
// Node created recently, without status.
// Expect no action from node controller (within startup grace period).
Node controller monitor node status
2015-03-11 23:00:52 -04:00
{
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
fakeNodeHandler: &testutil.FakeNodeHandler{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Existing: []*v1.Node{
Node controller monitor node status
2015-03-11 23:00:52 -04:00
{
refactor: use metav1.ObjectMeta in other types
2017-01-16 22:38:19 -05:00
ObjectMeta: metav1.ObjectMeta{
Node controller monitor node status
2015-03-11 23:00:52 -04:00
Name: "node0",
Use fake time in nodecontroller unittest; rename receiver 's' to 'nc'
2015-03-22 21:10:35 -04:00
CreationTimestamp: fakeNow,
Node controller monitor node status
2015-03-11 23:00:52 -04:00
},
},
},
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
Node controller monitor node status
2015-03-11 23:00:52 -04:00
},
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
expectedRequestCount: 1, // List
expectedNodes: nil,
expectedPodStatusUpdate: false,
Node controller monitor node status
2015-03-11 23:00:52 -04:00
},
Use fake time in nodecontroller unittest; rename receiver 's' to 'nc'
2015-03-22 21:10:35 -04:00
// Node created long time ago, with status updated by kubelet exceeds grace period.
// Expect Unknown status posted from node controller.
Node controller monitor node status
2015-03-11 23:00:52 -04:00
{
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
fakeNodeHandler: &testutil.FakeNodeHandler{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Existing: []*v1.Node{
Node controller monitor node status
2015-03-11 23:00:52 -04:00
{
refactor: use metav1.ObjectMeta in other types
2017-01-16 22:38:19 -05:00
ObjectMeta: metav1.ObjectMeta{
Node controller monitor node status
2015-03-11 23:00:52 -04:00
Name: "node0",
refactor: generated
2016-12-03 13:57:26 -05:00
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Node controller monitor node status
2015-03-11 23:00:52 -04:00
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
Node controller monitor node status
2015-03-11 23:00:52 -04:00
{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Type: v1.NodeReady,
Status: v1.ConditionTrue,
Use fake time in nodecontroller unittest; rename receiver 's' to 'nc'
2015-03-22 21:10:35 -04:00
// Node status hasn't been updated for 1hr.
refactor: generated
2016-12-03 13:57:26 -05:00
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
Node controller monitor node status
2015-03-11 23:00:52 -04:00
},
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
Add validation for node creation.
2015-03-24 13:24:07 -04:00
},
},
Move hostip to sync status
2015-01-30 15:50:47 -05:00
},
Sync node status from node controller to master.
2015-01-16 17:28:20 -05:00
},
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
Node controller monitor node status
2015-03-11 23:00:52 -04:00
},
Add timestamp cache to NodeController
2015-03-31 11:15:39 -04:00
expectedRequestCount: 3, // (List+)List+Update
timeToPass: time.Hour,
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
newNodeStatus: v1.NodeStatus{
Conditions: []v1.NodeCondition{
Add timestamp cache to NodeController
2015-03-31 11:15:39 -04:00
{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Type: v1.NodeReady,
Status: v1.ConditionTrue,
Add timestamp cache to NodeController
2015-03-31 11:15:39 -04:00
// Node status hasn't been updated for 1hr.
refactor: generated
2016-12-03 13:57:26 -05:00
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
Add timestamp cache to NodeController
2015-03-31 11:15:39 -04:00
},
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
Add timestamp cache to NodeController
2015-03-31 11:15:39 -04:00
},
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
expectedNodes: []*v1.Node{
Sync node status from node controller to master.
2015-01-16 17:28:20 -05:00
{
refactor: use metav1.ObjectMeta in other types
2017-01-16 22:38:19 -05:00
ObjectMeta: metav1.ObjectMeta{
Node controller monitor node status
2015-03-11 23:00:52 -04:00
Name: "node0",
refactor: generated
2016-12-03 13:57:26 -05:00
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Node controller monitor node status
2015-03-11 23:00:52 -04:00
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
Populate node statu transition timestamp and reason
2015-02-04 16:56:59 -05:00
{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
Use node out of disk condition in the scheduler while scheduling pods. Set the out of disk node condition to unknown in the node controller if the kubelet does not report its node condition in a long time. Update node controller unit tests. Implement a node condition predicate function that checks if a given node satisfies the conditions defined by the predicate and if it does, use that node for scheduling pods. The predicate function takes both NodeReady and NodeOutOfDisk into consideration to determine if a node is fit for scheduling pods. The predicate is then passed to the node lister in the scheduler factory so that the node lister can run the predicate function on the nodes when schedling pods thereby omitting nodes that does not satisfy the predicate. Also update listers test.
2015-10-22 15:47:43 -04:00
Reason: "NodeStatusUnknown",
Message: "Kubelet stopped posting node status.",
refactor: generated
2016-12-03 13:57:26 -05:00
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
nodelifecycle: set OutOfDisk unknown on node timeout
2018-02-02 14:15:27 -05:00
LastTransitionTime: metav1.Time{Time: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC).Add(time.Hour)},
},
set all node conditions to Unknown when node is unreachable
2016-11-10 13:09:27 -05:00
{
Type: v1.NodeMemoryPressure,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC), // should default to node creation time if condition was never updated
LastTransitionTime: metav1.Time{Time: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC).Add(time.Hour)},
},
{
Type: v1.NodeDiskPressure,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC), // should default to node creation time if condition was never updated
LastTransitionTime: metav1.Time{Time: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC).Add(time.Hour)},
},
NodePIDPressure condition should set to unknown when node lost connnection with contorl
2018-09-13 20:50:05 -04:00
{
Type: v1.NodePIDPressure,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC), // should default to node creation time if condition was never updated
LastTransitionTime: metav1.Time{Time: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC).Add(time.Hour)},
},
Populate node statu transition timestamp and reason
2015-02-04 16:56:59 -05:00
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
Add validation for node creation.
2015-03-24 13:24:07 -04:00
},
},
Node controller monitor node status
2015-03-11 23:00:52 -04:00
},
},
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
expectedPodStatusUpdate: true,
Node controller monitor node status
2015-03-11 23:00:52 -04:00
},
// Node created long time ago, with status updated recently.
Use fake time in nodecontroller unittest; rename receiver 's' to 'nc'
2015-03-22 21:10:35 -04:00
// Expect no action from node controller (within monitor grace period).
Node controller monitor node status
2015-03-11 23:00:52 -04:00
{
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
fakeNodeHandler: &testutil.FakeNodeHandler{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Existing: []*v1.Node{
Node controller monitor node status
2015-03-11 23:00:52 -04:00
{
refactor: use metav1.ObjectMeta in other types
2017-01-16 22:38:19 -05:00
ObjectMeta: metav1.ObjectMeta{
Node controller monitor node status
2015-03-11 23:00:52 -04:00
Name: "node0",
refactor: generated
2016-12-03 13:57:26 -05:00
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Node controller monitor node status
2015-03-11 23:00:52 -04:00
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
Node controller monitor node status
2015-03-11 23:00:52 -04:00
{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Type: v1.NodeReady,
Status: v1.ConditionTrue,
Use fake time in nodecontroller unittest; rename receiver 's' to 'nc'
2015-03-22 21:10:35 -04:00
// Node status has just been updated.
Remove Node Controllers ability to pull status from Kubelet
2015-03-27 10:09:51 -04:00
LastHeartbeatTime: fakeNow,
Use fake time in nodecontroller unittest; rename receiver 's' to 'nc'
2015-03-22 21:10:35 -04:00
LastTransitionTime: fakeNow,
Node controller monitor node status
2015-03-11 23:00:52 -04:00
},
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
Add validation for node creation.
2015-03-24 13:24:07 -04:00
},
},
Move hostip to sync status
2015-01-30 15:50:47 -05:00
},
Sync node status from node controller to master.
2015-01-16 17:28:20 -05:00
},
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
Sync node status from node controller to master.
2015-01-16 17:28:20 -05:00
},
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
expectedRequestCount: 1, // List
expectedNodes: nil,
expectedPodStatusUpdate: false,
Sync node status from node controller to master.
2015-01-16 17:28:20 -05:00
},
Rename minioncontroller to nodecontroller
2014-12-19 04:27:01 -05:00
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx := ktesting.Init(t)
Use node out of disk condition in the scheduler while scheduling pods. Set the out of disk node condition to unknown in the node controller if the kubelet does not report its node condition in a long time. Update node controller unit tests. Implement a node condition predicate function that checks if a given node satisfies the conditions defined by the predicate and if it does, use that node for scheduling pods. The predicate function takes both NodeReady and NodeOutOfDisk into consideration to determine if a node is fit for scheduling pods. The predicate is then passed to the node lister in the scheduler factory so that the node lister can run the predicate function on the nodes when schedling pods thereby omitting nodes that does not satisfy the predicate. Also update listers test.
2015-10-22 15:47:43 -04:00
for i, item := range table {
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
nodeController, _ := newNodeLifecycleControllerFromClient(
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx,
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
item.fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
testNodeMonitorPeriod,
Remove CLI flag enable-taint-manager Signed-off-by: kerthcet <kerthcet@gmail.com>
2022-09-14 05:04:08 -04:00
)
refactor: generated
2016-12-03 13:57:26 -05:00
nodeController.now = func() metav1.Time { return fakeNow }
Apply davidopps comments to TaintController PR
2017-02-13 05:48:34 -05:00
nodeController.recorder = testutil.NewFakeRecorder()
adding fakeGetPodsAssignedToNode
2019-09-27 11:28:58 -04:00
nodeController.getPodsAssignedToNode = fakeGetPodsAssignedToNode(item.fakeNodeHandler.Clientset)
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
if err := nodeController.syncNodeStore(item.fakeNodeHandler); err != nil {
NodeController listing nodes from cache instead of cache in apiserver
2016-12-19 05:15:39 -05:00
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
Sync node status from node controller to master.
2015-01-16 17:28:20 -05:00
t.Errorf("unexpected error: %v", err)
}
Add timestamp cache to NodeController
2015-03-31 11:15:39 -04:00
if item.timeToPass > 0 {
refactor: generated
2016-12-03 13:57:26 -05:00
nodeController.now = func() metav1.Time { return metav1.Time{Time: fakeNow.Add(item.timeToPass)} }
Add timestamp cache to NodeController
2015-03-31 11:15:39 -04:00
item.fakeNodeHandler.Existing[0].Status = item.newNodeStatus
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
if err := nodeController.syncNodeStore(item.fakeNodeHandler); err != nil {
NodeController listing nodes from cache instead of cache in apiserver
2016-12-19 05:15:39 -05:00
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
Add timestamp cache to NodeController
2015-03-31 11:15:39 -04:00
t.Errorf("unexpected error: %v", err)
}
}
Node controller monitor node status
2015-03-11 23:00:52 -04:00
if item.expectedRequestCount != item.fakeNodeHandler.RequestCount {
Sync node status from node controller to master.
2015-01-16 17:28:20 -05:00
t.Errorf("expected %v call, but got %v.", item.expectedRequestCount, item.fakeNodeHandler.RequestCount)
}
pkg/api: move Semantic equality to k8s.io/apimachinery/pkg/api/equality
2017-01-25 08:39:54 -05:00
if len(item.fakeNodeHandler.UpdatedNodes) > 0 && !apiequality.Semantic.DeepEqual(item.expectedNodes, item.fakeNodeHandler.UpdatedNodes) {
Replace uses of diff.ObjectDiff with cmp.Diff ObjectDiff is already a shim over cmp.Diff, so no actual output or behavior changes
2023-03-23 14:29:01 -04:00
t.Errorf("Case[%d] unexpected nodes: %s", i, cmp.Diff(item.expectedNodes[0], item.fakeNodeHandler.UpdatedNodes[0]))
Use node out of disk condition in the scheduler while scheduling pods. Set the out of disk node condition to unknown in the node controller if the kubelet does not report its node condition in a long time. Update node controller unit tests. Implement a node condition predicate function that checks if a given node satisfies the conditions defined by the predicate and if it does, use that node for scheduling pods. The predicate function takes both NodeReady and NodeOutOfDisk into consideration to determine if a node is fit for scheduling pods. The predicate is then passed to the node lister in the scheduler factory so that the node lister can run the predicate function on the nodes when schedling pods thereby omitting nodes that does not satisfy the predicate. Also update listers test.
2015-10-22 15:47:43 -04:00
}
pkg/api: move Semantic equality to k8s.io/apimachinery/pkg/api/equality
2017-01-25 08:39:54 -05:00
if len(item.fakeNodeHandler.UpdatedNodeStatuses) > 0 && !apiequality.Semantic.DeepEqual(item.expectedNodes, item.fakeNodeHandler.UpdatedNodeStatuses) {
Replace uses of diff.ObjectDiff with cmp.Diff ObjectDiff is already a shim over cmp.Diff, so no actual output or behavior changes
2023-03-23 14:29:01 -04:00
t.Errorf("Case[%d] unexpected nodes: %s", i, cmp.Diff(item.expectedNodes[0], item.fakeNodeHandler.UpdatedNodeStatuses[0]))
Sync node status from node controller to master.
2015-01-16 17:28:20 -05:00
}
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
podStatusUpdated := false
for _, action := range item.fakeNodeHandler.Actions() {
if action.GetVerb() == "update" && action.GetResource().Resource == "pods" && action.GetSubresource() == "status" {
podStatusUpdated = true
}
}
if podStatusUpdated != item.expectedPodStatusUpdate {
t.Errorf("Case[%d] expect pod status updated to be %v, but got %v", i, item.expectedPodStatusUpdate, podStatusUpdated)
}
}
}
func TestMonitorNodeHealthUpdateNodeAndPodStatusWithLease(t *testing.T) {
nodeCreationTime := metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC)
fakeNow := metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC)
testcases := []struct {
description string
fakeNodeHandler *testutil.FakeNodeHandler
Swtich nodelifecyclecontroller to coordination/v1
2019-10-16 03:11:03 -04:00
lease *coordv1.Lease
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
timeToPass time.Duration
newNodeStatus v1.NodeStatus
Swtich nodelifecyclecontroller to coordination/v1
2019-10-16 03:11:03 -04:00
newLease *coordv1.Lease
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
expectedRequestCount int
expectedNodes []*v1.Node
expectedPodStatusUpdate bool
}{
// Node created recently, without status. Node lease is missing.
// Expect no action from node controller (within startup grace period).
{
description: "Node created recently, without status. Node lease is missing.",
fakeNodeHandler: &testutil.FakeNodeHandler{
Existing: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
},
expectedRequestCount: 1, // List
expectedNodes: nil,
expectedPodStatusUpdate: false,
},
// Node created recently, without status. Node lease is renewed recently.
// Expect no action from node controller (within startup grace period).
{
description: "Node created recently, without status. Node lease is renewed recently.",
fakeNodeHandler: &testutil.FakeNodeHandler{
Existing: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
},
lease: createNodeLease("node0", metav1.NewMicroTime(fakeNow.Time)),
expectedRequestCount: 1, // List
expectedNodes: nil,
expectedPodStatusUpdate: false,
},
// Node created long time ago, without status. Node lease is missing.
// Expect Unknown status posted from node controller.
{
description: "Node created long time ago, without status. Node lease is missing.",
fakeNodeHandler: &testutil.FakeNodeHandler{
Existing: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: nodeCreationTime,
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
},
expectedRequestCount: 2, // List+Update
expectedNodes: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: nodeCreationTime,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: nodeCreationTime,
LastTransitionTime: fakeNow,
},
{
Type: v1.NodeMemoryPressure,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: nodeCreationTime,
LastTransitionTime: fakeNow,
},
{
Type: v1.NodeDiskPressure,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: nodeCreationTime,
LastTransitionTime: fakeNow,
},
{
Type: v1.NodePIDPressure,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: nodeCreationTime,
LastTransitionTime: fakeNow,
},
},
},
},
},
expectedPodStatusUpdate: false, // Pod was never scheduled because the node was never ready.
},
// Node created long time ago, without status. Node lease is renewed recently.
// Expect no action from node controller (within monitor grace period).
{
description: "Node created long time ago, without status. Node lease is renewed recently.",
fakeNodeHandler: &testutil.FakeNodeHandler{
Existing: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: nodeCreationTime,
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
},
lease: createNodeLease("node0", metav1.NewMicroTime(fakeNow.Time)),
timeToPass: time.Hour,
newLease: createNodeLease("node0", metav1.NewMicroTime(fakeNow.Time.Add(time.Hour))), // Lease is renewed after 1 hour.
expectedRequestCount: 2, // List+List
expectedNodes: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: nodeCreationTime,
},
},
},
expectedPodStatusUpdate: false,
},
// Node created long time ago, without status. Node lease is expired.
// Expect Unknown status posted from node controller.
{
description: "Node created long time ago, without status. Node lease is expired.",
fakeNodeHandler: &testutil.FakeNodeHandler{
Existing: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: nodeCreationTime,
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
},
lease: createNodeLease("node0", metav1.NewMicroTime(fakeNow.Time)),
timeToPass: time.Hour,
newLease: createNodeLease("node0", metav1.NewMicroTime(fakeNow.Time)), // Lease is not renewed after 1 hour.
expectedRequestCount: 3, // List+List+Update
expectedNodes: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: nodeCreationTime,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: nodeCreationTime,
LastTransitionTime: metav1.Time{Time: fakeNow.Add(time.Hour)},
},
{
Type: v1.NodeMemoryPressure,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: nodeCreationTime,
LastTransitionTime: metav1.Time{Time: fakeNow.Add(time.Hour)},
},
{
Type: v1.NodeDiskPressure,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: nodeCreationTime,
LastTransitionTime: metav1.Time{Time: fakeNow.Add(time.Hour)},
},
{
Type: v1.NodePIDPressure,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: nodeCreationTime,
LastTransitionTime: metav1.Time{Time: fakeNow.Add(time.Hour)},
},
},
},
},
},
expectedPodStatusUpdate: false,
},
// Node created long time ago, with status updated by kubelet exceeds grace period. Node lease is renewed.
// Expect no action from node controller (within monitor grace period).
{
description: "Node created long time ago, with status updated by kubelet exceeds grace period. Node lease is renewed.",
fakeNodeHandler: &testutil.FakeNodeHandler{
Existing: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: nodeCreationTime,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
{
cleanup logic related with OutOfDisk - cleanup OOD logic in scheduling and node controller - update comments and testcases
2018-12-13 03:31:46 -05:00
Type: v1.NodeDiskPressure,
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
Status: v1.ConditionFalse,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
},
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
},
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
},
lease: createNodeLease("node0", metav1.NewMicroTime(fakeNow.Time)),
expectedRequestCount: 2, // List+List
timeToPass: time.Hour,
newNodeStatus: v1.NodeStatus{
// Node status hasn't been updated for 1 hour.
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
{
cleanup logic related with OutOfDisk - cleanup OOD logic in scheduling and node controller - update comments and testcases
2018-12-13 03:31:46 -05:00
Type: v1.NodeDiskPressure,
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
Status: v1.ConditionFalse,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
},
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
},
},
newLease: createNodeLease("node0", metav1.NewMicroTime(fakeNow.Time.Add(time.Hour))), // Lease is renewed after 1 hour.
expectedNodes: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: nodeCreationTime,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
{
cleanup logic related with OutOfDisk - cleanup OOD logic in scheduling and node controller - update comments and testcases
2018-12-13 03:31:46 -05:00
Type: v1.NodeDiskPressure,
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
Status: v1.ConditionFalse,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
},
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
},
},
},
},
expectedPodStatusUpdate: false,
},
// Node created long time ago, with status updated by kubelet recently. Node lease is expired.
// Expect no action from node controller (within monitor grace period).
{
description: "Node created long time ago, with status updated by kubelet recently. Node lease is expired.",
fakeNodeHandler: &testutil.FakeNodeHandler{
Existing: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: nodeCreationTime,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
{
cleanup logic related with OutOfDisk - cleanup OOD logic in scheduling and node controller - update comments and testcases
2018-12-13 03:31:46 -05:00
Type: v1.NodeDiskPressure,
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
Status: v1.ConditionFalse,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
},
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
},
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
},
lease: createNodeLease("node0", metav1.NewMicroTime(fakeNow.Time)),
expectedRequestCount: 2, // List+List
timeToPass: time.Hour,
newNodeStatus: v1.NodeStatus{
// Node status is updated after 1 hour.
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Time{Time: fakeNow.Add(time.Hour)},
LastTransitionTime: fakeNow,
},
{
cleanup logic related with OutOfDisk - cleanup OOD logic in scheduling and node controller - update comments and testcases
2018-12-13 03:31:46 -05:00
Type: v1.NodeDiskPressure,
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
Status: v1.ConditionFalse,
LastHeartbeatTime: metav1.Time{Time: fakeNow.Add(time.Hour)},
LastTransitionTime: fakeNow,
},
},
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
},
},
newLease: createNodeLease("node0", metav1.NewMicroTime(fakeNow.Time)), // Lease is not renewed after 1 hour.
expectedNodes: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: nodeCreationTime,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Time{Time: fakeNow.Add(time.Hour)},
LastTransitionTime: fakeNow,
},
{
cleanup logic related with OutOfDisk - cleanup OOD logic in scheduling and node controller - update comments and testcases
2018-12-13 03:31:46 -05:00
Type: v1.NodeDiskPressure,
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
Status: v1.ConditionFalse,
LastHeartbeatTime: metav1.Time{Time: fakeNow.Add(time.Hour)},
LastTransitionTime: fakeNow,
},
},
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
},
},
},
},
expectedPodStatusUpdate: false,
},
// Node created long time ago, with status updated by kubelet exceeds grace period. Node lease is also expired.
// Expect Unknown status posted from node controller.
{
description: "Node created long time ago, with status updated by kubelet exceeds grace period. Node lease is also expired.",
fakeNodeHandler: &testutil.FakeNodeHandler{
Existing: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: nodeCreationTime,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
},
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
},
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
},
lease: createNodeLease("node0", metav1.NewMicroTime(fakeNow.Time)),
expectedRequestCount: 3, // List+List+Update
timeToPass: time.Hour,
newNodeStatus: v1.NodeStatus{
// Node status hasn't been updated for 1 hour.
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
},
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
},
},
newLease: createNodeLease("node0", metav1.NewMicroTime(fakeNow.Time)), // Lease is not renewed after 1 hour.
expectedNodes: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: nodeCreationTime,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
Reason: "NodeStatusUnknown",
Message: "Kubelet stopped posting node status.",
LastHeartbeatTime: fakeNow,
LastTransitionTime: metav1.Time{Time: fakeNow.Add(time.Hour)},
},
{
Type: v1.NodeMemoryPressure,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: nodeCreationTime, // should default to node creation time if condition was never updated
LastTransitionTime: metav1.Time{Time: fakeNow.Add(time.Hour)},
},
{
Type: v1.NodeDiskPressure,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: nodeCreationTime, // should default to node creation time if condition was never updated
LastTransitionTime: metav1.Time{Time: fakeNow.Add(time.Hour)},
},
{
Type: v1.NodePIDPressure,
Status: v1.ConditionUnknown,
Reason: "NodeStatusNeverUpdated",
Message: "Kubelet never posted node status.",
LastHeartbeatTime: nodeCreationTime, // should default to node creation time if condition was never updated
LastTransitionTime: metav1.Time{Time: fakeNow.Add(time.Hour)},
},
},
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
},
},
},
},
expectedPodStatusUpdate: true,
},
}
for _, item := range testcases {
t.Run(item.description, func(t *testing.T) {
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx := ktesting.Init(t)
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
nodeController, _ := newNodeLifecycleControllerFromClient(
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx,
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
item.fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
testNodeMonitorPeriod,
Remove CLI flag enable-taint-manager Signed-off-by: kerthcet <kerthcet@gmail.com>
2022-09-14 05:04:08 -04:00
)
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
nodeController.now = func() metav1.Time { return fakeNow }
nodeController.recorder = testutil.NewFakeRecorder()
adding fakeGetPodsAssignedToNode
2019-09-27 11:28:58 -04:00
nodeController.getPodsAssignedToNode = fakeGetPodsAssignedToNode(item.fakeNodeHandler.Clientset)
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
if err := nodeController.syncNodeStore(item.fakeNodeHandler); err != nil {
t.Fatalf("unexpected error: %v", err)
}
if err := nodeController.syncLeaseStore(item.lease); err != nil {
t.Fatalf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
t.Fatalf("unexpected error: %v", err)
}
if item.timeToPass > 0 {
nodeController.now = func() metav1.Time { return metav1.Time{Time: fakeNow.Add(item.timeToPass)} }
item.fakeNodeHandler.Existing[0].Status = item.newNodeStatus
if err := nodeController.syncNodeStore(item.fakeNodeHandler); err != nil {
t.Fatalf("unexpected error: %v", err)
}
if err := nodeController.syncLeaseStore(item.newLease); err != nil {
t.Fatalf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
t.Fatalf("unexpected error: %v", err)
}
}
if item.expectedRequestCount != item.fakeNodeHandler.RequestCount {
t.Errorf("expected %v call, but got %v.", item.expectedRequestCount, item.fakeNodeHandler.RequestCount)
}
if len(item.fakeNodeHandler.UpdatedNodes) > 0 && !apiequality.Semantic.DeepEqual(item.expectedNodes, item.fakeNodeHandler.UpdatedNodes) {
Replace uses of diff.ObjectDiff with cmp.Diff ObjectDiff is already a shim over cmp.Diff, so no actual output or behavior changes
2023-03-23 14:29:01 -04:00
t.Errorf("unexpected nodes: %s", cmp.Diff(item.expectedNodes[0], item.fakeNodeHandler.UpdatedNodes[0]))
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
}
if len(item.fakeNodeHandler.UpdatedNodeStatuses) > 0 && !apiequality.Semantic.DeepEqual(item.expectedNodes, item.fakeNodeHandler.UpdatedNodeStatuses) {
Replace uses of diff.ObjectDiff with cmp.Diff ObjectDiff is already a shim over cmp.Diff, so no actual output or behavior changes
2023-03-23 14:29:01 -04:00
t.Errorf("unexpected nodes: %s", cmp.Diff(item.expectedNodes[0], item.fakeNodeHandler.UpdatedNodeStatuses[0]))
NodeLifecycleController treats node lease renewal as a heartbeat signal
2018-10-01 14:32:56 -04:00
}
podStatusUpdated := false
for _, action := range item.fakeNodeHandler.Actions() {
if action.GetVerb() == "update" && action.GetResource().Resource == "pods" && action.GetSubresource() == "status" {
podStatusUpdated = true
}
}
if podStatusUpdated != item.expectedPodStatusUpdate {
t.Errorf("expect pod status updated to be %v, but got %v", item.expectedPodStatusUpdate, podStatusUpdated)
}
})
Rename minioncontroller to nodecontroller
2014-12-19 04:27:01 -05:00
}
Sync node status from node controller to master.
2015-01-16 17:28:20 -05:00
}
Rename node status to node health in NodeLifecycleController Since we are going to treat both node status and node lease as node heartbeat/health signals, this PR makes the renmae changes, so that the follow-up PRs are easier to review.
2018-10-01 14:32:56 -04:00
func TestMonitorNodeHealthMarkPodsNotReady(t *testing.T) {
refactor: generated
2016-12-03 13:57:26 -05:00
fakeNow := metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC)
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
table := []struct {
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
fakeNodeHandler *testutil.FakeNodeHandler
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
timeToPass time.Duration
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
newNodeStatus v1.NodeStatus
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
expectedPodStatusUpdate bool
}{
// Node created recently, without status.
// Expect no action from node controller (within startup grace period).
{
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
fakeNodeHandler: &testutil.FakeNodeHandler{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Existing: []*v1.Node{
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
{
refactor: use metav1.ObjectMeta in other types
2017-01-16 22:38:19 -05:00
ObjectMeta: metav1.ObjectMeta{
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
Name: "node0",
CreationTimestamp: fakeNow,
},
},
},
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
},
expectedPodStatusUpdate: false,
},
// Node created long time ago, with status updated recently.
// Expect no action from node controller (within monitor grace period).
{
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
fakeNodeHandler: &testutil.FakeNodeHandler{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Existing: []*v1.Node{
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
{
refactor: use metav1.ObjectMeta in other types
2017-01-16 22:38:19 -05:00
ObjectMeta: metav1.ObjectMeta{
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
Name: "node0",
refactor: generated
2016-12-03 13:57:26 -05:00
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Type: v1.NodeReady,
Status: v1.ConditionTrue,
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
// Node status has just been updated.
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
},
},
},
},
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
},
expectedPodStatusUpdate: false,
},
// Node created long time ago, with status updated by kubelet exceeds grace period.
// Expect pods status updated and Unknown node status posted from node controller
{
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
fakeNodeHandler: &testutil.FakeNodeHandler{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Existing: []*v1.Node{
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
{
refactor: use metav1.ObjectMeta in other types
2017-01-16 22:38:19 -05:00
ObjectMeta: metav1.ObjectMeta{
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
Name: "node0",
refactor: generated
2016-12-03 13:57:26 -05:00
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Type: v1.NodeReady,
Status: v1.ConditionTrue,
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
// Node status hasn't been updated for 1hr.
refactor: generated
2016-12-03 13:57:26 -05:00
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
},
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
},
},
},
},
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
},
timeToPass: 1 * time.Minute,
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
newNodeStatus: v1.NodeStatus{
Conditions: []v1.NodeCondition{
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Type: v1.NodeReady,
Status: v1.ConditionTrue,
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
// Node status hasn't been updated for 1hr.
refactor: generated
2016-12-03 13:57:26 -05:00
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
},
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
},
},
expectedPodStatusUpdate: true,
},
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx := ktesting.Init(t)
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
for i, item := range table {
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
nodeController, _ := newNodeLifecycleControllerFromClient(
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx,
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
item.fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
testNodeMonitorPeriod,
Remove CLI flag enable-taint-manager Signed-off-by: kerthcet <kerthcet@gmail.com>
2022-09-14 05:04:08 -04:00
)
refactor: generated
2016-12-03 13:57:26 -05:00
nodeController.now = func() metav1.Time { return fakeNow }
Apply davidopps comments to TaintController PR
2017-02-13 05:48:34 -05:00
nodeController.recorder = testutil.NewFakeRecorder()
adding fakeGetPodsAssignedToNode
2019-09-27 11:28:58 -04:00
nodeController.getPodsAssignedToNode = fakeGetPodsAssignedToNode(item.fakeNodeHandler.Clientset)
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
if err := nodeController.syncNodeStore(item.fakeNodeHandler); err != nil {
NodeController listing nodes from cache instead of cache in apiserver
2016-12-19 05:15:39 -05:00
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
t.Errorf("Case[%d] unexpected error: %v", i, err)
}
if item.timeToPass > 0 {
refactor: generated
2016-12-03 13:57:26 -05:00
nodeController.now = func() metav1.Time { return metav1.Time{Time: fakeNow.Add(item.timeToPass)} }
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
item.fakeNodeHandler.Existing[0].Status = item.newNodeStatus
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
if err := nodeController.syncNodeStore(item.fakeNodeHandler); err != nil {
NodeController listing nodes from cache instead of cache in apiserver
2016-12-19 05:15:39 -05:00
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
t.Errorf("Case[%d] unexpected error: %v", i, err)
}
}
podStatusUpdated := false
for _, action := range item.fakeNodeHandler.Actions() {
use fully qualified resource in fake clients actions
2016-04-13 18:33:15 -04:00
if action.GetVerb() == "update" && action.GetResource().Resource == "pods" && action.GetSubresource() == "status" {
update pod status once node becomes NotReady
2015-11-24 17:46:17 -05:00
podStatusUpdated = true
}
}
if podStatusUpdated != item.expectedPodStatusUpdate {
t.Errorf("Case[%d] expect pod status updated to be %v, but got %v", i, item.expectedPodStatusUpdate, podStatusUpdated)
}
}
}
controller/nodelifecycle: Make monitorNodeHealth process nodes concurrently Marking the pods not ready on a node requires looping over them and updating each pod's status one at a time. This is performed serially, and can take a while if we're processing each node serially as well. Since the time is spent waiting on io, there's an opportunity to go faster by processing multiple nodes concurrently. This change modifies the loop to process nodes in parallel, using the same number of workers as doNodeProcessingPassWorker. This change also introduces histogram metrics to better observe monitorNodeHealth.
2023-01-10 23:54:39 -05:00
// TestMonitorNodeHealthMarkPodsNotReadyWithWorkerSize tests the happy path of
// TestMonitorNodeHealthMarkPodsNotReady with a large number of nodes/pods and
// varying numbers of workers.
func TestMonitorNodeHealthMarkPodsNotReadyWithWorkerSize(t *testing.T) {
const numNodes = 50
const podsPerNode = 100
makeNodes := func() []*v1.Node {
nodes := make([]*v1.Node, numNodes)
// Node created long time ago, with status updated by kubelet exceeds grace period.
// Expect pods status updated and Unknown node status posted from node controller
for i := 0; i < numNodes; i++ {
nodes[i] = &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("node%d", i),
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
// Node status hasn't been updated for 1hr.
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
},
},
}
}
return nodes
}
makePods := func() []v1.Pod {
pods := make([]v1.Pod, numNodes*podsPerNode)
for i := 0; i < numNodes*podsPerNode; i++ {
pods[i] = *testutil.NewPod(fmt.Sprintf("pod%d", i), fmt.Sprintf("node%d", i%numNodes))
}
return pods
}
table := []struct {
workers int
}{
{workers: 0}, // will default to scheduler.UpdateWorkerSize
{workers: 1},
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx := ktesting.Init(t)
controller/nodelifecycle: Make monitorNodeHealth process nodes concurrently Marking the pods not ready on a node requires looping over them and updating each pod's status one at a time. This is performed serially, and can take a while if we're processing each node serially as well. Since the time is spent waiting on io, there's an opportunity to go faster by processing multiple nodes concurrently. This change modifies the loop to process nodes in parallel, using the same number of workers as doNodeProcessingPassWorker. This change also introduces histogram metrics to better observe monitorNodeHealth.
2023-01-10 23:54:39 -05:00
for i, item := range table {
fakeNow := metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC)
fakeNodeHandler := &testutil.FakeNodeHandler{
Existing: makeNodes(),
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: makePods()}),
}
nodeController, _ := newNodeLifecycleControllerFromClient(
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx,
controller/nodelifecycle: Make monitorNodeHealth process nodes concurrently Marking the pods not ready on a node requires looping over them and updating each pod's status one at a time. This is performed serially, and can take a while if we're processing each node serially as well. Since the time is spent waiting on io, there's an opportunity to go faster by processing multiple nodes concurrently. This change modifies the loop to process nodes in parallel, using the same number of workers as doNodeProcessingPassWorker. This change also introduces histogram metrics to better observe monitorNodeHealth.
2023-01-10 23:54:39 -05:00
fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
Remove CLI flag enable-taint-manager Signed-off-by: kerthcet <kerthcet@gmail.com>
2022-09-14 05:04:08 -04:00
testNodeMonitorPeriod)
controller/nodelifecycle: Make monitorNodeHealth process nodes concurrently Marking the pods not ready on a node requires looping over them and updating each pod's status one at a time. This is performed serially, and can take a while if we're processing each node serially as well. Since the time is spent waiting on io, there's an opportunity to go faster by processing multiple nodes concurrently. This change modifies the loop to process nodes in parallel, using the same number of workers as doNodeProcessingPassWorker. This change also introduces histogram metrics to better observe monitorNodeHealth.
2023-01-10 23:54:39 -05:00
nodeController.now = func() metav1.Time { return fakeNow }
nodeController.recorder = testutil.NewFakeRecorder()
nodeController.getPodsAssignedToNode = fakeGetPodsAssignedToNode(fakeNodeHandler.Clientset)
if item.workers != 0 {
nodeController.nodeUpdateWorkerSize = item.workers
}
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
controller/nodelifecycle: Make monitorNodeHealth process nodes concurrently Marking the pods not ready on a node requires looping over them and updating each pod's status one at a time. This is performed serially, and can take a while if we're processing each node serially as well. Since the time is spent waiting on io, there's an opportunity to go faster by processing multiple nodes concurrently. This change modifies the loop to process nodes in parallel, using the same number of workers as doNodeProcessingPassWorker. This change also introduces histogram metrics to better observe monitorNodeHealth.
2023-01-10 23:54:39 -05:00
t.Errorf("Case[%d] unexpected error: %v", i, err)
}
nodeController.now = func() metav1.Time { return metav1.Time{Time: fakeNow.Add(1 * time.Minute)} }
for i := range fakeNodeHandler.Existing {
fakeNodeHandler.Existing[i].Status = v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
// Node status hasn't been updated for 1hr.
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
Capacity: v1.ResourceList{
v1.ResourceName(v1.ResourceCPU): resource.MustParse("10"),
v1.ResourceName(v1.ResourceMemory): resource.MustParse("10G"),
},
}
}
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
controller/nodelifecycle: Make monitorNodeHealth process nodes concurrently Marking the pods not ready on a node requires looping over them and updating each pod's status one at a time. This is performed serially, and can take a while if we're processing each node serially as well. Since the time is spent waiting on io, there's an opportunity to go faster by processing multiple nodes concurrently. This change modifies the loop to process nodes in parallel, using the same number of workers as doNodeProcessingPassWorker. This change also introduces histogram metrics to better observe monitorNodeHealth.
2023-01-10 23:54:39 -05:00
t.Errorf("Case[%d] unexpected error: %v", i, err)
}
podStatusUpdates := 0
for _, action := range fakeNodeHandler.Actions() {
if action.GetVerb() == "update" && action.GetResource().Resource == "pods" && action.GetSubresource() == "status" {
podStatusUpdates++
}
}
const expectedPodStatusUpdates = numNodes * podsPerNode
if podStatusUpdates != expectedPodStatusUpdates {
t.Errorf("Case[%d] expect pod status updated to be %v, but got %v", i, expectedPodStatusUpdates, podStatusUpdates)
}
}
}
adding test for retrying MarkPodsNotReady
2019-10-30 09:46:30 -04:00
func TestMonitorNodeHealthMarkPodsNotReadyRetry(t *testing.T) {
type nodeIteration struct {
timeToPass time.Duration
newNodes []*v1.Node
}
timeNow := metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC)
timePlusTwoMinutes := metav1.Date(2015, 1, 1, 12, 0, 2, 0, time.UTC)
makeNodes := func(status v1.ConditionStatus, lastHeartbeatTime, lastTransitionTime metav1.Time) []*v1.Node {
return []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: timeNow,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: status,
LastHeartbeatTime: lastHeartbeatTime,
LastTransitionTime: lastTransitionTime,
},
},
},
},
}
}
table := []struct {
adding pods lister
2019-08-01 12:59:22 -04:00
desc string
fakeNodeHandler *testutil.FakeNodeHandler
updateReactor func(action testcore.Action) (bool, runtime.Object, error)
fakeGetPodsAssignedToNode func(c *fake.Clientset) func(string) ([]*v1.Pod, error)
nodeIterations []nodeIteration
expectedPodStatusUpdates int
adding test for retrying MarkPodsNotReady
2019-10-30 09:46:30 -04:00
}{
// Node created long time ago, with status updated by kubelet exceeds grace period.
// First monitorNodeHealth check will update pod status to NotReady.
// Second monitorNodeHealth check will do no updates (no retry).
{
desc: "successful pod status update, no retry required",
fakeNodeHandler: &testutil.FakeNodeHandler{
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
},
adding pods lister
2019-08-01 12:59:22 -04:00
fakeGetPodsAssignedToNode: fakeGetPodsAssignedToNode,
adding test for retrying MarkPodsNotReady
2019-10-30 09:46:30 -04:00
nodeIterations: []nodeIteration{
{
timeToPass: 0,
newNodes: makeNodes(v1.ConditionTrue, timeNow, timeNow),
},
{
timeToPass: 1 * time.Minute,
newNodes: makeNodes(v1.ConditionTrue, timeNow, timeNow),
},
{
timeToPass: 1 * time.Minute,
newNodes: makeNodes(v1.ConditionFalse, timePlusTwoMinutes, timePlusTwoMinutes),
},
},
expectedPodStatusUpdates: 1,
},
// Node created long time ago, with status updated by kubelet exceeds grace period.
// First monitorNodeHealth check will fail to update pod status to NotReady.
// Second monitorNodeHealth check will update pod status to NotReady (retry).
{
desc: "unsuccessful pod status update, retry required",
fakeNodeHandler: &testutil.FakeNodeHandler{
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
},
updateReactor: func() func(action testcore.Action) (bool, runtime.Object, error) {
i := 0
return func(action testcore.Action) (bool, runtime.Object, error) {
if action.GetVerb() == "update" && action.GetResource().Resource == "pods" && action.GetSubresource() == "status" {
i++
switch i {
case 1:
return true, nil, fmt.Errorf("fake error")
default:
return true, testutil.NewPod("pod0", "node0"), nil
}
}
return true, nil, fmt.Errorf("unsupported action")
}
}(),
adding pods lister
2019-08-01 12:59:22 -04:00
fakeGetPodsAssignedToNode: fakeGetPodsAssignedToNode,
adding test for retrying MarkPodsNotReady
2019-10-30 09:46:30 -04:00
nodeIterations: []nodeIteration{
{
timeToPass: 0,
newNodes: makeNodes(v1.ConditionTrue, timeNow, timeNow),
},
{
timeToPass: 1 * time.Minute,
newNodes: makeNodes(v1.ConditionTrue, timeNow, timeNow),
},
{
timeToPass: 1 * time.Minute,
newNodes: makeNodes(v1.ConditionFalse, timePlusTwoMinutes, timePlusTwoMinutes),
},
},
expectedPodStatusUpdates: 2, // One failed and one retry.
},
adding pods lister
2019-08-01 12:59:22 -04:00
// Node created long time ago, with status updated by kubelet exceeds grace period.
// First monitorNodeHealth check will fail to list pods.
// Second monitorNodeHealth check will update pod status to NotReady (retry).
{
desc: "unsuccessful pod list, retry required",
fakeNodeHandler: &testutil.FakeNodeHandler{
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
},
fakeGetPodsAssignedToNode: func(c *fake.Clientset) func(string) ([]*v1.Pod, error) {
i := 0
f := fakeGetPodsAssignedToNode(c)
return func(nodeName string) ([]*v1.Pod, error) {
i++
if i == 1 {
return nil, fmt.Errorf("fake error")
}
return f(nodeName)
}
},
nodeIterations: []nodeIteration{
{
timeToPass: 0,
newNodes: makeNodes(v1.ConditionTrue, timeNow, timeNow),
},
{
timeToPass: 1 * time.Minute,
newNodes: makeNodes(v1.ConditionTrue, timeNow, timeNow),
},
{
timeToPass: 1 * time.Minute,
newNodes: makeNodes(v1.ConditionFalse, timePlusTwoMinutes, timePlusTwoMinutes),
},
},
expectedPodStatusUpdates: 1,
},
adding test for retrying MarkPodsNotReady
2019-10-30 09:46:30 -04:00
}
for _, item := range table {
t.Run(item.desc, func(t *testing.T) {
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx := ktesting.Init(t)
adding test for retrying MarkPodsNotReady
2019-10-30 09:46:30 -04:00
nodeController, _ := newNodeLifecycleControllerFromClient(
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx,
adding test for retrying MarkPodsNotReady
2019-10-30 09:46:30 -04:00
item.fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
testNodeMonitorPeriod,
Remove CLI flag enable-taint-manager Signed-off-by: kerthcet <kerthcet@gmail.com>
2022-09-14 05:04:08 -04:00
)
adding test for retrying MarkPodsNotReady
2019-10-30 09:46:30 -04:00
if item.updateReactor != nil {
item.fakeNodeHandler.Clientset.PrependReactor("update", "pods", item.updateReactor)
}
nodeController.now = func() metav1.Time { return timeNow }
nodeController.recorder = testutil.NewFakeRecorder()
adding pods lister
2019-08-01 12:59:22 -04:00
nodeController.getPodsAssignedToNode = item.fakeGetPodsAssignedToNode(item.fakeNodeHandler.Clientset)
adding test for retrying MarkPodsNotReady
2019-10-30 09:46:30 -04:00
for _, itertion := range item.nodeIterations {
nodeController.now = func() metav1.Time { return metav1.Time{Time: timeNow.Add(itertion.timeToPass)} }
item.fakeNodeHandler.Existing = itertion.newNodes
if err := nodeController.syncNodeStore(item.fakeNodeHandler); err != nil {
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
adding test for retrying MarkPodsNotReady
2019-10-30 09:46:30 -04:00
t.Errorf("unexpected error: %v", err)
}
}
podStatusUpdates := 0
for _, action := range item.fakeNodeHandler.Actions() {
if action.GetVerb() == "update" && action.GetResource().Resource == "pods" && action.GetSubresource() == "status" {
podStatusUpdates++
}
}
if podStatusUpdates != item.expectedPodStatusUpdates {
t.Errorf("expect pod status updated to happen %d times, but got %d", item.expectedPodStatusUpdates, podStatusUpdates)
}
})
}
}
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
// TestApplyNoExecuteTaints, ensures we just have a NoExecute taint applied to node.
// NodeController is just responsible for enqueuing the node to tainting queue from which taint manager picks up
// and evicts the pods on the node.
func TestApplyNoExecuteTaints(t *testing.T) {
unit tests: Skip flaky tests on Windows Some of the unit tests are currently flaky on Windows. This commit skips them until they are resolved.
2023-02-21 07:48:02 -05:00
// TODO: Remove skip once https://github.com/kubernetes/kubernetes/pull/114607 merges.
if goruntime.GOOS == "windows" {
t.Skip("Skipping test on Windows.")
}
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
fakeNow := metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC)
fakeNodeHandler := &testutil.FakeNodeHandler{
Existing: []*v1.Node{
// Unreachable Taint with effect 'NoExecute' should be applied to this node.
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
Use topology labels instead of old beta names (#96033) * Rename const for topology.../zone * Rename const for topology.../region * Rename const for failure-domain.../zone * Rename const for failure-domain.../region * Restore old names for compat
2020-11-05 23:26:50 -05:00
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
// Because of the logic that prevents NC from evicting anything when all Nodes are NotReady
// we need second healthy node in tests.
{
ObjectMeta: metav1.ObjectMeta{
Name: "node1",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
Use topology labels instead of old beta names (#96033) * Rename const for topology.../zone * Rename const for topology.../region * Rename const for failure-domain.../zone * Rename const for failure-domain.../region * Restore old names for compat
2020-11-05 23:26:50 -05:00
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
// NotReady Taint with NoExecute effect should be applied to this node.
{
ObjectMeta: metav1.ObjectMeta{
Name: "node2",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
Use topology labels instead of old beta names (#96033) * Rename const for topology.../zone * Rename const for topology.../region * Rename const for failure-domain.../zone * Rename const for failure-domain.../region * Restore old names for compat
2020-11-05 23:26:50 -05:00
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionFalse,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
fix node lifecycle controller panic when conditionType ready is nil
2024-01-19 09:07:55 -05:00
// NotReady Taint with NoExecute effect should not be applied to a node if the NodeCondition Type NodeReady has been set to nil in the interval between the NodeController enqueuing the node and the taint manager picking up.
{
ObjectMeta: metav1.ObjectMeta{
Name: "node3",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2016, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2016, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
}
healthyNodeNewStatus := v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2017, 1, 1, 12, 10, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
}
fix node lifecycle controller panic when conditionType ready is nil
2024-01-19 09:07:55 -05:00
unhealthyNodeNewStatus := v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionFalse,
LastHeartbeatTime: metav1.Date(2017, 1, 1, 12, 10, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
}
overrideNodeNewStatusConditions := []v1.NodeCondition{
{
Type: "MemoryPressure",
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2017, 1, 1, 12, 10, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC),
},
}
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
originalTaint := UnreachableTaintTemplate
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx := ktesting.Init(t)
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
nodeController, _ := newNodeLifecycleControllerFromClient(
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx,
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
testNodeMonitorPeriod,
Remove CLI flag enable-taint-manager Signed-off-by: kerthcet <kerthcet@gmail.com>
2022-09-14 05:04:08 -04:00
)
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
nodeController.now = func() metav1.Time { return fakeNow }
nodeController.recorder = testutil.NewFakeRecorder()
adding fakeGetPodsAssignedToNode
2019-09-27 11:28:58 -04:00
nodeController.getPodsAssignedToNode = fakeGetPodsAssignedToNode(fakeNodeHandler.Clientset)
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
nodeController.doNoExecuteTaintingPass(tCtx)
node0, err := fakeNodeHandler.Get(tCtx, "node0", metav1.GetOptions{})
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
if err != nil {
t.Errorf("Can't get current node0...")
return
}
if !taintutils.TaintExists(node0.Spec.Taints, UnreachableTaintTemplate) {
t.Errorf("Can't find taint %v in %v", originalTaint, node0.Spec.Taints)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
node2, err := fakeNodeHandler.Get(tCtx, "node2", metav1.GetOptions{})
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
if err != nil {
t.Errorf("Can't get current node2...")
return
}
if !taintutils.TaintExists(node2.Spec.Taints, NotReadyTaintTemplate) {
t.Errorf("Can't find taint %v in %v", NotReadyTaintTemplate, node2.Spec.Taints)
}
// Make node3 healthy again.
node2.Status = healthyNodeNewStatus
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
_, err = fakeNodeHandler.UpdateStatus(tCtx, node2, metav1.UpdateOptions{})
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
if err != nil {
Fix Go vet errors for master golang Co-authored-by: Rajalakshmi-Girish <rajalakshmi.girish1@ibm.com> Co-authored-by: Abhishek Kr Srivastav <Abhishek.kr.srivastav@ibm.com>
2024-09-12 08:45:22 -04:00
t.Error(err.Error())
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
return
}
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
nodeController.doNoExecuteTaintingPass(tCtx)
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
node2, err = fakeNodeHandler.Get(tCtx, "node2", metav1.GetOptions{})
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
if err != nil {
t.Errorf("Can't get current node2...")
return
}
// We should not see any taint on the node(especially the Not-Ready taint with NoExecute effect).
if taintutils.TaintExists(node2.Spec.Taints, NotReadyTaintTemplate) || len(node2.Spec.Taints) > 0 {
t.Errorf("Found taint %v in %v, which should not be present", NotReadyTaintTemplate, node2.Spec.Taints)
}
fix node lifecycle controller panic when conditionType ready is nil
2024-01-19 09:07:55 -05:00
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
node3, err := fakeNodeHandler.Get(tCtx, "node3", metav1.GetOptions{})
fix node lifecycle controller panic when conditionType ready is nil
2024-01-19 09:07:55 -05:00
if err != nil {
t.Errorf("Can't get current node3...")
return
}
node3.Status = unhealthyNodeNewStatus
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
_, err = fakeNodeHandler.UpdateStatus(tCtx, node3, metav1.UpdateOptions{})
fix node lifecycle controller panic when conditionType ready is nil
2024-01-19 09:07:55 -05:00
if err != nil {
Fix Go vet errors for master golang Co-authored-by: Rajalakshmi-Girish <rajalakshmi.girish1@ibm.com> Co-authored-by: Abhishek Kr Srivastav <Abhishek.kr.srivastav@ibm.com>
2024-09-12 08:45:22 -04:00
t.Error(err.Error())
fix node lifecycle controller panic when conditionType ready is nil
2024-01-19 09:07:55 -05:00
return
}
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
fix node lifecycle controller panic when conditionType ready is nil
2024-01-19 09:07:55 -05:00
t.Errorf("unexpected error: %v", err)
}
// Before taint manager work, the status has been replaced(maybe merge-patch replace).
node3.Status.Conditions = overrideNodeNewStatusConditions
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
_, err = fakeNodeHandler.UpdateStatus(tCtx, node3, metav1.UpdateOptions{})
fix node lifecycle controller panic when conditionType ready is nil
2024-01-19 09:07:55 -05:00
if err != nil {
Fix Go vet errors for master golang Co-authored-by: Rajalakshmi-Girish <rajalakshmi.girish1@ibm.com> Co-authored-by: Abhishek Kr Srivastav <Abhishek.kr.srivastav@ibm.com>
2024-09-12 08:45:22 -04:00
t.Error(err.Error())
fix node lifecycle controller panic when conditionType ready is nil
2024-01-19 09:07:55 -05:00
return
}
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
nodeController.doNoExecuteTaintingPass(tCtx)
node3, err = fakeNodeHandler.Get(tCtx, "node3", metav1.GetOptions{})
fix node lifecycle controller panic when conditionType ready is nil
2024-01-19 09:07:55 -05:00
if err != nil {
t.Errorf("Can't get current node3...")
return
}
// We should not see any taint on the node(especially the Not-Ready taint with NoExecute effect).
if taintutils.TaintExists(node3.Spec.Taints, NotReadyTaintTemplate) || len(node3.Spec.Taints) > 0 {
t.Errorf("Found taint %v in %v, which should not be present", NotReadyTaintTemplate, node3.Spec.Taints)
}
Add test cases for taintbasedevictions
2018-10-14 20:09:42 -04:00
}
NoExecute taint should be added when a Node's ready condition becomes Unknown After a Node has stopped posting heartbeats for nodeMonitorGracePeriod, it will be considered unreachable, its ready condition will be set to Unknown, NoSchedule taint will be added, all Pods on it will be set to NotReady, but there is always a delay of 5s before NoExecute taint is added to the Node, adding 5s to the recovery time of Pods which are supposed to be evicted by the taint and recreated on other Nodes sooner. The delay is because processTaintBaseEviction() uses the last observed ready condition of the Node instead of the current one to determine whether it should add the Node to the taint queue. When a Node is set to unreachable due to missing heartbeats, the last observed ready condition is still true and the current ready condition is unknown, we should use the latter for processTaintBaseEviction(). Signed-off-by: Quan Tian <qtian@vmware.com>
2023-09-21 11:32:51 -04:00
// TestApplyNoExecuteTaintsToUnreachableNode ensures a NoExecute taint is applied to node that hasn't posted the
// node status for a period greater than nodeMonitorGracePeriod.
func TestApplyNoExecuteTaintsToUnreachableNode(t *testing.T) {
// TODO: Remove skip once https://github.com/kubernetes/kubernetes/pull/114607 merges.
if goruntime.GOOS == "windows" {
t.Skip("Skipping test on Windows.")
}
time1 := metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC)
// time2 is set to NodeMonitorGracePeriod plus 1 second after time1.
time2 := metav1.Time{Time: time1.Add(testNodeMonitorGracePeriod + time.Second)}
fakeNodeHandler := &testutil.FakeNodeHandler{
Existing: []*v1.Node{
// Unreachable Taint with effect 'NoExecute' should be applied to this node after 2015-01-01 12:00:40.
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: time1,
LastTransitionTime: time1,
},
},
},
},
// Because of the logic that prevents NC from evicting anything when all Nodes are NotReady
// we need second healthy node in tests.
{
ObjectMeta: metav1.ObjectMeta{
Name: "node1",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: time1,
LastTransitionTime: time1,
},
},
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
}
_, ctx := ktesting.NewTestContext(t)
nodeController, _ := newNodeLifecycleControllerFromClient(
ctx,
fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
testNodeMonitorPeriod,
)
// No taint should be added when the NodeReady conditions were not set longer ago than NodeMonitorGracePeriod.
nodeController.now = func() metav1.Time { return time1 }
nodeController.recorder = testutil.NewFakeRecorder()
nodeController.getPodsAssignedToNode = fakeGetPodsAssignedToNode(fakeNodeHandler.Clientset)
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
t.Errorf("unexpected error: %v", err)
}
if err := nodeController.monitorNodeHealth(ctx); err != nil {
t.Errorf("unexpected error: %v", err)
}
nodeController.doNoExecuteTaintingPass(ctx)
node0, err := fakeNodeHandler.Get(ctx, "node0", metav1.GetOptions{})
if err != nil {
t.Errorf("Can't get current node0...")
return
}
if len(node0.Spec.Taints) > 0 {
t.Errorf("Got unexpected taints %v", node0.Spec.Taints)
}
node1, err := fakeNodeHandler.Get(ctx, "node1", metav1.GetOptions{})
if err != nil {
t.Errorf("Can't get current node1...")
return
}
if len(node1.Spec.Taints) > 0 {
t.Errorf("Got unexpected taints %v", node1.Spec.Taints)
}
// Advance the clock to time2 to expire the previous NodeReady condition.
nodeController.now = func() metav1.Time { return time2 }
// Only node1 sends a new heartbeat at time2.
node1.Status = v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: time2,
LastTransitionTime: time1,
},
},
}
_, err = fakeNodeHandler.UpdateStatus(ctx, node1, metav1.UpdateOptions{})
if err != nil {
t.Errorf("unexpected error: %v", err)
return
}
// Unreachable taint should be added to node0 and not node1 because the former's NodeReadyCondition was set longer ago than NodeMonitorGracePeriod.
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
t.Errorf("unexpected error: %v", err)
}
if err := nodeController.monitorNodeHealth(ctx); err != nil {
t.Errorf("unexpected error: %v", err)
}
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
t.Errorf("unexpected error: %v", err)
}
nodeController.doNoExecuteTaintingPass(ctx)
node0, err = fakeNodeHandler.Get(ctx, "node0", metav1.GetOptions{})
if err != nil {
t.Errorf("Can't get current node0...")
return
}
if !taintutils.TaintExists(node0.Spec.Taints, UnreachableTaintTemplate) {
t.Errorf("Can't find taint %v in %v", UnreachableTaintTemplate, node0.Spec.Taints)
}
node1, err = fakeNodeHandler.Get(ctx, "node1", metav1.GetOptions{})
if err != nil {
t.Errorf("Can't get current node1...")
return
}
if len(node1.Spec.Taints) > 0 {
t.Errorf("Got unexpected taints %v", node1.Spec.Taints)
}
// Unreachable taint should be deleted from node0 after it sends a new heartbeat.
node0.Status = v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: time2,
LastTransitionTime: time2,
},
},
}
_, err = fakeNodeHandler.UpdateStatus(ctx, node0, metav1.UpdateOptions{})
if err != nil {
t.Errorf("unexpected error: %v", err)
return
}
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
t.Errorf("unexpected error: %v", err)
}
if err := nodeController.monitorNodeHealth(ctx); err != nil {
t.Errorf("unexpected error: %v", err)
}
nodeController.doNoExecuteTaintingPass(ctx)
node0, err = fakeNodeHandler.Get(ctx, "node0", metav1.GetOptions{})
if err != nil {
t.Errorf("Can't get current node0...")
return
}
// We should not see any taint on the node.
if len(node0.Spec.Taints) > 0 {
t.Errorf("Got unexpected taints %v", node0.Spec.Taints)
}
}
fix nodelifecyle controller not add NoExecute taint bug
2020-11-25 20:55:43 -05:00
// TestApplyNoExecuteTaintsToNodesEnqueueTwice ensures we taint every node with NoExecute even if enqueued twice
func TestApplyNoExecuteTaintsToNodesEnqueueTwice(t *testing.T) {
unit tests: Skip flaky tests on Windows (part 2) Some of the unit tests are currently flaky on Windows. This commit skips them until they are resolved.
2023-03-09 18:58:53 -05:00
// TODO: Remove skip once https://github.com/kubernetes/kubernetes/pull/114607 merges.
if goruntime.GOOS == "windows" {
t.Skip("Skipping test on Windows.")
}
fix nodelifecyle controller not add NoExecute taint bug
2020-11-25 20:55:43 -05:00
fakeNow := metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC)
fakeNodeHandler := &testutil.FakeNodeHandler{
Existing: []*v1.Node{
// Unreachable Taint with effect 'NoExecute' should be applied to this node.
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
// Because of the logic that prevents NC from evicting anything when all Nodes are NotReady
// we need second healthy node in tests.
{
ObjectMeta: metav1.ObjectMeta{
Name: "node1",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
// NotReady Taint with NoExecute effect should be applied to this node.
{
ObjectMeta: metav1.ObjectMeta{
Name: "node2",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionFalse,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
}
healthyNodeNewStatus := v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2017, 1, 1, 12, 10, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx := ktesting.Init(t)
fix nodelifecyle controller not add NoExecute taint bug
2020-11-25 20:55:43 -05:00
nodeController, _ := newNodeLifecycleControllerFromClient(
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx,
fix nodelifecyle controller not add NoExecute taint bug
2020-11-25 20:55:43 -05:00
fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
testNodeMonitorPeriod,
Remove CLI flag enable-taint-manager Signed-off-by: kerthcet <kerthcet@gmail.com>
2022-09-14 05:04:08 -04:00
)
fix nodelifecyle controller not add NoExecute taint bug
2020-11-25 20:55:43 -05:00
nodeController.now = func() metav1.Time { return fakeNow }
nodeController.recorder = testutil.NewFakeRecorder()
nodeController.getPodsAssignedToNode = fakeGetPodsAssignedToNode(fakeNodeHandler.Clientset)
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
t.Errorf("unexpected error: %v", err)
}
// 1. monitor node health twice, add untainted node once
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
fix nodelifecyle controller not add NoExecute taint bug
2020-11-25 20:55:43 -05:00
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
fix nodelifecyle controller not add NoExecute taint bug
2020-11-25 20:55:43 -05:00
t.Errorf("unexpected error: %v", err)
}
// 2. mark node0 healthy
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
node0, err := fakeNodeHandler.Get(tCtx, "node0", metav1.GetOptions{})
fix nodelifecyle controller not add NoExecute taint bug
2020-11-25 20:55:43 -05:00
if err != nil {
t.Errorf("Can't get current node0...")
return
}
node0.Status = healthyNodeNewStatus
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
_, err = fakeNodeHandler.UpdateStatus(tCtx, node0, metav1.UpdateOptions{})
fix nodelifecyle controller not add NoExecute taint bug
2020-11-25 20:55:43 -05:00
if err != nil {
Fix Go vet errors for master golang Co-authored-by: Rajalakshmi-Girish <rajalakshmi.girish1@ibm.com> Co-authored-by: Abhishek Kr Srivastav <Abhishek.kr.srivastav@ibm.com>
2024-09-12 08:45:22 -04:00
t.Error(err.Error())
fix nodelifecyle controller not add NoExecute taint bug
2020-11-25 20:55:43 -05:00
return
}
// add other notReady nodes
fakeNodeHandler.Existing = append(fakeNodeHandler.Existing, []*v1.Node{
// Unreachable Taint with effect 'NoExecute' should be applied to this node.
{
ObjectMeta: metav1.ObjectMeta{
Name: "node3",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
// Because of the logic that prevents NC from evicting anything when all Nodes are NotReady
// we need second healthy node in tests.
{
ObjectMeta: metav1.ObjectMeta{
Name: "node4",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
// NotReady Taint with NoExecute effect should be applied to this node.
{
ObjectMeta: metav1.ObjectMeta{
Name: "node5",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionFalse,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
}...)
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
t.Errorf("unexpected error: %v", err)
}
// 3. start monitor node health again, add untainted node twice, construct UniqueQueue with duplicated node cache
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
fix nodelifecyle controller not add NoExecute taint bug
2020-11-25 20:55:43 -05:00
t.Errorf("unexpected error: %v", err)
}
// 4. do NoExecute taint pass
// when processing with node0, condition.Status is NodeReady, and return true with default case
// then remove the set value and queue value both, the taint job never stuck
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
nodeController.doNoExecuteTaintingPass(tCtx)
fix nodelifecyle controller not add NoExecute taint bug
2020-11-25 20:55:43 -05:00
// 5. get node3 and node5, see if it has ready got NoExecute taint
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
node3, err := fakeNodeHandler.Get(tCtx, "node3", metav1.GetOptions{})
fix nodelifecyle controller not add NoExecute taint bug
2020-11-25 20:55:43 -05:00
if err != nil {
t.Errorf("Can't get current node3...")
return
}
if !taintutils.TaintExists(node3.Spec.Taints, UnreachableTaintTemplate) || len(node3.Spec.Taints) == 0 {
t.Errorf("Not found taint %v in %v, which should be present in %s", UnreachableTaintTemplate, node3.Spec.Taints, node3.Name)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
node5, err := fakeNodeHandler.Get(tCtx, "node5", metav1.GetOptions{})
fix nodelifecyle controller not add NoExecute taint bug
2020-11-25 20:55:43 -05:00
if err != nil {
t.Errorf("Can't get current node5...")
return
}
if !taintutils.TaintExists(node5.Spec.Taints, NotReadyTaintTemplate) || len(node5.Spec.Taints) == 0 {
t.Errorf("Not found taint %v in %v, which should be present in %s", NotReadyTaintTemplate, node5.Spec.Taints, node5.Name)
}
}
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
func TestSwapUnreachableNotReadyTaints(t *testing.T) {
fakeNow := metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC)
fakeNodeHandler := &testutil.FakeNodeHandler{
Existing: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
Use topology labels instead of old beta names (#96033) * Rename const for topology.../zone * Rename const for topology.../region * Rename const for failure-domain.../zone * Rename const for failure-domain.../region * Restore old names for compat
2020-11-05 23:26:50 -05:00
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
// Because of the logic that prevents NC from evicting anything when all Nodes are NotReady
// we need second healthy node in tests. Because of how the tests are written we need to update
// the status of this Node.
{
ObjectMeta: metav1.ObjectMeta{
Name: "node1",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
Use topology labels instead of old beta names (#96033) * Rename const for topology.../zone * Rename const for topology.../region * Rename const for failure-domain.../zone * Rename const for failure-domain.../region * Restore old names for compat
2020-11-05 23:26:50 -05:00
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
}
newNodeStatus := v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionFalse,
// Node status has just been updated, and is NotReady for 10min.
LastHeartbeatTime: metav1.Date(2017, 1, 1, 12, 9, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
}
healthyNodeNewStatus := v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2017, 1, 1, 12, 10, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
}
originalTaint := UnreachableTaintTemplate
updatedTaint := NotReadyTaintTemplate
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx := ktesting.Init(t)
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
nodeController, _ := newNodeLifecycleControllerFromClient(
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx,
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
testNodeMonitorPeriod,
Remove CLI flag enable-taint-manager Signed-off-by: kerthcet <kerthcet@gmail.com>
2022-09-14 05:04:08 -04:00
)
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
nodeController.now = func() metav1.Time { return fakeNow }
nodeController.recorder = testutil.NewFakeRecorder()
adding fakeGetPodsAssignedToNode
2019-09-27 11:28:58 -04:00
nodeController.getPodsAssignedToNode = fakeGetPodsAssignedToNode(fakeNodeHandler.Clientset)
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
nodeController.doNoExecuteTaintingPass(tCtx)
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
node0, err := fakeNodeHandler.Get(tCtx, "node0", metav1.GetOptions{})
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
if err != nil {
t.Errorf("Can't get current node0...")
return
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
node1, err := fakeNodeHandler.Get(tCtx, "node1", metav1.GetOptions{})
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
if err != nil {
t.Errorf("Can't get current node1...")
return
}
Refactoring taints to reduce sprawl
2017-07-06 09:13:13 -04:00
if originalTaint != nil && !taintutils.TaintExists(node0.Spec.Taints, originalTaint) {
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
t.Errorf("Can't find taint %v in %v", originalTaint, node0.Spec.Taints)
}
Remove deprecated pod-eviction-timeout flag from controller-manager
2023-03-07 09:27:14 -05:00
nodeController.now = func() metav1.Time { return metav1.Time{Time: fakeNow.Time} }
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
node0.Status = newNodeStatus
node1.Status = healthyNodeNewStatus
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
_, err = fakeNodeHandler.UpdateStatus(tCtx, node0, metav1.UpdateOptions{})
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
if err != nil {
Fix Go vet errors for master golang Co-authored-by: Rajalakshmi-Girish <rajalakshmi.girish1@ibm.com> Co-authored-by: Abhishek Kr Srivastav <Abhishek.kr.srivastav@ibm.com>
2024-09-12 08:45:22 -04:00
t.Error(err.Error())
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
return
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
_, err = fakeNodeHandler.UpdateStatus(tCtx, node1, metav1.UpdateOptions{})
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
if err != nil {
Fix Go vet errors for master golang Co-authored-by: Rajalakshmi-Girish <rajalakshmi.girish1@ibm.com> Co-authored-by: Abhishek Kr Srivastav <Abhishek.kr.srivastav@ibm.com>
2024-09-12 08:45:22 -04:00
t.Error(err.Error())
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
return
}
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
nodeController.doNoExecuteTaintingPass(tCtx)
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
node0, err = fakeNodeHandler.Get(tCtx, "node0", metav1.GetOptions{})
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
if err != nil {
t.Errorf("Can't get current node0...")
return
}
if updatedTaint != nil {
Refactoring taints to reduce sprawl
2017-07-06 09:13:13 -04:00
if !taintutils.TaintExists(node0.Spec.Taints, updatedTaint) {
Allow swapping NotReady and Unschedulable Taints
2017-04-04 09:35:44 -04:00
t.Errorf("Can't find taint %v in %v", updatedTaint, node0.Spec.Taints)
}
}
}
Tainted node by condition.
2017-07-19 11:51:19 -04:00
func TestTaintsNodeByCondition(t *testing.T) {
fakeNow := metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC)
fakeNodeHandler := &testutil.FakeNodeHandler{
Existing: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
Use topology labels instead of old beta names (#96033) * Rename const for topology.../zone * Rename const for topology.../region * Rename const for failure-domain.../zone * Rename const for failure-domain.../region * Restore old names for compat
2020-11-05 23:26:50 -05:00
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
Tainted node by condition.
2017-07-19 11:51:19 -04:00
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx := ktesting.Init(t)
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
nodeController, _ := newNodeLifecycleControllerFromClient(
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx,
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
testNodeMonitorPeriod,
Remove CLI flag enable-taint-manager Signed-off-by: kerthcet <kerthcet@gmail.com>
2022-09-14 05:04:08 -04:00
)
Tainted node by condition.
2017-07-19 11:51:19 -04:00
nodeController.now = func() metav1.Time { return fakeNow }
nodeController.recorder = testutil.NewFakeRecorder()
adding fakeGetPodsAssignedToNode
2019-09-27 11:28:58 -04:00
nodeController.getPodsAssignedToNode = fakeGetPodsAssignedToNode(fakeNodeHandler.Clientset)
Tainted node by condition.
2017-07-19 11:51:19 -04:00
networkUnavailableTaint := &v1.Taint{
Move out const strings in pkg/scheduler/api/well_known_labels.go
2019-11-04 14:31:16 -05:00
Key: v1.TaintNodeNetworkUnavailable,
Tainted node by condition.
2017-07-19 11:51:19 -04:00
Effect: v1.TaintEffectNoSchedule,
}
simplify code and add unit test for NotReady taint
2018-05-14 02:55:42 -04:00
notReadyTaint := &v1.Taint{
Move out const strings in pkg/scheduler/api/well_known_labels.go
2019-11-04 14:31:16 -05:00
Key: v1.TaintNodeNotReady,
simplify code and add unit test for NotReady taint
2018-05-14 02:55:42 -04:00
Effect: v1.TaintEffectNoSchedule,
}
fix an issue that scheduling doesn't respect NodeLost status of a node - if Node is in UnknowStatus, apply unreachable taint with NoSchedule effect - some internal data structure refactoring - update unit test
2018-08-22 18:26:46 -04:00
unreachableTaint := &v1.Taint{
Move out const strings in pkg/scheduler/api/well_known_labels.go
2019-11-04 14:31:16 -05:00
Key: v1.TaintNodeUnreachable,
fix an issue that scheduling doesn't respect NodeLost status of a node - if Node is in UnknowStatus, apply unreachable taint with NoSchedule effect - some internal data structure refactoring - update unit test
2018-08-22 18:26:46 -04:00
Effect: v1.TaintEffectNoSchedule,
}
Tainted node by condition.
2017-07-19 11:51:19 -04:00
tests := []struct {
Name string
Node *v1.Node
ExpectedTaints []*v1.Taint
}{
{
Name: "NetworkUnavailable is true",
Node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
Use topology labels instead of old beta names (#96033) * Rename const for topology.../zone * Rename const for topology.../region * Rename const for failure-domain.../zone * Rename const for failure-domain.../region * Restore old names for compat
2020-11-05 23:26:50 -05:00
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
Tainted node by condition.
2017-07-19 11:51:19 -04:00
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
{
Type: v1.NodeNetworkUnavailable,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
ExpectedTaints: []*v1.Taint{networkUnavailableTaint},
},
{
cleanup logic related with OutOfDisk - cleanup OOD logic in scheduling and node controller - update comments and testcases
2018-12-13 03:31:46 -05:00
Name: "NetworkUnavailable is true",
Tainted node by condition.
2017-07-19 11:51:19 -04:00
Node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
Use topology labels instead of old beta names (#96033) * Rename const for topology.../zone * Rename const for topology.../region * Rename const for failure-domain.../zone * Rename const for failure-domain.../region * Restore old names for compat
2020-11-05 23:26:50 -05:00
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
Tainted node by condition.
2017-07-19 11:51:19 -04:00
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
{
Type: v1.NodeNetworkUnavailable,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
ExpectedTaints: []*v1.Taint{networkUnavailableTaint},
},
simplify code and add unit test for NotReady taint
2018-05-14 02:55:42 -04:00
{
Name: "Ready is false",
Node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
Use topology labels instead of old beta names (#96033) * Rename const for topology.../zone * Rename const for topology.../region * Rename const for failure-domain.../zone * Rename const for failure-domain.../region * Restore old names for compat
2020-11-05 23:26:50 -05:00
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
simplify code and add unit test for NotReady taint
2018-05-14 02:55:42 -04:00
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionFalse,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
ExpectedTaints: []*v1.Taint{notReadyTaint},
},
fix an issue that scheduling doesn't respect NodeLost status of a node - if Node is in UnknowStatus, apply unreachable taint with NoSchedule effect - some internal data structure refactoring - update unit test
2018-08-22 18:26:46 -04:00
{
Name: "Ready is unknown",
Node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
Use topology labels instead of old beta names (#96033) * Rename const for topology.../zone * Rename const for topology.../region * Rename const for failure-domain.../zone * Rename const for failure-domain.../region * Restore old names for compat
2020-11-05 23:26:50 -05:00
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
fix an issue that scheduling doesn't respect NodeLost status of a node - if Node is in UnknowStatus, apply unreachable taint with NoSchedule effect - some internal data structure refactoring - update unit test
2018-08-22 18:26:46 -04:00
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
ExpectedTaints: []*v1.Taint{unreachableTaint},
},
Tainted node by condition.
2017-07-19 11:51:19 -04:00
}
for _, test := range tests {
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if _, err := fakeNodeHandler.Update(tCtx, test.Node, metav1.UpdateOptions{}); err != nil {
t.Errorf("unexpected error: %v", err)
}
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
Tainted node by condition.
2017-07-19 11:51:19 -04:00
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.doNoScheduleTaintingPass(tCtx, test.Node.Name); err != nil {
t.Errorf("unexpected error: %v", err)
}
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
Tainted node by condition.
2017-07-19 11:51:19 -04:00
t.Errorf("unexpected error: %v", err)
}
node0, err := nodeController.nodeLister.Get("node0")
if err != nil {
t.Errorf("Can't get current node0...")
return
}
if len(node0.Spec.Taints) != len(test.ExpectedTaints) {
t.Errorf("%s: Unexpected number of taints: expected %d, got %d",
test.Name, len(test.ExpectedTaints), len(node0.Spec.Taints))
}
for _, taint := range test.ExpectedTaints {
if !taintutils.TaintExists(node0.Spec.Taints, taint) {
t.Errorf("%s: Can't find taint %v in %v", test.Name, taint, node0.Spec.Taints)
}
}
}
}
fix node controller event uid issue
2016-08-13 21:41:20 -04:00
func TestNodeEventGeneration(t *testing.T) {
refactor: generated
2016-12-03 13:57:26 -05:00
fakeNow := metav1.Date(2016, 9, 10, 12, 0, 0, 0, time.UTC)
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
fakeNodeHandler := &testutil.FakeNodeHandler{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Existing: []*v1.Node{
fix node controller event uid issue
2016-08-13 21:41:20 -04:00
{
refactor: use metav1.ObjectMeta in other types
2017-01-16 22:38:19 -05:00
ObjectMeta: metav1.ObjectMeta{
fix node controller event uid issue
2016-08-13 21:41:20 -04:00
Name: "node0",
UID: "1234567890",
refactor: generated
2016-12-03 13:57:26 -05:00
CreationTimestamp: metav1.Date(2015, 8, 10, 0, 0, 0, 0, time.UTC),
fix node controller event uid issue
2016-08-13 21:41:20 -04:00
},
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
fix node controller event uid issue
2016-08-13 21:41:20 -04:00
{
cmd/kube-controller-manager
2016-11-18 15:50:17 -05:00
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
refactor: generated
2016-12-03 13:57:26 -05:00
LastHeartbeatTime: metav1.Date(2015, 8, 10, 0, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 8, 10, 0, 0, 0, 0, time.UTC),
fix node controller event uid issue
2016-08-13 21:41:20 -04:00
},
},
},
},
},
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
fix node controller event uid issue
2016-08-13 21:41:20 -04:00
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx := ktesting.Init(t)
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
nodeController, _ := newNodeLifecycleControllerFromClient(
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx,
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
testNodeMonitorPeriod,
Remove CLI flag enable-taint-manager Signed-off-by: kerthcet <kerthcet@gmail.com>
2022-09-14 05:04:08 -04:00
)
refactor: generated
2016-12-03 13:57:26 -05:00
nodeController.now = func() metav1.Time { return fakeNow }
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
fakeRecorder := testutil.NewFakeRecorder()
fix node controller event uid issue
2016-08-13 21:41:20 -04:00
nodeController.recorder = fakeRecorder
adding fakeGetPodsAssignedToNode
2019-09-27 11:28:58 -04:00
nodeController.getPodsAssignedToNode = fakeGetPodsAssignedToNode(fakeNodeHandler.Clientset)
consolidate node deletion logic between node lifecycle and cloud node controller
2018-10-28 21:57:23 -04:00
Split the NodeController into lifecycle and ipam pieces. Prepatory work fpr removing cloud provider dependency from node controller running in Kube Controller Manager. Splitting the node controller into its two major pieces life-cycle and CIDR/IP management. Both pieces currently need the the cloud system to do their work. Removing lifecycles dependency on cloud will be fixed ina followup PR. Moved node scheduler code to live with node lifecycle controller. Got the IPAM/Lifecycle split completed. Still need to rename pieces. Made changes to the utils and tests so they would be in the appropriate package. Moved the node based ipam code to nodeipam. Made the relevant tests pass. Moved common node controller util code to nodeutil. Removed unneeded pod informer sync from node ipam controller. Fixed linter issues. Factored in feedback from @gmarek. Factored in feedback from @mtaufen. Undoing unneeded change.
2017-10-11 19:36:39 -04:00
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
NodeController listing nodes from cache instead of cache in apiserver
2016-12-19 05:15:39 -05:00
t.Errorf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
fix node controller event uid issue
2016-08-13 21:41:20 -04:00
t.Errorf("unexpected error: %v", err)
}
consolidate node deletion logic between node lifecycle and cloud node controller
2018-10-28 21:57:23 -04:00
if len(fakeRecorder.Events) != 1 {
t.Fatalf("unexpected events, got %v, expected %v: %+v", len(fakeRecorder.Events), 1, fakeRecorder.Events)
fix node controller event uid issue
2016-08-13 21:41:20 -04:00
}
consolidate node deletion logic between node lifecycle and cloud node controller
2018-10-28 21:57:23 -04:00
if fakeRecorder.Events[0].Reason != "RegisteredNode" {
Move orphaned Pod deletion logic to PodGC
2016-09-12 10:47:17 -04:00
var reasons []string
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
for _, event := range fakeRecorder.Events {
Move orphaned Pod deletion logic to PodGC
2016-09-12 10:47:17 -04:00
reasons = append(reasons, event.Reason)
}
t.Fatalf("unexpected events generation: %v", strings.Join(reasons, ","))
fix node controller event uid issue
2016-08-13 21:41:20 -04:00
}
Make NodeController test utils usable from outside
2016-11-23 05:30:36 -05:00
for _, event := range fakeRecorder.Events {
fix node controller event uid issue
2016-08-13 21:41:20 -04:00
involvedObject := event.InvolvedObject
actualUID := string(involvedObject.UID)
if actualUID != "1234567890" {
t.Fatalf("unexpected event uid: %v", actualUID)
}
}
}
nodelifecycle controller: reconcile node OS/arch labels
2019-02-22 19:09:07 -05:00
func TestReconcileNodeLabels(t *testing.T) {
fakeNow := metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC)
fakeNodeHandler := &testutil.FakeNodeHandler{
Existing: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
Use topology labels instead of old beta names (#96033) * Rename const for topology.../zone * Rename const for topology.../region * Rename const for failure-domain.../zone * Rename const for failure-domain.../region * Restore old names for compat
2020-11-05 23:26:50 -05:00
v1.LabelTopologyRegion: "region1",
v1.LabelTopologyZone: "zone1",
v1.LabelFailureDomainBetaRegion: "region1",
v1.LabelFailureDomainBetaZone: "zone1",
nodelifecycle controller: reconcile node OS/arch labels
2019-02-22 19:09:07 -05:00
},
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
LastTransitionTime: metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC),
},
},
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx := ktesting.Init(t)
nodelifecycle controller: reconcile node OS/arch labels
2019-02-22 19:09:07 -05:00
nodeController, _ := newNodeLifecycleControllerFromClient(
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx,
nodelifecycle controller: reconcile node OS/arch labels
2019-02-22 19:09:07 -05:00
fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
testNodeMonitorPeriod,
Remove CLI flag enable-taint-manager Signed-off-by: kerthcet <kerthcet@gmail.com>
2022-09-14 05:04:08 -04:00
)
nodelifecycle controller: reconcile node OS/arch labels
2019-02-22 19:09:07 -05:00
nodeController.now = func() metav1.Time { return fakeNow }
nodeController.recorder = testutil.NewFakeRecorder()
adding fakeGetPodsAssignedToNode
2019-09-27 11:28:58 -04:00
nodeController.getPodsAssignedToNode = fakeGetPodsAssignedToNode(fakeNodeHandler.Clientset)
nodelifecycle controller: reconcile node OS/arch labels
2019-02-22 19:09:07 -05:00
tests := []struct {
Name string
Node *v1.Node
ExpectedLabels map[string]string
}{
{
Name: "No-op if node has no labels",
Node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
},
},
ExpectedLabels: nil,
},
{
Name: "No-op if no target labels present",
Node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
Use topology labels instead of old beta names (#96033) * Rename const for topology.../zone * Rename const for topology.../region * Rename const for failure-domain.../zone * Rename const for failure-domain.../region * Restore old names for compat
2020-11-05 23:26:50 -05:00
v1.LabelTopologyRegion: "region1",
nodelifecycle controller: reconcile node OS/arch labels
2019-02-22 19:09:07 -05:00
},
},
},
ExpectedLabels: map[string]string{
Use topology labels instead of old beta names (#96033) * Rename const for topology.../zone * Rename const for topology.../region * Rename const for failure-domain.../zone * Rename const for failure-domain.../region * Restore old names for compat
2020-11-05 23:26:50 -05:00
v1.LabelTopologyRegion: "region1",
nodelifecycle controller: reconcile node OS/arch labels
2019-02-22 19:09:07 -05:00
},
},
{
Remove the 'beta' version of the node label (os and arch types)
2020-05-12 20:22:43 -04:00
Name: "Create OS/arch beta labels when they don't exist",
nodelifecycle controller: reconcile node OS/arch labels
2019-02-22 19:09:07 -05:00
Node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
Remove the 'beta' version of the node label (os and arch types)
2020-05-12 20:22:43 -04:00
v1.LabelOSStable: "linux",
v1.LabelArchStable: "amd64",
nodelifecycle controller: reconcile node OS/arch labels
2019-02-22 19:09:07 -05:00
},
},
},
ExpectedLabels: map[string]string{
kubeletapis.LabelOS: "linux",
kubeletapis.LabelArch: "amd64",
v1.LabelOSStable: "linux",
v1.LabelArchStable: "amd64",
},
},
{
Remove the 'beta' version of the node label (os and arch types)
2020-05-12 20:22:43 -04:00
Name: "Reconcile OS/arch beta labels to match stable labels",
nodelifecycle controller: reconcile node OS/arch labels
2019-02-22 19:09:07 -05:00
Node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: metav1.Date(2012, 1, 1, 0, 0, 0, 0, time.UTC),
Labels: map[string]string{
Remove the 'beta' version of the node label (os and arch types)
2020-05-12 20:22:43 -04:00
kubeletapis.LabelOS: "windows",
kubeletapis.LabelArch: "arm",
v1.LabelOSStable: "linux",
v1.LabelArchStable: "amd64",
nodelifecycle controller: reconcile node OS/arch labels
2019-02-22 19:09:07 -05:00
},
},
},
ExpectedLabels: map[string]string{
kubeletapis.LabelOS: "linux",
kubeletapis.LabelArch: "amd64",
v1.LabelOSStable: "linux",
v1.LabelArchStable: "amd64",
},
},
}
for _, test := range tests {
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if _, err := fakeNodeHandler.Update(tCtx, test.Node, metav1.UpdateOptions{}); err != nil {
t.Fatalf("unexpected error: %v", err)
}
nodelifecycle controller: reconcile node OS/arch labels
2019-02-22 19:09:07 -05:00
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
t.Fatalf("unexpected error: %v", err)
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.reconcileNodeLabels(tCtx, test.Node.Name); err != nil {
t.Fatalf("unexpected error: %v", err)
}
nodelifecycle controller: reconcile node OS/arch labels
2019-02-22 19:09:07 -05:00
if err := nodeController.syncNodeStore(fakeNodeHandler); err != nil {
t.Fatalf("unexpected error: %v", err)
}
node0, err := nodeController.nodeLister.Get("node0")
if err != nil {
t.Fatalf("Can't get current node0...")
}
if len(node0.Labels) != len(test.ExpectedLabels) {
t.Errorf("%s: Unexpected number of taints: expected %d, got %d",
test.Name, len(test.ExpectedLabels), len(node0.Labels))
}
for key, expectedValue := range test.ExpectedLabels {
actualValue, ok := node0.Labels[key]
if !ok {
t.Errorf("%s: Can't find label %v in %v", test.Name, key, node0.Labels)
}
if actualValue != expectedValue {
t.Errorf("%s: label %q: expected value %q, got value %q", test.Name, key, expectedValue, actualValue)
}
}
}
}
removing redundant code
2019-08-14 08:16:29 -04:00
func TestTryUpdateNodeHealth(t *testing.T) {
fakeNow := metav1.Date(2017, 1, 1, 12, 0, 0, 0, time.UTC)
fakeOld := metav1.Date(2016, 1, 1, 12, 0, 0, 0, time.UTC)
fakeNodeHandler := &testutil.FakeNodeHandler{
Existing: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
},
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
}
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx := ktesting.Init(t)
removing redundant code
2019-08-14 08:16:29 -04:00
nodeController, _ := newNodeLifecycleControllerFromClient(
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx,
removing redundant code
2019-08-14 08:16:29 -04:00
fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
testNodeMonitorPeriod,
Remove CLI flag enable-taint-manager Signed-off-by: kerthcet <kerthcet@gmail.com>
2022-09-14 05:04:08 -04:00
)
removing redundant code
2019-08-14 08:16:29 -04:00
nodeController.now = func() metav1.Time { return fakeNow }
nodeController.recorder = testutil.NewFakeRecorder()
adding fakeGetPodsAssignedToNode
2019-09-27 11:28:58 -04:00
nodeController.getPodsAssignedToNode = fakeGetPodsAssignedToNode(fakeNodeHandler.Clientset)
removing redundant code
2019-08-14 08:16:29 -04:00
getStatus := func(cond *v1.NodeCondition) *v1.ConditionStatus {
if cond == nil {
return nil
}
return &cond.Status
}
tests := []struct {
name string
node *v1.Node
}{
{
name: "Status true",
node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
},
},
},
},
{
name: "Status false",
node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionFalse,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
},
},
},
},
{
name: "Status unknown",
node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
},
},
},
},
{
name: "Status nil",
node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{},
},
},
},
{
name: "Status true - after grace period",
node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeOld,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: fakeOld,
LastTransitionTime: fakeOld,
},
},
},
},
},
{
name: "Status false - after grace period",
node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeOld,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionFalse,
LastHeartbeatTime: fakeOld,
LastTransitionTime: fakeOld,
},
},
},
},
},
{
name: "Status unknown - after grace period",
node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeOld,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionUnknown,
LastHeartbeatTime: fakeOld,
LastTransitionTime: fakeOld,
},
},
},
},
},
{
name: "Status nil - after grace period",
node: &v1.Node{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeOld,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{},
},
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
adding lock to node data map
2019-09-09 08:02:38 -04:00
nodeController.nodeHealthMap.set(test.node.Name, &nodeHealthData{
removing redundant code
2019-08-14 08:16:29 -04:00
status: &test.node.Status,
probeTimestamp: test.node.CreationTimestamp,
readyTransitionTimestamp: test.node.CreationTimestamp,
adding lock to node data map
2019-09-09 08:02:38 -04:00
})
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
_, _, currentReadyCondition, err := nodeController.tryUpdateNodeHealth(tCtx, test.node)
removing redundant code
2019-08-14 08:16:29 -04:00
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
move pkg/util/node to component-helpers/node/util (#105347) Signed-off-by: Neha Lohia <nehapithadiya444@gmail.com>
2021-11-12 10:52:27 -05:00
_, savedReadyCondition := controllerutil.GetNodeCondition(nodeController.nodeHealthMap.getDeepCopy(test.node.Name).status, v1.NodeReady)
removing redundant code
2019-08-14 08:16:29 -04:00
savedStatus := getStatus(savedReadyCondition)
currentStatus := getStatus(currentReadyCondition)
adding lock to node data map
2019-09-09 08:02:38 -04:00
if !apiequality.Semantic.DeepEqual(currentStatus, savedStatus) {
removing redundant code
2019-08-14 08:16:29 -04:00
t.Errorf("expected %v, got %v", savedStatus, currentStatus)
}
})
}
}
Node lifecycle controller should use a label for excluding nodes The current mechanism for excluding "master" nodes based on node names is fragile and should be fixed by using a label exclusion similar to service load balancers. The legacy code path is preserved behind a defaulted-on gate and will be removed in the future.
2019-07-16 22:24:21 -04:00
func Test_isNodeExcludedFromDisruptionChecks(t *testing.T) {
validNodeStatus := v1.NodeStatus{Conditions: []v1.NodeCondition{{Type: "Test"}}}
tests := []struct {
name string
set LegacyNodeRoleBehavior to false and mv ServiceNodeExclusion to GA Signed-off-by: pacoxu <paco.xu@daocloud.io>
2020-12-28 02:52:59 -05:00
input *v1.Node
want bool
Node lifecycle controller should use a label for excluding nodes The current mechanism for excluding "master" nodes based on node names is fragile and should be fixed by using a label exclusion similar to service load balancers. The legacy code path is preserved behind a defaulted-on gate and will be removed in the future.
2019-07-16 22:24:21 -04:00
}{
{want: false, input: &v1.Node{Status: validNodeStatus, ObjectMeta: metav1.ObjectMeta{Labels: map[string]string{}}}},
{want: false, input: &v1.Node{Status: validNodeStatus, ObjectMeta: metav1.ObjectMeta{Name: "master-abc"}}},
set LegacyNodeRoleBehavior to false and mv ServiceNodeExclusion to GA Signed-off-by: pacoxu <paco.xu@daocloud.io>
2020-12-28 02:52:59 -05:00
{want: true, input: &v1.Node{Status: validNodeStatus, ObjectMeta: metav1.ObjectMeta{Labels: map[string]string{labelNodeDisruptionExclusion: ""}}}},
Node lifecycle controller should use a label for excluding nodes The current mechanism for excluding "master" nodes based on node names is fragile and should be fixed by using a label exclusion similar to service load balancers. The legacy code path is preserved behind a defaulted-on gate and will be removed in the future.
2019-07-16 22:24:21 -04:00
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if result := isNodeExcludedFromDisruptionChecks(tt.input); result != tt.want {
t.Errorf("isNodeExcludedFromDisruptionChecks() = %v, want %v", result, tt.want)
}
})
}
}
node-lifecycle-controller: improve processPod test-coverage
2024-10-09 06:26:38 -04:00
func TestProcessPodMarkPodNotReady(t *testing.T) {
fakeNow := metav1.Date(2015, 1, 1, 12, 0, 0, 0, time.UTC)
table := []struct {
desc string
fakeNodeHandler *testutil.FakeNodeHandler
pod *v1.Pod
expectedPodStatusUpdate bool
monitorNodeHealth bool
}{
{
desc: "Do not mark pod as NotReady when the scheduled node's healthy is not gathered yet",
fakeNodeHandler: &testutil.FakeNodeHandler{
Existing: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionFalse,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
},
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
},
pod: testutil.NewPod("pod0", "node0"),
monitorNodeHealth: false,
expectedPodStatusUpdate: false,
},
{
desc: "Do not mark pod as NotReady when the scheduled node is ready",
fakeNodeHandler: &testutil.FakeNodeHandler{
Existing: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionTrue,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
},
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
},
pod: testutil.NewPod("pod0", "node0"),
monitorNodeHealth: true,
expectedPodStatusUpdate: false,
},
{
desc: "Pod marked as NotReady when the scheduled node is not ready",
fakeNodeHandler: &testutil.FakeNodeHandler{
Existing: []*v1.Node{
{
ObjectMeta: metav1.ObjectMeta{
Name: "node0",
CreationTimestamp: fakeNow,
},
Status: v1.NodeStatus{
Conditions: []v1.NodeCondition{
{
Type: v1.NodeReady,
Status: v1.ConditionFalse,
LastHeartbeatTime: fakeNow,
LastTransitionTime: fakeNow,
},
},
},
},
},
Clientset: fake.NewSimpleClientset(&v1.PodList{Items: []v1.Pod{*testutil.NewPod("pod0", "node0")}}),
},
pod: testutil.NewPod("pod0", "node0"),
monitorNodeHealth: true,
expectedPodStatusUpdate: true,
},
}
for _, item := range table {
t.Run(item.desc, func(t *testing.T) {
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx := ktesting.Init(t)
node-lifecycle-controller: improve processPod test-coverage
2024-10-09 06:26:38 -04:00
nodeController, _ := newNodeLifecycleControllerFromClient(
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
tCtx,
node-lifecycle-controller: improve processPod test-coverage
2024-10-09 06:26:38 -04:00
item.fakeNodeHandler,
testRateLimiterQPS,
testRateLimiterQPS,
testLargeClusterThreshold,
testUnhealthyThreshold,
testNodeMonitorGracePeriod,
testNodeStartupGracePeriod,
testNodeMonitorPeriod,
)
nodeController.now = func() metav1.Time { return fakeNow }
nodeController.recorder = testutil.NewFakeRecorder()
nodeController.getPodsAssignedToNode = fakeGetPodsAssignedToNode(item.fakeNodeHandler.Clientset)
if err := nodeController.syncNodeStore(item.fakeNodeHandler); err != nil {
t.Errorf("unexpected error: %v", err)
}
if item.monitorNodeHealth {
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
if err := nodeController.monitorNodeHealth(tCtx); err != nil {
node-lifecycle-controller: improve processPod test-coverage
2024-10-09 06:26:38 -04:00
t.Errorf("unexpected error: %v", err)
}
}
if err := nodeController.syncPodStore(item.pod); err != nil {
t.Errorf("unexpected error: %v", err)
}
nodeController.podUpdated(nil, item.pod)
Use `k8s.io/kubernetes/test/utils/ktesting`
2024-10-16 09:15:08 -04:00
nodeController.processPod(tCtx, podUpdateItem{name: item.pod.Name, namespace: item.pod.Namespace})
node-lifecycle-controller: improve processPod test-coverage
2024-10-09 06:26:38 -04:00
podStatusUpdated := false
for _, action := range item.fakeNodeHandler.Actions() {
if action.GetVerb() == "update" && action.GetResource().Resource == "pods" && action.GetSubresource() == "status" {
podStatusUpdated = true
}
}
if podStatusUpdated != item.expectedPodStatusUpdate {
t.Errorf("expect pod status updated to be %v, but got %v", item.expectedPodStatusUpdate, podStatusUpdated)
}
})
}
}
Reference in a new issue Copy permalink