Moves endpoint logging from endpointslicecache.go down to endpointschangetracker.go merge and unmerge methods to be consistent with service updates, without losing endpoint tracking information.
Change-Id: I09369a4b32929d8df4b520921d74223090872a60
Logs when a nodePort is configured, cleared, or removed from a service, similar to how endpoints are logged, to improve traceability in kube-proxy.
Change-Id: I75179292cb7ef52649b6fb3081afea1d0c9534af
detectNumCPU sized nf_conntrack_max using github.com/google/cadvisor/lib
machine topology. nf_conntrack_max is host-wide, so it must be sized from the
node's CPU count, not runtime.NumCPU(): the latter honors the process cpuset
and undercounts when kube-proxy runs under a static CPU policy, which is the
behavior cadvisor worked around (kubernetes/kubernetes#99225).
Use cpuset.NumCPU() from k8s.io/utils, which reads the node's online CPU count
from /sys/devices/system/cpu/online, with a runtime.NumCPU() fallback. This
drops the cadvisor dependency from pkg/proxy/conntrack and bumps k8s.io/utils
to pick up cpuset.NumCPU.
Add import-boss .import-restrictions so the lean github.com/google/cadvisor/lib module is allowed only in the directories that consume it -- pkg/kubelet, cmd/kubelet/app, and pkg/proxy/conntrack -- and forbidden everywhere else. The repo-root .import-restrictions default-denies all github.com/google/cadvisor imports; the three consumer subtrees re-allow only .../lib.
Migrate the kubelet (and the kube-proxy conntrack helper) off the full github.com/google/cadvisor module onto the lean github.com/google/cadvisor/lib: repoint info/v1+info/v2 type usage to lib/model and the manager/fs/cache/etc. consumers to lib/*; regenerate the cadvisor.Interface mocks; keep the kubelet-pinned cAdvisor global flags via lib/cadvisorflags.
go.mod: require + replace github.com/google/cadvisor/lib (=> the dims/cadvisor/lib fork for now, until lib is tagged) and drop the full github.com/google/cadvisor module entirely; keep github.com/containerd/containerd/api at v1.11.0 (matching upstream master); add github.com/google/cadvisor to unwanted-dependencies.json unwantedModules so the full module cannot be re-vendored.
test/e2e_node: the one remaining consumer of the full module -- the node-e2e ResourceCollector, which used the v2 HTTP client (client/v2) + v2 API types (info/v2) -- now scrapes the standalone cAdvisor pod's /api/v2.1/stats directly over HTTP+JSON, so test/e2e_node depends on no cAdvisor package. No change to the kubelet's stats surfaces.
The conntrack reconciler skips services without serving endpoints, so
conntrack entries established while endpoints existed are never removed
when a UDP service scales down to zero. The REJECT (iptables) / reject
(nftables) rule installed for such services does not cover those flows:
they are DNATed to the deleted endpoint IP before the rule, which
matches on the service IP, can be evaluated. One-way UDP senders (e.g.
statsd clients) refresh the 30s conntrack timeout with every packet, so
the stale flows blackhole traffic to the deleted pod IP indefinitely;
recovery only happens when the service gets an endpoint again.
This was handled before the reconciler rewrite (kubernetes#127318):
the event-based cleanup cleared entries for every deleted UDP endpoint
regardless of how many endpoints remained.
Process services with an empty endpoints set instead of skipping them,
so every entry directed to their ClusterIP, LoadBalancer IP and
ExternalIP frontends is treated as stale and deleted.
NodePort cleanup is still skipped for services without serving
endpoints: NodePort entries are matched on the destination port only,
and with an empty endpoints set that would also remove UDP flows not
owned by kube-proxy (e.g. traffic to an unrelated host on the same
port).
The classifyLBError function returns lbErrNone when err is nil,
but the test was incorrectly expecting lbErrOther.
Also add lbErrNone to TestLBErrorTypeConstants verification.
GetAllLocalAddressesExcept previously iterated over net.Interfaces() and
called iface.Addrs() for each interface. iface.Addrs() internally performs
a full RTM_GETADDR netlink dump for the entire node and then filters in
user space. With many interfaces and many addresses (for example tens of
thousands of ClusterIPs bound to kube-ipvs0) the cost is
O(N_interfaces * N_addresses) and dominates syncProxyRules latency.
This change replaces the per-interface loop with a single
netlink.AddrList(nil, unix.AF_UNSPEC) call that dumps all addresses on
the node in one RTM_GETADDR request, then filters by LinkIndex in user
space. This makes the call O(N_addresses) and avoids the per-interface
fan-out.
On a production node with 251 interfaces and 19757 addresses, this
reduces GetAllLocalAddressesExcept latency from 34.8s to 60ms (~705x).
Simplify the interface between cmd/kube-proxy and the backends by
passing the complete KubeProxyConfiguration to the backend rather than
having kube-proxy need to know specifically which fields each backend
cares about.
Periodic full-syncs are just reconcile loops just in case somehow
the dataplane has drifted, however, they have an important cost on large
clusters.
We can avoid to perform full-sync if kube-proxy is in the "largecluster"
mode, we are already doing some optimization, so it is reasonable to
avoid the penalty of a full sync for a "just in case" operation.
Signed-off-by: Antonio Ojea <aojea@google.com>
This fixes a bug that caused log calls involving `klog.Logger` to not be
checked.
As a result we have to fix some code that is now considered faulty:
ERROR: pkg/controller/serviceaccount/tokens_controller.go:382:1: A function should accept either a context or a logger, but not both. Having both makes calling the function harder because it must be defined whether the context must contain the logger and callers have to follow that. (logcheck)
ERROR: func (e *TokensController) generateTokenIfNeeded(ctx context.Context, logger klog.Logger, serviceAccount *v1.ServiceAccount, cachedSecret *v1.Secret) ( /* retry */ bool, error) {
ERROR: ^
ERROR: pkg/controller/storageversionmigrator/storageversionmigrator.go:299:1: A function should accept either a context or a logger, but not both. Having both makes calling the function harder because it must be defined whether the context must contain the logger and callers have to follow that. (logcheck)
ERROR: func (svmc *SVMController) runMigration(ctx context.Context, logger klog.Logger, gvr schema.GroupVersionResource, resourceMonitor *garbagecollector.Monitor, toBeProcessedSVM *svmv1beta1.StorageVersionMigration, listResourceVersion string) (err error, failed bool) {
ERROR: ^
ERROR: pkg/proxy/node.go:121:3: logging function "Error" should not use format specifier "%q" (logcheck)
ERROR: klog.FromContext(ctx).Error(nil, "Timed out waiting for node %q to exist", nodeName)
ERROR: ^
ERROR: pkg/proxy/node.go:123:3: logging function "Error" should not use format specifier "%q" (logcheck)
ERROR: klog.FromContext(ctx).Error(nil, "Timed out waiting for node %q to be assigned IPs", nodeName)
ERROR: ^
ERROR: pkg/scheduler/backend/queue/scheduling_queue.go:610:1: A function should accept either a context or a logger, but not both. Having both makes calling the function harder because it must be defined whether the context must contain the logger and callers have to follow that. (logcheck)
ERROR: func (p *PriorityQueue) runPreEnqueuePlugin(ctx context.Context, logger klog.Logger, pl fwk.PreEnqueuePlugin, pInfo *framework.QueuedPodInfo, shouldRecordMetric bool) *fwk.Status {
ERROR: ^
ERROR: pkg/scheduler/framework/plugins/dynamicresources/extendeddynamicresources.go:286:1: A function should accept either a context or a logger, but not both. Having both makes calling the function harder because it must be defined whether the context must contain the logger and callers have to follow that. (logcheck)
ERROR: func (pl *DynamicResources) deleteClaim(ctx context.Context, claim *resourceapi.ResourceClaim, logger klog.Logger) error {
ERROR: ^
ERROR: pkg/scheduler/framework/plugins/dynamicresources/extendeddynamicresources.go:499:1: A function should accept either a context or a logger, but not both. Having both makes calling the function harder because it must be defined whether the context must contain the logger and callers have to follow that. (logcheck)
ERROR: func (pl *DynamicResources) waitForExtendedClaimInAssumeCache(
ERROR: ^
ERROR: pkg/scheduler/framework/plugins/dynamicresources/extendeddynamicresources.go:528:1: A function should accept either a context or a logger, but not both. Having both makes calling the function harder because it must be defined whether the context must contain the logger and callers have to follow that. (logcheck)
ERROR: func (pl *DynamicResources) createExtendedResourceClaimInAPI(
ERROR: ^
ERROR: pkg/scheduler/framework/plugins/dynamicresources/extendeddynamicresources.go:592:1: A function should accept either a context or a logger, but not both. Having both makes calling the function harder because it must be defined whether the context must contain the logger and callers have to follow that. (logcheck)
ERROR: func (pl *DynamicResources) unreserveExtendedResourceClaim(ctx context.Context, logger klog.Logger, pod *v1.Pod, state *stateData) {
ERROR: ^
ERROR: pkg/scheduler/framework/runtime/batch.go:171:1: A function should accept either a context or a logger, but not both. Having both makes calling the function harder because it must be defined whether the context must contain the logger and callers have to follow that. (logcheck)
ERROR: func (b *OpportunisticBatch) batchStateCompatible(ctx context.Context, logger klog.Logger, pod *v1.Pod, signature fwk.PodSignature, cycleCount int64, state fwk.CycleState, nodeInfos fwk.NodeInfoLister) bool {
ERROR: ^
ERROR: staging/src/k8s.io/component-base/featuregate/feature_gate.go:890:4: Additional arguments to Info should always be Key Value pairs. Please check if there is any key or value missing. (logcheck)
ERROR: logger.Info("Warning: SetEmulationVersionAndMinCompatibilityVersion will change already queried feature", "featureGate", feature, "oldValue", oldVal, newVal)
ERROR: ^
ERROR: test/images/sample-device-plugin/sampledeviceplugin.go:108:2: logging function "Info" should not use format specifier "%s" (logcheck)
ERROR: logger.Info("pluginSocksDir: %s", pluginSocksDir)
ERROR: ^
ERROR: test/images/sample-device-plugin/sampledeviceplugin.go:123:2: logging function "Info" should not use format specifier "%s" (logcheck)
ERROR: logger.Info("CDI_ENABLED: %s", cdiEnabled)
ERROR: ^
While waiting for this to merge, another call was added which also doesn't
follow conventions:
ERROR: pkg/kubelet/kubelet.go:2454:1: A function should accept either a context or a logger, but not both. Having both makes calling the function harder because it must be defined whether the context must contain the logger and callers have to follow that. (logcheck)
ERROR: func (kl *Kubelet) deletePod(ctx context.Context, logger klog.Logger, pod *v1.Pod) error {
ERROR: ^
Contextual logging has been beta and enabled by default for several releases
now. It's mostly just a matter of wrapping up and declaring it GA. Therefore
the calls which directly call WithName or WithValues (always have an effect)
are left as-is instead of converting them to use the klog wrappers (support
disabling the effect). To allow that, the linter gets reconfigured to not
complain about this anymore, anywhere.
The calls which would have to be fixed otherwise are:
ERROR: pkg/kubelet/cm/dra/claiminfo.go:170:11: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: logger = logger.WithName("dra-claiminfo")
ERROR: ^
ERROR: pkg/kubelet/cm/dra/healthinfo.go:45:11: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: logger = logger.WithName("dra-healthinfo")
ERROR: ^
ERROR: pkg/kubelet/cm/dra/healthinfo.go:89:11: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: logger = logger.WithName("dra-healthinfo")
ERROR: ^
ERROR: pkg/kubelet/cm/dra/healthinfo.go:157:11: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: logger = logger.WithName("dra-healthinfo")
ERROR: ^
ERROR: pkg/kubelet/cm/dra/manager.go:175:12: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: logger := klog.FromContext(ctx).WithName("dra-manager")
ERROR: ^
ERROR: pkg/kubelet/cm/dra/manager.go:239:12: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: logger := klog.FromContext(ctx).WithName("dra-manager")
ERROR: ^
ERROR: pkg/kubelet/cm/dra/manager.go:593:12: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: logger := klog.FromContext(ctx).WithName("dra-manager")
ERROR: ^
ERROR: pkg/kubelet/cm/dra/manager.go:781:12: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: logger := klog.FromContext(context.Background()).WithName("dra-manager")
ERROR: ^
ERROR: pkg/kubelet/cm/dra/manager.go:898:12: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: logger := klog.FromContext(ctx).WithName("dra-manager")
ERROR: ^
ERROR: pkg/kubelet/cm/dra/manager_test.go:1638:15: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: logger := klog.FromContext(streamCtx).WithName(st.Name())
ERROR: ^
ERROR: pkg/kubelet/cm/dra/plugin/dra_plugin.go:77:12: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: logger := klog.FromContext(ctx).WithName("dra-plugin")
ERROR: ^
ERROR: pkg/kubelet/cm/dra/plugin/dra_plugin.go:108:12: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: logger := klog.FromContext(ctx).WithName("dra-plugin")
ERROR: ^
ERROR: pkg/kubelet/cm/dra/plugin/dra_plugin.go:161:12: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: logger := klog.FromContext(ctx).WithName("dra-plugin")
ERROR: ^
ERROR: staging/src/k8s.io/dynamic-resource-allocation/resourceslice/tracker/tracker.go:695:14: function "WithValues" should be called through klogr.LoggerWithValues (logcheck)
ERROR: logger := logger.WithValues("device", deviceID)
ERROR: ^
ERROR: test/integration/apiserver/watchcache_test.go:42:54: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: etcd0URL, stopEtcd0, err := framework.RunCustomEtcd(klog.FromContext(ctx).WithName("etcd0"), "etcd_watchcache0", etcdArgs)
ERROR: ^
ERROR: test/integration/apiserver/watchcache_test.go:47:54: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: etcd1URL, stopEtcd1, err := framework.RunCustomEtcd(klog.FromContext(ctx).WithName("etcd1"), "etcd_watchcache1", etcdArgs)
ERROR: ^
ERROR: test/integration/scheduler_perf/scheduler_perf.go:1149:12: function "WithName" should be called through klogr.LoggerWithName (logcheck)
ERROR: logger = logger.WithName(tCtx.Name())
ERROR: ^
In case of dual-stack, kube-proxy tries to bind both IPv4 and IPv6
health check instances to the same address and port pair which causes
the following error message in the log: 'bind: address already in use'.
Fix the issue by binding IPv4 instance to a 'tcp4' socket and IPv6 instance
to a 'tcp6' socket.
Signed-off-by: Tero Kauppinen <tero.kauppinen@est.tech>
Previously it was leaving NodeName unset in many cases. Give all of
the endpoints an explicit NodeName, making them explicitly local in
all the test cases that don't care either way, and explicitly
non-local in those test cases that did care but were previously just
relying on the fact that a nil NodeName would be treated as remote.
