EndpointSlices mirror controller for User-Defined Networks¶

Summary¶

Pods that use a user-defined network as their primary network will still have the cluster default network IP in their status. For services this results in the EndpointSlices providing the IPs of the cluster default network in the Kubernetes API. To enable services support for primary user-defined networks, the EndpointSlices mirror controller was introduced to create custom EndpointSlices with user-defined network IP addresses extracted from OVN-Kubernetes annotations.

Implementation¶

The introduced controller duplicates the default EndpointSlices, creating new copies that include IP addresses from primary user-defined network. It bypasses EndpointSlices in namespaces that do not have a user-defined primary network. The controller lacks specific logic for selecting endpoints, it only replicates those generated by the default controller and replaces the IP addresses. For host-networked pods, the controller retains the same IP addresses as the default controller. Custom EndpointSlices not created by the default controller are not processed.

The default EndpointSlices controller creates objects that contain the following labels:

endpointslice.kubernetes.io/managed-by:endpointslice-controller.k8s.io - Indicates that the EndpointSlice is managed by the default Kubernetes EndpointSlice controller.
kubernetes.io/service-name:<service-name> - The service that this EndpointSlice belongs to, used by the default network service controller.

The EndpointSlices mirror controller uses a separate set of labels:

endpointslice.kubernetes.io/managed-by:endpointslice-mirror-controller.k8s.ovn.org - Indicates that the EndpointSlice is managed by the mirror controller.
k8s.ovn.org/service-name:<service-name> - The service that this mirrored EndpointSlice belongs to, used by the user-defined network service controller. Note that the label key is different from the default EndpointSlice.
k8s.ovn.org/source-endpointslice-version:<default-endpointslice-resourceversion> - The last reconciled resource version from the default EndpointSlice.

and annotations (Label values have a length limit of 63 characters): - k8s.ovn.org/endpointslice-network:<udn-network-name> - The user-defined network that the IP addresses in the mirrored EndpointSlice belong to. - k8s.ovn.org/source-endpointslice:<default-endpointslice> - The name of the default EndpointSlice that was the source of the mirrored EndpointSlice.

Example¶

With the following NetworkAttachmentDefinition:

apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: l3-network
  namespace: nad-l3
spec:
  config: |2
    {
            "cniVersion": "1.0.0",
            "name": "l3-network",
            "type": "ovn-k8s-cni-overlay",
            "topology":"layer3",
            "subnets": "10.128.0.0/16/24",
            "mtu": 1300,
            "netAttachDefName": "nad-l3/l3-network",
            "role": "primary"
    }

We can observe the following EndpointSlices created for a one-replica deployment exposed through a sample-deployment service:

Default EndpointSlice Mirrored EndpointSlice

kind: EndpointSlice
apiVersion: discovery.k8s.io/v1
metadata:
  name: sample-deployment-rkk4n
  generateName: sample-deployment-
  generation: 1
  labels:
    app: l3pod
    endpointslice.kubernetes.io/managed-by: endpointslice-controller.k8s.io
    kubernetes.io/service-name: sample-deployment
  name: sample-deployment-rkk4n
  namespace: nad-l3
  resourceVersion: "31533"
addressType: IPv4
endpoints:
- addresses:
  - 10.244.1.17
  conditions:
    ready: true
    serving: true
    terminating: false
  nodeName: ovn-worker
  targetRef:
    kind: Pod
    name: sample-deployment-6b64bd4868-7ftt6
    namespace: nad-l3
    uid: 6eb5d05c-cff4-467d-bc1b-890443750463
ports:
- name: ""
  port: 80
  protocol: TCP

kind: EndpointSlice
apiVersion: discovery.k8s.io/v1
metadata:
  name: l3-network-sample-deployment-hgkmw
  generateName: l3-network-sample-deployment-
  labels:
    endpointslice.kubernetes.io/managed-by: endpointslice-mirror-controller.k8s.ovn.org
    k8s.ovn.org/service-name: sample-deployment
    k8s.ovn.org/source-endpointslice-version: "31533"
  annotations:
    k8s.ovn.org/endpointslice-network: l3-network
    k8s.ovn.org/source-endpointslice: sample-deployment-rkk4n
  namespace: nad-l3
  resourceVersion: "31535"
addressType: IPv4
endpoints:
- addresses:
  - 10.128.1.3
  conditions:
    ready: true
    serving: true
    terminating: false
  nodeName: ovn-worker
  targetRef:
    kind: Pod
    name: sample-deployment-6b64bd4868-7ftt6
    namespace: nad-l3
    uid: 6eb5d05c-cff4-467d-bc1b-890443750463
ports:
- name: ""
  port: 80
  protocol: TCP