The last blogpost discussed a real world 5G reference architecture based on a hyperscaler/datacenter mix. The architecture is based on a Cloud Native design that leverages Kubernetes (Amazon EKS in this case) as the orchestrator that provides automated deployment and lifecycle management of CNFs deployed on pods. This post discusses how to overcome the first limitation of K8s from a 5G standpoint – enabling pods to assume multiple network interfaces.
Kubernetes Networking considerations for 5G Networks
Mobile wireless core based on 5G, has a stated goal to support interoperability with 4G LTE, edge networking and various elements such as SDN/NFV. It needs to support high volume of data transfer while supporting high performance networking.
A Pod is the basic runtime unit in a K8s environment. It is analogous to a VM and is scheduled and managed by the K8s master which has a scheduler interface. The pod runs one or more containers and leverages the CNI interface to interact with the network. In enterprise applications, a pod has a single network interface (eth0) which works well when used in conjunction with a CNI such as Cilium, Calico, Flannel etc.
On the other hand, as opposed to enterprise applications, Telco workloads require more sophisticated network interfaces to support multi-homed networks with various QoS (Quality of Service) guarantees. Various CNFs that run on pods can belong to different planes: control, management, and data. Also, they may be associated with different QoS requirements and different network isolation domains. The distribution of K8s running these CNFs should support multi-network interfaces. Furthermore, these pods should be able to call out container networking interfaces like Cilium, Calico and DPDK, SR-IOV-DPDK. The default behavior in Kubernetes is that every Pod only has a single network interface, which is highly restrictive for such use cases.
Introducing Multus
Multus (https://github.com/k8snetworkplumbingwg/multus-cni) is a plugin for Kubernetes, developed by Intel, to enable attaching multiple network interfaces to pods. Multus is configured via CRDs and enables each pod to assume multiple network interfaces. It acts as a metaplugin to call various other CNI plugins. It is configured via CRDs, so it is Kubernetes-native by design.
From the github page –
Here’s an illustration of the network interfaces attached to a pod, as provisioned by Multus CNI. The diagram shows the pod with three interfaces: eth0, net0 and net1. eth0 connects Kubernetes cluster network to connect with Kubernetes server/services (e.g. Kubernetes api-server, kubelet and so on). net0 and net1 are additional network attachments and connect to other networks by using other CNI plugins (e.g. vlan/vxlan/ptp).
Multus allows pods to have multiple network interface connections that can address various use cases; for example:
- Separating the control/data plane traffic as in the case of vnf/cnf applications
- Supporting specialized hardware
- Legacy/storage-intensive applications
- Supporting chaining of CNIs to achieve specific use case functionality
- Multi-Tenant networks
Multus among others has emerged as the most popular open-source meta CNI plugin. In the 5G architecture, S‐GW and P‐GW components need to be set up with multiple network interfaces (S1U, SGi, Control Plane).
In many Telco use cases, Pods need to be multi-homed so they are able to operate across multiple networks. Multus is a Kubernetes CNI that enables multiple interfaces to be associated with a Pod. One possible design approach for many use cases is to create a K8s operator that installs Multus, MACVLAN and IPVLAN (MACVLAN and IPVLAN are Linux network drivers that expose underlay or host interfaces directly to VMs or Containers running in the host).
Conclusion
Multus adds capabilities to Kubernetes that enable it to be extended to support multiple network interfaces. These interfaces can also include high-performing ones such as SR-IOV and DPDK. The next blog will introduce how the 5G data plane can be accelerated with SR-IOV and DPDK.