Saikrishna Edupuganti
0990006efb
Testpmd manifests of the last 3 LTS and 1 latest release from stable repo to help test actual DPDK app instead of sleep. Currently 17.11 and 19.11 are the only functioning ones without privileged. ``` NAME READY STATUS RESTARTS AGE dpdk-1711 1/1 Running 0 3m5s dpdk-1811 0/1 Error 0 3m5s dpdk-1911 1/1 Running 0 3m5s dpdk-2002 0/1 Error 0 3m5s ``` ``` EAL: PCI device 0000:07:06.4 on NUMA socket 0 EAL: probe driver: 8086:154c net_i40e_vf EAL: Getting a vfio_dev_fd for 0000:07:06.4 failed EAL: Requested device 0000:07:06.4 cannot be used … testpmd: No probed ethernet devices EAL: Error - exiting with code: 1 Cause: Invalid port 1 ``` Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
Multi-Network
Daemonset
We launch a Daemonset with an initContainer which sets up the CNI
directories on the host with the necessary binaries and configuration files.
NOTE: SR-IOV devices are not necessary to test multi-network capability
Customization
The device plugin will register the SR-IOV enabled devices on the host, specified with
selectors in sriov-conf.yaml. Helper systemd unit
file is provided, which enables SR-IOV for the above devices. More config options
are listed here.
Pre-req (SR-IOV only)
One each SR-IOV node make sure VT-d is enabled in the BIOS and intel_iommu=on on kernel commandline.
Setup systemd to bring up VFs on designated interfaces bound to network driver or vfio-pci
# Make sure vfio-pci is loaded on boot
echo 'vfio-pci' | sudo tee /etc/modules-load.d/sriov.conf
sudo systemctl restart systemd-modules-load.service
sudo cp systemd/sriov.sh /usr/bin/sriov.sh
sudo cp systemd/sriov.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now sriov.service
Install
To install and configure multus-cni on all nodes, along with
sriov-cni, vfioveth-cni and sriov-network-device-plugin
kubectl apply -f .
kubectl get nodes -o json | jq '.items[].status.allocatable' # should list "intel.com/sriov_*"
Tests
Default only
To test if default connectivity is working
kubectl apply -f test/pod.yaml
kubectl exec test -- ip a # should see one interface only
Bridge
To test multus with second interface created by bridge plugin
kubectl apply -f test/bridge
kubectl exec test-bridge -- ip a # should see two interfaces
ip a show mynet # bridge created on host if it doesnt exist already
SR-IOV
To test multus with second interface created by sriov plugin
kubectl apply -f test/sriov
kubectl exec test-sriov -- ip a # second interface is a VF
kubectl exec test-sriov-dpdk -- ip a # veth pair with details of VF
kubectl exec test-sriov-dpdk -- ls -l /dev/vfio