OpenShift Virtualization is a feature of Red Hat OpenShift that enables you to run and manage virtual machines alongside containers on the same Kubernetes platform. It brings traditional VM workloads into the modern, cloud-native ecosystem by leveraging KubeVirt to integrate virtual machines directly into OpenShift. This allows organizations to consolidate infrastructure, simplify operations, and gradually modernize legacy applications without needing to migrate everything to containers all at once. With native support for VM lifecycle management, networking, and storage, OpenShift Virtualization offers a unified control plane for both VMs and containers, all managed through the same Kubernetes-native APIs and tools.
NodeNetworkConfigurationPolicies (NNCPs) and NetworkAttachmentDefinitions (NADs) are key components in OpenShift for managing advanced network configurations. NNCPs, part of the NMState Operator, allow you to declaratively configure node-level network settings such as VLANs, bonds, bridges, and static IPs using YAML manifests. These configurations are applied directly to the host operating system, making them ideal for persistent, low-level networking. In contrast, NADs are Kubernetes Custom Resources defined by the Multus CNI plugin that enable pods to attach to multiple networks beyond the default. NADs are used to assign additional interfaces or custom network settings to pods, often for use cases like SR-IOV, DPDK, or VLAN segmentation within the cluster. Together, NNCPs and NADs give you full-stack control of both host and pod networking in OpenShift.
First thing is to setup the NNCP to connect to the vlan you want to connect to. Here’s an example NNCP for setting up a linux bridge on my homelab OpenShift cluster, which is using a bonded ethernet connection.
apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
name: vlan4-bridge-on-bond0
spec:
desiredState:
interfaces:
- name: vlan4
state: up
type: vlan
vlan:
base-iface: bond0
id: 4
- bridge:
options:
stp:
enabled: false
port:
- name: vlan4
name: br-vlan4
state: up
type: linux-bridge
nodeSelector:
node-role.kubernetes.io/worker: ''Using cloud-init
The NNCP is at the cluster level. Now, for our VMs to be able to use it, we need to create a NAD which is namespace based.
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
name: nad-vlan4
namespace: vms
spec:
config: |-
{
"cniVersion": "0.3.1",
"name": "nad-vlan4",
"type": "bridge",
"bridge": "br-vlan4",
"ipam": {},
"macspoofchk": true,
"vlan": 4
}When creating the VM, you can then use the cloud-init to set the static ip and use the NAD.
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: rhel9-vlan4-staticip
namespace: vms
labels:
app: rhel9-vlan4-staticip
kubevirt.io/dynamic-credentials-support: 'true'
vm.kubevirt.io/template: rhel9-server-small
vm.kubevirt.io/template.namespace: openshift
vm.kubevirt.io/template.revision: '1'
vm.kubevirt.io/template.version: v0.32.2
spec:
dataVolumeTemplates:
- apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
creationTimestamp: null
name: rhel9-vlan4-staticip
spec:
sourceRef:
kind: DataSource
name: rhel9
namespace: openshift-virtualization-os-images
storage:
resources:
requests:
storage: 30Gi
runStrategy: RerunOnFailure
template:
metadata:
annotations:
vm.kubevirt.io/flavor: small
vm.kubevirt.io/os: rhel9
vm.kubevirt.io/workload: server
creationTimestamp: null
labels:
kubevirt.io/domain: rhel9-vlan4-staticip
kubevirt.io/size: small
network.kubevirt.io/headlessService: headless
spec:
architecture: amd64
domain:
cpu:
cores: 1
sockets: 1
threads: 1
devices:
disks:
- disk:
bus: virtio
name: rootdisk
- disk:
bus: virtio
name: cloudinitdisk
interfaces:
- macAddress: '02:08:5f:00:00:45'
masquerade: {}
model: virtio
name: default
- bridge: {}
macAddress: '02:08:5f:00:00:46'
model: virtio
name: nic-vlan4
rng: {}
features:
acpi: {}
smm:
enabled: true
firmware:
bootloader:
efi: {}
machine:
type: pc-q35-rhel9.4.0
memory:
guest: 2Gi
resources: {}
networks:
- name: default
pod: {}
- multus:
networkName: nad-vlan4
name: nic-vlan4
terminationGracePeriodSeconds: 180
volumes:
- dataVolume:
name: rhel9-vlan4-staticip
name: rootdisk
- cloudInitNoCloud:
networkData: |
ethernets:
eth1:
addresses:
- 10.4.0.50
gateway4: 10.4.0.1
version: 2
userData: |-
#cloud-config
user: cloud-user
password: Pass123!
chpasswd: { expire: False }
name: cloudinitdiskNotice the cloud init.
networkData: |
ethernets:
eth1:
addresses:
- 10.4.0.50
gateway4: 10.4.0.1
version: 2
Using the NAD Itself
Another way to configure the static ip is to do so in the NAD itself. This takes away the VM based config and moves it to the platform and thus into a more controlled environment. Admins can set static IPs for workloads and even GitOps the entire configuration.
We are going to use the same NNCP, but create a different NAD.
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
name: nad-vlan4-staticip-74
namespace: vms
spec:
config: |-
{
"name": "nad-vlan4-staticip-74",
"cniVersion": "0.3.1",
"type": "bridge",
"bridge": "br-vlan4",
"vlan": 4,
"ipam": {
"type": "static",
"addresses": [
{ "address": "10.4.0.74/24", "gateway": "10.4.0.1" }
]
}
}Then we can just use the NAD in the VM and it’ll grab that IP.
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: rhel9-vlan-4-staticip-74
namespace: vms
labels:
app: rhel9-vlan-4-staticip-74
kubevirt.io/dynamic-credentials-support: 'true'
vm.kubevirt.io/template: rhel9-server-small
vm.kubevirt.io/template.namespace: openshift
vm.kubevirt.io/template.revision: '1'
vm.kubevirt.io/template.version: v0.32.2
spec:
dataVolumeTemplates:
- apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
creationTimestamp: null
name: rhel9-vlan-4-staticip-74
spec:
sourceRef:
kind: DataSource
name: rhel9
namespace: openshift-virtualization-os-images
storage:
resources:
requests:
storage: 30Gi
runStrategy: RerunOnFailure
template:
metadata:
annotations:
vm.kubevirt.io/flavor: small
vm.kubevirt.io/os: rhel9
vm.kubevirt.io/workload: server
creationTimestamp: null
labels:
kubevirt.io/domain: rhel9-vlan-4-staticip-74
kubevirt.io/size: small
network.kubevirt.io/headlessService: headless
spec:
architecture: amd64
domain:
cpu:
cores: 1
sockets: 1
threads: 1
devices:
disks:
- disk:
bus: virtio
name: rootdisk
- disk:
bus: virtio
name: cloudinitdisk
interfaces:
- macAddress: '02:08:5f:00:00:16'
masquerade: {}
model: virtio
name: default
- bridge: {}
macAddress: '02:08:5f:00:00:17'
model: virtio
name: nic-vlan-staticip-74
rng: {}
features:
acpi: {}
smm:
enabled: true
firmware:
bootloader:
efi: {}
machine:
type: pc-q35-rhel9.4.0
memory:
guest: 2Gi
resources: {}
networks:
- name: default
pod: {}
- multus:
networkName: nad-vlan4-staticip-74
name: nic-vlan-staticip-74
terminationGracePeriodSeconds: 180
volumes:
- dataVolume:
name: rhel9-vlan-4-staticip-74
name: rootdisk
- cloudInitNoCloud:
userData: |-
#cloud-config
user: cloud-user
password: 2y04-al7n-128h
chpasswd: { expire: False }
name: cloudinitdisk