CNI Comparison: Calico vs Cilium vs Flannel vs Multus in 2026
Every Kubernetes platform team picks a Container Network Interface early—usually within the first week—and then lives with that choice for years. A bad CNI pick cascades: cross-AZ latency that nobody debugs, eBPF kernel version quirks that block upgrades, NetworkPolicy that silently fails to enforce, or the day you need multi-homed pods and realize your CNI was never designed for it.
By 2026, four CNI plugins have emerged as production-grade: Calico (BGP-native, BGP-optional eBPF), Cilium (eBPF-first, service-mesh-integrated), Flannel (simple VXLAN overlay, used by k3s), and Multus (not a CNI, but a meta-orchestrator that lets you run multiple CNIs simultaneously—critical for telco and industrial edge). This post compares all four across performance, dataplane choices, NetworkPolicy, observability, and edge-readiness. We’ll also cover the 2026 production picks for greenfield builds, conservative shops, and telco/industrial clusters that need multiple networks.
What a CNI Plugin Actually Does
The Container Network Interface is a specification—just 200 lines of JSON contract between kubelet and a plugin binary. When a Pod is created, kubelet calls the CNI ADD verb with the Pod ID and network namespace. When the Pod is deleted, it calls DEL. That’s the entire interface.
What happens inside that contract, though, is where the CNI choices matter:
IPAM (IP Address Management): Every pod needs an IP. The CNI allocates from a pool (per-node blocks, per-cluster, or even external IPAM). Calico assigns per-node blocks to minimize API calls; Cilium integrates with the same architecture. Flannel uses simple per-node CIDR blocks.
Dataplane: Once pods are IP’d, packets between them must be forwarded. The dataplane is where performance and kernel requirements diverge:
- iptables: Traditional, mature, CPU-intensive at scale (O(n) rule evaluation). Still the default for Flannel and Calico’s iptables mode.
- IPVS: For service load-balancing, more efficient than iptables but not a full dataplane.
- eBPF (Extended Berkeley Packet Filter): Kernel JIT-compiled bytecode at the TC (traffic control) or XDP (eXpress Data Path) hooks. O(1) lookups, native L7 awareness, real-time observability. Cilium is eBPF-native; Calico added it in 3.13.
- BGP (Border Gateway Protocol): Calico and Cilium (1.15+) advertise pod subnets as BGP routes, letting the underlay TOR switch forward packets natively—no overlay, no encapsulation overhead.
Overlay vs. Underlay: The big architectural fork.
- Overlay: Flannel’s VXLAN or Calico’s VXLAN wraps pod packets in IP-in-IP or VXLAN headers. Works across any underlay but adds MTU complexity and encapsulation overhead.
- Underlay/BGP: Calico and Cilium peer with the network’s BGP infrastructure (route reflectors, TOR switches) and advertise pod routes directly. Zero encapsulation, lowest latency, requires BGP-capable infrastructure or a BGP simulator in dev.
Encryption in Transit: Both Calico and Cilium support WireGuard tunnels for encrypted inter-node communication, critical for multi-cloud and untrusted networks.
Calico: BGP-Native, eBPF-Optional
Calico (Tigera’s open-source project) ships with two core components:
- Felix: A DaemonSet that runs on every node. Felix watches the Kubernetes API for Pod and NetworkPolicy changes, updates IPAM, and programs the dataplane (iptables or eBPF).
- Bird BGP Daemon: Also runs on each node and speaks BGP to the network’s route reflectors or TOR switches, advertising pod routes and accepting routes from other nodes.
For scale, Calico adds Typha, a caching layer between the API server and Felix. Without Typha, every node’s Felix watches the API directly; with Typha, Typha caches and fans out updates, reducing API load by 10–100x.
NetworkPolicy Enforcement: Calico’s strength. It implements both Kubernetes NetworkPolicy (pod-to-pod) and Calico’s own GlobalNetworkPolicy (cluster-wide, including host interfaces). Rules are compiled down to iptables or eBPF rules.
eBPF Dataplane (Calico 3.13+): If you enable the eBPF mode, Felix programs eBPF hooks at TC ingress instead of iptables. Same routing, faster enforcement, access to in-kernel packet data for L7 policies. Requires kernel >5.4.
Wireguard Encryption: Both inter-node and cross-cluster tunnels. Set WireguardEnabled: true in the FelixConfiguration and Calico creates WireGuard interfaces for all pods’ traffic.
ARM64 and Edge Friendly: Calico ships prebuilt for ARM64. Felix is lightweight (~50MB memory baseline), and the BGP model scales to thousands of nodes with route reflection.
Use Cases: Production greenfield clusters, on-prem data centers with BGP infrastructure, multi-cloud with private interconnects, edge sites with strong network ops discipline.
Cilium: eBPF-First, Service Mesh–Adjacent
Cilium (Isovalent, now Cisco) is architecturally different: eBPF-first, observability-native, service-mesh-adjacent. Every packet decision happens in kernel-mode eBPF, not userspace.
The eBPF Datapath: Cilium loads eBPF programs at the TC (traffic control) ingress hook on every pod’s veth interface. When a packet arrives, the eBPF program:
- Extracts the pod’s identity (from packet headers or tunnel metadata).
- Looks up the destination pod in an in-kernel eBPF map.
- Applies NetworkPolicy rules (also in eBPF maps).
- Decides to forward, drop, or redirect to the L7 proxy.
All in microseconds, zero context switches. Cilium’s agent runs in userspace and pushes policy and identity updates into the eBPF maps.
Hubble Observability: Hubble taps into the same eBPF events that enforce policy, capturing flow metadata (source/dest identities, verdicts, latencies) without sidecar proxies. You get request-level observability—HTTP headers, gRPC metadata, Kafka topics—natively, because the eBPF layer sees it. No Prometheus scraping overhead, just tail-latency histograms.
L7 Policy (Cilium 1.16+): NetworkPolicy is L3/L4 (IP, port). Cilium now bakes in L7 policy enforcement: allow GET /api/v1/users but deny POST /api/admin. It uses an embedded Envoy sidecar for application-level inspection, but the policy decisions are driven by Cilium’s identity system.
ClusterMesh: Cilium’s multi-cluster story. ClusterMesh syncs pod identities and policies across clusters, so a pod in cluster-A can talk to a pod in cluster-B as if they’re in the same cluster, enforcing the same NetworkPolicy.
BGP Control Plane (Cilium 1.15+): Cilium now includes BGP support, so you can peer with TOR switches and advertise pod routes directly, just like Calico. This is new in 2026; earlier versions required an overlay.
Wireguard Encryption: Full inter-node and inter-cluster encryption, same as Calico.
Kernel Requirements: eBPF at scale requires Linux kernel >5.4 (ideally >5.10 for all features). This is a hard constraint for edge clusters running older distros.
Memory Footprint: Cilium agent + eBPF maps require ~100–200MB per node baseline, plus memory for policy rules. Larger than Calico’s Felix.
Use Cases: Greenfield cloud-native stacks (EKS, GKE, AKS), telco clouds, observability-first teams, multi-cluster platforms.
Flannel: Simple, Boring, Still Useful
Flannel (from CoreOS lineage, now community) is the minimalist choice. It is not a policy engine; it is pure networking: VXLAN overlay, simple IPAM, that’s it.
How It Works: Flannel DaemonSet on each node allocates a /24 (or /25) CIDR block per node. When you deploy a pod, it gets an IP from that block. Flannel encapsulates inter-node traffic in VXLAN headers and sends it across the network. The underlay doesn’t need to know about pods—it just sees VXLAN (UDP 4789).
No NetworkPolicy: Flannel doesn’t implement NetworkPolicy. If you need it, you layer Calico on top (via Canal) or use kube-router for policy enforcement.
VXLAN Overhead: Every inter-node packet adds a 50-byte VXLAN header. Not huge, but on a 1500-byte MTU, that’s real. Flannel adjusts pod MTU to 1450 to avoid fragmentation.
Why Use It? Three reasons:
- k3s ships Flannel by default. It’s the path of least resistance for edge clusters and k3s deployments.
- Ultra-simple to debug. No BGP, no eBPF, no kernel version quirks. VXLAN is VXLAN.
- Lightweight: Flannel binary is ~30MB, memory baseline is negligible.
Flannel + kube-router: If you need NetworkPolicy on Flannel, layer kube-router on top. kube-router is a separate DaemonSet that watches NetworkPolicy and programs iptables rules, then runs a BGP daemon to propagate routes.
Use Cases: k3s deployments, development clusters, edge sites where “boring” beats “clever,” labs and proof-of-concepts.
Multus: Multiple Networks, Not a Replacement CNI
Multus is a meta-CNI: it is not a dataplane; it orchestrates multiple CNI plugins.
The Problem It Solves: Standard Kubernetes pods have one network interface (eth0). Telco/5G use cases, industrial fieldbus clusters, and automotive edge need multiple networks per pod: one for cluster communication, another for high-speed fieldbus (SR-IOV), another for legacy VLAN trunking (MACVLAN).
How It Works: You annotate the Pod manifest:
apiVersion: v1
kind: Pod
metadata:
annotations:
k8s.v1.cni.cncf.io/networks: "default,sr-iov-net,macvlan-legacy"
spec:
containers:
- name: app
image: myapp:latest
kubelet invokes Multus instead of a single CNI. Multus parses the annotation and calls:
- The primary CNI (usually Cilium or Calico) to create eth0 on the cluster network.
- Secondary CNI plugins (SR-IOV, MACVLAN, OVS, etc.) to create net1, net2, etc.
The pod sees three interfaces: eth0 (cluster, 10.0.x.x), net1 (fieldbus, 192.168.x.x), net2 (legacy VLAN, 172.16.x.x).
Multus + Cilium: Primary CNI is Cilium for cluster networking and eBPF policy. Secondary CNIs handle hardware acceleration or legacy protocols.
Multus + Calico: Calico as primary, then layer SR-IOV or OVS secondaries for hardware pass-through.
Multus + kube-router: If you need BGP dynamic routing across industrial subnets.
Use Cases: Telco/5G UPF (User Plane Function) clusters, automotive edge (V2X), industrial IoT with hardwired fieldbus interfaces, time-sensitive networks (TSN) requiring dedicated hardware paths.
Decision Matrix: Performance, Policy, Observability, Edge Fit
Let’s compare the four across the dimensions that matter:
| Feature | Calico | Cilium | Flannel | Multus |
|---|---|---|---|---|
| NetworkPolicy | ✓ Full | ✓ Full | — | Via primary CNI |
| L7 Policy | Partial (Calico Enterprise) | ✓ Native (1.16+) | — | Via CNI |
| eBPF Dataplane | ✓ Optional (3.13+) | ✓ Native | — | Via CNI |
| BGP | ✓ Native | ✓ 1.15+ | Partial (via kube-router) | Via CNI |
| WireGuard Encryption | ✓ | ✓ | — | Via CNI |
| Multi-network Support | — | — | — | ✓ (core feature) |
| Throughput (no policy) | 95+ Gbps | 95+ Gbps | 85+ Gbps* | Via CNI |
| CPU Overhead @ Scale | Medium (iptables) / Low (eBPF) | Low (eBPF) | Very low | Via CNI |
| Observability | Logs + syslog | ✓ Hubble (eBPF-native) | Logs only | Via CNI |
| Kernel >5.10 Required? | Optional (eBPF mode) | Yes (eBPF) | No | No (via CNI) |
| Memory / Node | 50–100 MB | 100–200 MB | ~20 MB | ~10 MB |
| ARM64 Support | ✓ Full | ✓ Full | ✓ Full | ✓ Full |
| Service Mesh Integration | — | ✓ (Cilium mesh) | — | Via CNI |
| Multi-Cluster | Via Calico mesh | ✓ ClusterMesh | — | Via CNI |
Throughput notes: All three achieve wire-rate forwarding (95+ Gbps on 100G NICs) when policies are simple. VXLAN’s 50-byte header in Flannel reduces effective payload throughput by ~5% on small packets.
CPU overhead: iptables rule evaluation scales O(n) with rule count. At 10+ rules, eBPF (Calico eBPF or Cilium) pulls ahead. Flannel has no rules, so overhead is constant.
Kernel Requirement Trade-off: If your edge cluster runs CentOS 7 or Ubuntu 18.04, eBPF isn’t an option. Calico’s iptables mode or Flannel will be your only choices.
Trade-offs and Where Each Falls Short
Calico’s Weak Points:
- BGP Complexity: Requires route reflectors or TOR peering. In flat L2 networks (small offices, labs), BGP adds unnecessary complexity. Easier to use Flannel or Cilium’s overlay mode.
- eBPF Learning Curve: Calico’s eBPF mode is powerful but requires understanding eBPF debugging tools. Most operators stick with iptables.
- Enterprise-Grade Features: L7 policy, ClusterMesh, and advanced observability require Calico Enterprise (not open-source).
Cilium’s Weak Points:
- Kernel Dependency: eBPF-first means >5.4 (really >5.10 for all features). If you have old hardware or distros, Cilium is off the table.
- Observability Overhead: Hubble taps eBPF events, which adds ~5–10% CPU overhead in high-throughput clusters. You can disable it, but then you lose the main advantage.
- Debugging Complexity: eBPF programs are harder to debug than iptables rules. When something breaks, you need
bpftooland kernel tracing knowledge. - Service Mesh Not a Replacement for Istio: Cilium’s embedded mesh is lightweight but lacks the advanced traffic management of Istio (retries, circuit breakers, weighted routing). For complex meshes, Istio + Cilium is still the combo.
Flannel’s Weak Points:
- No NetworkPolicy: You need a separate plugin (kube-router, Calico-via-Canal) for policy enforcement. That adds complexity.
- VXLAN Overhead: 50-byte header per packet. On WAN links or latency-sensitive workloads, this adds ~1–2 microseconds. Not huge, but measurable.
- Limited Observability: Flannel logs to syslog. No eBPF events, no per-packet insight. Harder to debug packet loss.
- No L7 Awareness: Flannel is L3/L4 only. No Kafka topic filtering, no HTTP method matching.
Multus’s Weak Points:
- Complexity: Ordering matters. If the primary CNI is misconfigured, the secondaries can inherit the problem. Debugging multiple CNIs at once is hard.
- No Secondary-to-Secondary Policy: NetworkPolicy only works on the primary network. Secondary networks (SR-IOV, MACVLAN) are not identity-aware.
- Resource Overhead: Each CNI has its own agent and memory footprint. Multus + Cilium + SR-IOV = three agents consuming memory on every node.
- Maturity Variance: If the primary CNI is mature but a secondary plugin is new, you inherit its bugs and limitations.
Practical Recommendations
Greenfield Cluster (2026): Go Cilium. eBPF is no longer experimental; every major cloud provider (AWS, Azure, GCP) runs Cilium at scale. You get eBPF observability, L7 policy, multi-cluster support, and embedded service mesh, all out of the box. If your kernel is >5.10, there’s no downside.
Conservative Shop with Strong NetOps: Calico. Your network team understands BGP, route reflectors, and TOR configurations. Calico integrates cleanly and lets you run with or without eBPF. If you need L7 policy or ClusterMesh, Calico Enterprise is the mature choice.
Minimal Edge Cluster (k3s, 50 nodes or fewer): Flannel. It ships with k3s by default. No debugging required, no kernel surprises, no BGP configuration. If you need NetworkPolicy, layer kube-router on top (add 1 DaemonSet).
Telco/5G/Industrial IoT with Multiple Networks: Multus + Cilium (primary for cluster, SR-IOV + MACVLAN secondaries for fieldbus and legacy protocols). This is the production pattern for telco UPF and automotive edge.
Hybrid / Multi-Cloud: Cilium + ClusterMesh. Cilium’s native multi-cluster support lets you stretch identity and policy across clouds without VPNs or explicit cross-cluster networking rules.
FAQ
Q: Can I migrate from Flannel to Cilium without downtime?
A: Yes, but plan for pod re-scheduling. The safest approach is to drain one node at a time, remove Flannel, install Cilium, and cordon the node until Cilium is ready. Most teams do this over a maintenance window (30–60 minutes for a 10-node cluster). Ensure your workloads tolerate pod restarts; stateful workloads should use persistent volumes.
Q: Does Calico eBPF mode work with NetworkPolicy created for iptables mode?
A: Yes, they’re compatible. eBPF mode reads the same NetworkPolicy resources and enforces the same semantics. You can enable eBPF mode on an existing iptables Calico cluster without rewriting policies. The transition is transparent.
Q: Which CNI has the lowest latency?
A: Calico and Cilium with BGP mode and native routing have the lowest latency (microseconds between pods). Flannel with VXLAN adds ~1–2 microseconds per packet due to encapsulation. In practice, all three are well under 100 microseconds pod-to-pod on modern hardware.
Q: Can I run Cilium and Calico together on the same cluster?
A: No, not as primary CNIs. However, you can run Cilium as the primary and Calico’s VXLAN as a secondary network (via Multus), though this is unusual. Most teams pick one primary and stick with it.
Q: Is Hubble observability free, or does it require Cilium Enterprise?
A: Hubble is free and open-source, included in the OSS Cilium. Cilium Enterprise adds dashboards, Prometheus integrations, and managed Hubble, but the core eBPF observability is in the open-source distribution.
Further Reading
- Cilium eBPF, Service Mesh, Networking & Observability
- eBPF for Kubernetes Observability: Replacing APM at Scale
- K3s for Edge: Production Guide for Industrial Kubernetes
- Container Networking Interface Specification
- Cilium Documentation
About the author: Riju is a platform engineer and Kubernetes specialist focused on cloud infrastructure, observability, and industrial IoT edge clusters. He writes on advanced Kubernetes networking, eBPF, and production DevOps practices.
