Kubernetes Multi-Cluster Management with Cluster API: A 2026 Reference Architecture

One cluster is a computer. A hundred clusters is a distributed system whose nodes are themselves distributed systems, and that recursion is where most platform teams get hurt. Kubernetes multi cluster management with Cluster API is the discipline of treating a whole fleet as one declarative surface: you describe the clusters you want, the add-ons they carry and the policies they obey, and controllers make it true.

Done badly, a fleet becomes a spreadsheet of snowflake clusters and a pager that never sleeps. Done well, it is boring. A new region is a pull request, an upgrade is a rolling change with a revert button, and blast radius stops at a cluster boundary.

This is a working reference architecture for teams already running Kubernetes in production who now need to run twenty or two hundred clusters without scaling headcount linearly. It is current as of October 2026, which matters: Cluster API has shipped in-place updates, chained upgrades and a new v1beta2 API since most fleet guides were written.

What this covers: why fleets exist, the hub-and-spoke model, a Cluster API deep-dive (objects, ClusterClass, the 2026 changes), GitOps fan-out with ApplicationSets and Flux, fleet policy, cross-cluster networking and identity, hub capacity maths, upgrade rings, failure modes and a build checklist.

Context and Background

The single-cluster era had a comfortable lie built in: scale up until you cannot, then figure out the rest later. Kubernetes itself sets the ceiling. The project’s documented large-cluster envelope is roughly 5,000 nodes, 150,000 pods and 300,000 containers, with no more than 110 pods per node (see the Kubernetes large-cluster considerations). Those are maxima under favourable conditions, not comfortable operating points. Long before them, etcd write latency, the API server watch cache and controller reconcile loops become the bottleneck.

Raw scale was never the main reason teams split clusters, though. Blast radius was. A cluster is one failure domain for its control plane, its CNI, its admission webhooks and its cluster-scoped CRDs. A bad conversion webhook or a wedged etcd quorum takes down everything scheduled there. Splitting workloads converts a company-wide incident into a regional one.

Then come the drivers that are not negotiable:

Data residency: EU data must live in the EU, so the control plane and the workloads sit in EU infrastructure.
Regulatory isolation: a PCI-scoped cluster with its own admission rules and audit trail is easier to defend than a namespace.
Hard tenancy: a hostile or noisy tenant that must never share a kernel, a node pool or an etcd. Our piece on Kubernetes multi-tenancy with vCluster covers the in-cluster alternative; this article is about when isolation must be a separate cluster.
Edge: hundreds of small clusters in stores, plants or vehicles that are offline part of the day. If that is your world, pair this with our k3s edge production guide.

The tooling arrived in waves. The first federation attempt, KubeFed, tried to teach one control plane to write objects into many. SIG Multicluster archived it in January 2023 after the model proved too rigid and too tightly coupled to resource types. What replaced it is not one tool but a layered stack: Cluster API for lifecycle, GitOps for configuration, a policy engine for guardrails, and a connectivity layer for the traffic that genuinely has to cross cluster lines.

The layering matters because each concern has a different failure mode and a different blast radius. Lifecycle mistakes destroy clusters. Configuration mistakes break workloads. Policy mistakes block deployments. Networking mistakes create latency and egress bills. Keeping these as separate, individually replaceable layers is what lets you swap a GitOps engine or a mesh without rebuilding the fleet.

There is also a CNCF signal worth noting. Karmada, the multi-cluster scheduler, graduated from the CNCF in September 2026 after joining the sandbox in 2021 and incubating from December 2023. That is a maturity marker for the placement layer, which sits beside, not instead of, Cluster API. We cover the placement decision in the Karmada graduation ADR.

The thesis: separate the four planes

The framing this article argues for is simple. A fleet has four planes, and mixing them is the root cause of most fleet pain:

Lifecycle plane: creating, upgrading and deleting clusters. Cluster API owns this.
Configuration plane: what runs on each cluster. GitOps owns this.
Policy plane: what is allowed to run. Admission policy owns this.
Connectivity plane: how clusters talk, and how users reach them. Mesh, CNI and DNS own this.

Most failed fleets bolt lifecycle into the configuration plane (a Terraform pipeline that also runs Helm) or configuration into the lifecycle plane (cloud-init scripts that install add-ons). Both couplings make upgrades terrifying, because one change touches two failure modes at once. Everything below keeps the planes apart.

Reference Architecture: Hub, Spokes and Four Planes

A production 2026 fleet is a hub-and-spoke system. One management cluster (the hub) runs the controllers that create, upgrade and govern every workload cluster (the spokes). Spokes run tenant workloads plus minimal agents. Git is the source of truth, the hub is the single declarative surface, and spokes are disposable and reconstructible from definitions alone.

Figure 1: The management cluster consumes fleet configuration from Git, drives infrastructure providers to build regional and edge spokes, distributes policy through the GitOps engine, and aggregates telemetry into a fleet inventory.

Figure 1 shows the four responsibilities on the hub. Cluster API turns declarative Cluster objects into real infrastructure through providers. A GitOps engine reconciles add-on and application manifests onto each spoke. A policy engine distributes admission rules. An observability layer pulls metrics, inventory and cost data back, so the fleet is legible from one place. Spokes hold workloads and lightweight agents; if the hub vanishes, spokes keep serving traffic. They simply stop changing.

The hub owns lifecycle, not runtime

The most important boundary in the design is that the hub is a control plane for clusters, not a data plane for workloads. No production request flows through it. That is what allows the hub to be small, heavily guarded and occasionally offline for maintenance without a customer noticing. A wedged hub means you cannot create or upgrade clusters for a while. It does not mean an outage.

Treat the hub as a tier-0 service anyway. Back up its etcd, restrict who can touch it, and make it self-managed where you can: a CAPI management cluster that has been “pivoted” with clusterctl move so that it manages its own definition. Then a lost hub is a rebuild from Git, not an archaeology project. If you also need workload-level backup and restore for the spokes, our Velero backup and disaster recovery runbook covers that layer.

Spokes are declarative and disposable

A spoke should be reconstructible from its Cluster definition plus the Git repo that configures it. If rebuilding a cluster needs tribal knowledge or a runbook of manual kubectl steps, you have a snowflake, and snowflakes are exactly what fleets exist to abolish.

The test is blunt: could you delete a spoke and have it return, fully configured and serving, from Git alone? If yes, upgrades and disaster recovery become the same mechanism. If no, fix that before you increase the cluster count. Stateful data is the exception, and it should live outside the cluster lifecycle (managed databases, replicated object storage, or backups) so that deleting a cluster never deletes the only copy of anything.

Pick the connectivity model first: push or pull

Cluster API handles lifecycle. “Fleet management” in the sense of placement, propagation and status aggregation is a separate concern with real contenders. Karmada offers a Kubernetes-native API to schedule workloads across clusters with override policies. Open Cluster Management uses a hub-and-agent pull model suited to clusters behind firewalls. Rancher Fleet scales GitOps to large cluster counts. Cloud vendors ship their own managed fleets, such as Azure Kubernetes Fleet Manager and Google’s GKE fleet features. Choose by connectivity model and by how much you want to own.

The connectivity axis is the one teams underweight. A push hub, such as Argo CD reaching out to registered clusters, needs network reachability and credentials for every spoke’s API server. That is clean inside one cloud network, awkward across accounts and impossible for edge sites behind NAT. A pull model, such as an OCM klusterlet or Flux running inside each spoke, inverts the trust: the spoke dials home and reconciles locally, so the hub never needs an inbound route to the spoke.

Concern	Push hub (Argo CD style)	Pull agent (Flux, OCM style)
Network requirement	Hub reaches every spoke API	Spoke reaches Git and the hub only
Credentials on hub	Per-spoke cluster credentials	None to spokes; hub holds read access to Git
Behind NAT or firewall	Poor	Good
Central visibility	Excellent, one UI	Needs an aggregation layer
Blast radius of hub compromise	Can act on all spokes	Limited to what the repo can express
Best fit	Cloud spokes you fully control	Edge, multi-account and regulated spokes

For fleets that mix cloud regions with firewalled edge sites, pull is usually the only model that survives contact with real networks. The hybrid is common and legitimate: push for cloud spokes, pull agents at the edge, one Git repository behind both. Our Argo CD versus Flux decision record walks through the trade-offs between the two engines.

Fleet observability and cost are hub concerns

A fleet you cannot see is a fleet you cannot operate. Each spoke has its own metrics, logs and API events, and none of that is visible from the hub by default. The standard pattern is a two-tier pipeline: a lightweight agent per spoke (a Prometheus agent or an OpenTelemetry collector) remote-writes metrics and forwards logs to a central store, with a cluster label on everything so any query can slice by spoke. Thanos or Mimir on the hub side give a global query view without co-locating storage.

The non-obvious requirement is a fleet inventory: a live list of which clusters exist, their Kubernetes versions, add-on sets and health, so drift and orphaned clusters surface instead of hiding. Cost follows the same shape. Per-cluster and per-namespace attribution rolled up to a fleet total is essential, because the characteristic failure of fleets is not one expensive cluster but fifty slightly over-provisioned ones that nobody is watching. For the attribution mechanics see our OpenCost allocation and rightsizing guide.

Cluster API Deep-Dive: Objects, ClusterClass and the 2026 Changes

Cluster API is the SIG Cluster Lifecycle project that makes cluster creation a declarative Kubernetes operation. You install the CAPI controllers on a management cluster, apply a Cluster custom resource, and controllers provision infrastructure and bootstrap Kubernetes. The model is deliberately familiar: the Cluster API book describes it as the same reconcile-to-desired-state pattern you already use for Pods, applied to machines and clusters.

Where the project stands in October 2026

Version facts date quickly, so here is the state as verified for this article. Cluster API v1.14.0 was released on 11 August 2026, and the project’s release notes list support for Kubernetes v1.33 to v1.36 on the management cluster and v1.31 to v1.36 on workload clusters. The v1.14.1 patch was planned to add Kubernetes 1.37 support, and Kubernetes 1.37 is now the current upstream minor release, so confirm the exact support matrix for the patch you deploy.

The project cuts roughly three minor releases a year and supports the latest two minors, with each minor supported for about twelve months. Two API facts matter for anything you write today. The v1beta2 API is the supported version, and v1beta1 has been deprecated since v1.11 and is on track to stop being served in v1.16, which the project’s version-support page places in April 2027. If your Git repository still holds cluster.x-k8s.io/v1beta1 manifests, you have a dated migration to schedule.

Two features from v1.12 (January 2026) changed the day-2 story and are worth understanding properly:

In-place updates. Cluster API can now choose between an immutable rollout (create a new Machine, delete the old one) and an in-place update delegated to an update extension built on the runtime SDK. Both KubeadmControlPlane and MachineDeployment support it. It is meant for changes that would not drain a node or restart pods anyway, such as rotating a Machine’s user credentials. If a change will disrupt workloads regardless, a normal rollout is still preferable.
Chained upgrades. For clusters using managed topologies, you set the target Kubernetes version and Cluster API computes an upgrade plan, stepping through each required minor version in the correct order. A cluster on v1.33 can be pointed at v1.35 in one change. Workers skip intermediate minors where the skew policy allows. The project is explicit that this suits teams that upgrade rarely and is not a reason to stop patching.

A note on scope. The v1.12 announcement describes both features as part of ongoing work, so treat the extension APIs as something to pin and test rather than assume stable forever. I have not benchmarked either feature, and no performance claims are made here.

Providers: the part that makes it portable

CAPI’s power is its provider model. The core controllers are cloud-agnostic and delegate infrastructure specifics to pluggable providers, in three categories.

Infrastructure providers create compute and networking: CAPA for AWS, CAPZ for Azure, CAPG for Google Cloud, CAPV for vSphere, plus bare-metal and OpenStack providers.
Bootstrap providers turn a machine into a Kubernetes node. The default, CABPK, renders cloud-init from kubeadm configuration.
Control-plane providers manage the control plane as a unit. KubeadmControlPlane (KCP) is the reference implementation; managed variants defer to EKS, AKS or GKE control planes instead.

Add-ons have their own provider type. The Cluster API Add-on Provider for Helm (CAAPH) installs Helm charts onto workload clusters selected by label through HelmChartProxy resources, which is a reasonable way to land a CNI before GitOps takes over. Whether you use it or let your GitOps engine do the job is a matter of taste; the important rule is to choose one owner per add-on.

The object model: Machine, MachineSet, MachineDeployment

CAPI mirrors the workload API on purpose. A Machine is one node’s declarative record, the cluster equivalent of a Pod. A MachineSet guarantees N replicas of identical Machines, like a ReplicaSet. A MachineDeployment manages MachineSets to give you rolling, versioned node updates, exactly as a Deployment does for Pods. Change the Kubernetes version or machine image in a MachineDeployment and CAPI rolls new Machines in and old ones out, honouring your surge and unavailability settings.

Figure 2: A ClusterClass templates the topology. The Cluster states desired version and replicas. Controllers reconcile the control plane and MachineDeployments down to individual Machines, each backed by an infrastructure machine and a bootstrap config. MachineHealthCheck remediates unhealthy Machines.

Figure 2 traces the hierarchy. The control plane has its own analogue: KCP rolls control-plane nodes one at a time, waiting for etcd and API server health between each. That sequencing is why a CAPI control-plane upgrade is usually uneventful and why running three control-plane replicas (so quorum survives one node being replaced) is the baseline for production spokes.

ClusterClass and managed topologies

Writing the full set of objects per cluster is how fleets rot. A ClusterClass is a reusable template capturing the shape of a cluster: control-plane class, worker classes, provider template references, variables and patches. With managed topologies, you declare a Cluster that references a ClusterClass and sets a handful of values, and CAPI generates and maintains all the underlying objects.

That is the difference between templating a hundred clusters and hand-authoring a hundred clusters. Variables are typed and validated with OpenAPI schemas, and patches inject variable values into the underlying templates, so a field like instance type or subnet becomes a single validated knob rather than an edit to five YAML files. A ClusterClass change can propagate to every cluster that references it, which is powerful and dangerous in equal measure; the rollout controls in the upgrade section exist for that reason.

apiVersion: cluster.x-k8s.io/v1beta2
kind: Cluster
metadata:
  name: eu-west-prod-03
  labels:
    region: eu-west
    tier: prod
spec:
  topology:
    classRef:
      name: standard-regional
    version: v1.36.2
    controlPlane:
      replicas: 3
    workers:
      machineDeployments:
        - class: general-worker
          name: md-0
          replicas: 6
    variables:
      - name: instanceType
        value: m6i.xlarge

Check the exact v1beta2 field spellings against the API reference for your installed version, because the v1beta2 shape differs from v1beta1 in places (for example, how the class reference is expressed). The sketch above is the intent; your provider’s quick start is the authority.

The reconcile loop is the self-healing

Everything above is driven by the standard controller pattern: observe actual state, compare to desired state, act to close the gap, record status, requeue. If an infrastructure VM dies, its Machine becomes unhealthy, the MachineSet notices the replica shortfall and creates a replacement, the infrastructure provider provisions a new VM and the bootstrap provider joins it.

MachineHealthCheck formalises this by remediating nodes that fail health criteria for a configured duration. The loop never finishes. It continuously converges, which is what makes a fleet self-healing rather than merely automated. Set MachineHealthCheck thresholds conservatively, with a cap on the percentage of machines it may remediate at once, so a regional network blip does not trigger a mass replacement.

Worked example: how big does the hub need to be?

Hub capacity is the question every team asks and few answer with arithmetic. The following is a derivation from the object model, not a benchmark, and the per-object costs are assumptions you should measure in your own environment.

Take a spoke with 3 control-plane Machines and 6 workers, so 9 Machines. Each Machine typically carries an infrastructure machine object and a bootstrap config, which gives about 27 objects. Add the Cluster, its infrastructure cluster object, KCP, its template, one MachineDeployment, MachineSet(s) and their templates, and a MachineHealthCheck: roughly another 8 to 12. Call it 40 CAPI objects per cluster as a planning figure. Then add GitOps objects (one Application or Kustomization per add-on bundle, say 10) and fleet inventory records.

At 200 clusters that is on the order of 10,000 objects before status churn, which is trivial storage for etcd but non-trivial watch and reconcile load, because every controller re-queues on status changes and every Machine reconciles against a remote workload cluster API. The practical limits arrive as controller concurrency and remote API latency long before etcd size. That is why the sharding advice later in this article is stated in terms of failure-domain size and reconcile fan-out, not a magic cluster count.

Rolling Configuration with GitOps

Lifecycle gets you empty clusters. Configuration fills them, and at fleet scale it must be pull-based and templated. Two engines dominate. Argo CD, whose ApplicationSet controller templates one Application per target cluster, is the push-style choice with an excellent UI. Flux, whose Kustomization and HelmRelease controllers run on the cluster and reconcile from Git, is the natural pull-style choice. As of this writing the Argo CD 3.x line is current (the 3.5 release candidate was reported in June 2026), and Flux’s documentation lists v2.9 as its newest minor, so check release pages before you pin versions.

Figure 3: A pull request bumps desired state in the fleet repo. The hub rolls a canary first, then a small cohort, then the rest. Each gate checks health signals, and any failure halts the rollout. A Git revert is the rollback.

Figure 3 is the pattern for any fleet-wide change, whether it is a Kubernetes version, a ClusterClass revision or an add-on bump. The ApplicationSet clusters generator reads clusters registered with Argo CD and emits an Application per cluster, substituting per-cluster values from labels. Add a spoke, label it, and it inherits the whole add-on baseline with no new manifests. Flux reaches the same result with a per-cluster Kustomization overlay or a shared base with patches.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: platform-baseline
  namespace: argocd
spec:
  goTemplate: true
  generators:
    - clusters:
        selector:
          matchLabels:
            tier: prod
  template:
    metadata:
      name: 'baseline-{{.name}}'
    spec:
      project: platform
      source:
        repoURL: https://git.example.com/fleet/addons
        targetRevision: main
        path: baseline
      destination:
        server: '{{.server}}'
        namespace: platform-system
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

One operational note: prune: true on a fleet-wide ApplicationSet is a loaded gun, because a bad path or emptied directory deletes resources on every matching cluster at once. Use separate ApplicationSets per ring (or progressive sync) so production trails the canary.

For an industrial example where hundreds of sites are the clusters, see our Argo CD and Flux GitOps for industrial fleets tutorial, and for the upgrade mechanics of the engine itself see the Argo CD 3 upgrade guide.

Who owns the CAPI objects themselves?

A subtle but important design choice is whether the CAPI Cluster objects are themselves delivered by GitOps. They should be. Cluster definitions in Git, reconciled onto the hub by a GitOps controller, give cluster lifecycle and cluster configuration one audit trail and one rollback path. The loop closes neatly: Git defines the clusters, CAPI builds them, GitOps configures them, and the whole thing is reproducible from a clone.

The catch is bootstrapping. The first GitOps controller on a new spoke has to arrive from somewhere. Common answers are a ClusterResourceSet that applies a small bootstrap manifest, a CAAPH HelmChartProxy that installs the agent, or the hub’s GitOps engine pushing a bootstrap Application. Pick one and document it, because it is the step that most often gets handled by hand and forgotten.

Policy and add-ons as fleet concerns

Add-ons (CNI, CSI, ingress or gateway, DNS, metrics) travel on the same GitOps mechanism, but policy deserves its own engine. Kyverno and OPA Gatekeeper enforce admission rules, such as required labels, banned latest tags, forbidden host mounts and resource-limit floors, as resources synced to every cluster. The fleet win is uniformity: one policy bundle in Git guarantees identical guardrails everywhere, and an auditor can read the enforced posture from one repository.

Since Kubernetes has stabilised CEL-based admission, consider whether a webhook-free policy suffices for simple rules; our guide to ValidatingAdmissionPolicy and CEL covers when it does. Fewer admission webhooks per spoke means fewer ways for a spoke’s API server to wedge, which is worth real money at fleet scale. Placement managers such as OCM and Karmada add selection APIs on top, so “this policy applies only to PCI-labelled clusters” becomes declarative rather than manual.

Day-2 Operations: Upgrades, Rings and Hub Sharding

Day 1 is provisioning. Day 2 is where fleets are won or lost, and the dominant day-2 task is the Kubernetes upgrade, which arrives every four months or so whether you are ready or not. Kubernetes 1.34 reaches end of life today, 27 October 2026, according to the Kubernetes releases page, which lists 1.37, 1.36 and 1.35 as the maintained branches. Any spoke still on 1.34 is now on an unpatched version, which is a good concrete reason to have an upgrade machine that works.

The upgrade mechanics

An upgrade in CAPI is a declarative version bump: change spec.topology.version on the Cluster (or the ClusterClass for a class-wide change). KCP then rolls control-plane nodes one at a time, checking etcd and API server health between each, and MachineDeployments perform a surge-and-drain rollout of workers. This respects Kubernetes version skew rules: control plane first, then nodes, never the reverse. With chained upgrades, a cluster several minors behind gets a computed plan instead of a series of manual bumps.

The fleet discipline is never to bump everything at once. Define rings: a canary cluster that mirrors production shape, then a small cohort, then the long tail in batches. Gate each ring on real signals rather than on elapsed time alone: API server error rate, node-ready percentage, workload SLO burn rate and add-on health. Because the version lives in Git, a bad upgrade is a revert.

A caution on that last sentence. Kubernetes control-plane upgrades are not freely reversible; etcd schema and API storage versions can move forward. “Revert” means reverting the desired state so that the fleet stops advancing and worker nodes can be replaced with the prior image, not that every component can be rolled back mid-flight. The honest rollback story is: halt early, canary rigorously, and keep a tested restore for the control plane. Test that restore at least once per quarter.

Hub scaling and sharding

Scaling to hundreds of clusters stresses the hub rather than the spokes. The management cluster’s API server and etcd carry the CAPI object graph (tens of objects per cluster, as computed earlier), the GitOps controllers’ objects, and whatever inventory you keep. Controllers on the hub also hold remote client connections and caches for each workload cluster, which is why Cluster API v1.14’s reported ClusterCache tuning options and memory-footprint work are relevant to large installations.

Past a few hundred clusters, or earlier if your risk appetite is lower, shard the hubs: one management cluster per region or business unit, each treated as tier-0, with a thin aggregation layer for a global view. Sharding bounds control-plane blast radius too. A wedged hub degrades one shard’s lifecycle operations, not the whole fleet’s. The decision rule I recommend is by failure domain: shard by whatever boundary you would be unwilling to lose lifecycle control across at the same moment, which is usually a region, a regulatory zone or an environment tier.

Hosted control planes and the cost of three nodes

One cost driver deserves explicit arithmetic. A conventional KCP spoke uses three control-plane nodes. A hundred spokes therefore means 300 control-plane machines that do no tenant work. For small edge or dev clusters, that overhead can exceed the workload itself. Options include hosted control planes (the managed control planes of EKS, AKS or GKE through CAPI managed providers), or community projects that run control planes as pods on a shared cluster. These trade isolation and upgrade independence for density, so use them for low-criticality or numerous small clusters rather than for your regulated tier. The right answer is often a mix: dedicated KCP for the few tier-one spokes, managed or hosted control planes for the long tail.

Cross-Cluster Networking, Identity and Secrets

Multi-cluster is not free isolation; it is a new distributed system with its own failure modes, and the first one is networking, which does not span clusters by default. Pods in cluster A cannot reach Services in cluster B without deliberate plumbing.

Figure 4: A global load balancer fronts two regional clusters. Cilium Cluster Mesh and service export let pods resolve and fail over to the other region, while SPIFFE trust bundles and a central secrets manager provide cross-cluster identity without copied credentials.

Three ways to connect clusters

Three approaches compete, and they sit at different layers.

Cilium Cluster Mesh connects clusters at the CNI layer with global services and identity-aware policy. Per the Cilium documentation, the default limit is 255 connected clusters, and raising maxConnectedClusters to 511 halves the cluster-local identity space from 65,535 to 32,767. The value must match on every cluster and be set at install time. Pod CIDRs must be unique across clusters, nodes need IP connectivity between clusters, all clusters must use the same datapath mode, and Cilium versions may differ by no more than one minor release. Treat that last rule as a fleet-upgrade constraint: it forces you to roll the mesh in lockstep rings.
Submariner builds encrypted tunnels between cluster networks and handles overlapping-address cases through its own mechanisms.
The Multi-Cluster Services (MCS) API from SIG Multicluster standardises ServiceExport and ServiceImport. A Service exported in one cluster becomes discoverable in other clusters in the same ClusterSet, relying on namespace sameness: the same namespace name means the same logical namespace across the set. Implementations differ in maturity, so verify which one your CNI or mesh provides. The Cilium setup page mentions an optional mcsapi setting with CoreDNS auto-configuration; check the current docs for its status before depending on it.

Service meshes add a fourth option. Istio’s ambient mode now supports multi-cluster topologies; see our walk-through of Istio ambient multicluster with Gateway API for one worked example. Whichever you choose, the gotcha is the same: cross-cluster calls cross real network boundaries with real latency and real egress cost. A naive global service can turn a same-zone hop into an inter-region round trip that you pay for on every request. Prefer local-first routing with remote failover, and measure the cross-region share of traffic as a first-class metric.

Identity: each cluster is its own trust domain

Each cluster has its own service-account issuer and certificate authority. There is no fleet-wide workload identity out of the box, so a workload in cluster A has no inherent, verifiable identity that cluster B will accept. Teams close the gap in three ways, and they compose.

SPIFFE and SPIRE issue short-lived, verifiable workload identities (SVIDs). A mesh can trust them across cluster boundaries, giving mutual TLS between spokes without shared long-lived credentials. Our comparison of Kubernetes pod certificates, SPIFFE/SPIRE and cert-manager covers when each fits.
A central secrets manager such as HashiCorp Vault or a cloud KMS holds the source of truth. Each cluster authenticates with its own service-account token, validated against the cluster’s OIDC issuer, and pulls secrets at runtime.
External Secrets Operator runs in each spoke and syncs named secrets from the central store into native Secrets on demand, so Git never holds secret material.

The anti-pattern is copying a static secret into every cluster’s repository, even sealed or SOPS-encrypted. Rotation then means a fleet-wide commit and re-sync, and one leaked decryption key compromises every cluster at once. Prefer per-cluster authentication to a central authority over per-cluster copies of a shared secret.

The same logic applies to the credentials CAPI itself uses to talk to cloud providers. Scope them per management cluster, rotate them, and never let a spoke hold keys that could provision more infrastructure. Where the cloud supports it, use workload-identity federation for the hub’s own cloud access rather than static keys.

Trade-offs, Gotchas and What Goes Wrong

Multi-cluster trades one big failure domain for many small ones, and the bill arrives as operational complexity. The failure modes below are the ones I would plan for explicitly.

Version skew and drift. Kubernetes tolerates limited skew between the control plane and kubelets (recent releases allow kubelets to be up to three minor versions behind the API server). Across a fleet, an aborted or half-applied upgrade leaves clusters straddling versions, and a CRD that exists on some clusters but not others makes your supposedly identical GitOps manifests fail on a subset. Fleet inventory with per-cluster version and add-on drift alerts is how you find the three clusters that silently diverged.

The API deprecation clock. Because v1beta1 stops being served in CAPI v1.16, every tool that reads or writes cluster.x-k8s.io/v1beta1 (dashboards, scripts, Terraform providers, custom controllers) has a deadline. Inventory those consumers now. The project’s guidance to prefer Patch over Update when clients lag behind API versions is worth following, since Update can drop fields an older client does not know.

Hub as single point of change. A compromised or misconfigured hub can delete or alter every spoke. Mitigate with branch protection and required reviews on the fleet repo, separate credentials per environment tier, restricted RBAC on the hub, and deletion protection (such as cloud-level termination protection on production control planes). Consider an explicit approval step for any change that touches ClusterClass objects, because their blast radius is every referencing cluster.

ClusterClass coupling. A shared class is a force multiplier and a shared dependency. Editing it can trigger rolling updates on every cluster that uses it. Version your classes (standard-regional-v3) and migrate clusters between them deliberately rather than mutating one class in place.

Mass remediation. A MachineHealthCheck without a remediation cap can respond to a network partition or a bad node image by deleting healthy-but-unreachable machines. Use the unhealthy-range limit, and alert when remediation rates spike.

Cross-cluster chattiness. Global services make it trivially easy to introduce hidden dependencies between regions. A regional outage then cascades through calls nobody realised crossed a cluster boundary. Keep latency-sensitive paths cluster-local and make every cross-cluster dependency explicit and reviewed.

None of these are reasons to avoid a fleet; they are reasons to build the guardrails first. The common thread is that each failure is amplified by fleet scale, so each guardrail pays back more at a hundred clusters than at five.

Practical Recommendations

Start with the hub and make it recoverable before making it clever. A self-managed CAPI management cluster whose own definition and every spoke live in Git means a hub loss is a rebuild, not a resume-writing event. Adopt ClusterClass from the first cluster, even if you only have two, because retrofitting managed topologies onto hand-rolled clusters is painful. Keep the number of distinct cluster shapes small. Variety, not count, is the real cost driver: a hundred identical clusters is cheaper to run than ten bespoke ones.

Push everything through GitOps, including the CAPI objects, so lifecycle and configuration share one audit trail and one rollback mechanism. Treat cross-cluster networking as a deliberate, minimised surface and connect only the services that truly need it. Invest early in inventory and drift detection, because you cannot manage what you cannot see across a hundred API servers. Finally, schedule the v1beta1 to v1beta2 migration now, since it is cheap today and urgent in 2027.

Checklist for a production-ready fleet:

[ ] Management cluster is self-managed, its etcd is backed up and restore is tested.
[ ] Every spoke reconstructs from its Cluster object plus Git configuration, with no snowflakes.
[ ] ClusterClass with managed topologies is in use, versioned, and the number of shapes is small.
[ ] CAPI objects, add-ons and policy are all delivered through GitOps with ring-based rollout.
[ ] Manifests use cluster.x-k8s.io/v1beta2, and every v1beta1 consumer is inventoried.
[ ] MachineHealthChecks have remediation caps; canary clusters are production-representative.
[ ] A Kyverno, Gatekeeper or CEL policy bundle is enforced fleet-wide from one repo.
[ ] Cross-cluster connectivity is explicit, encrypted and latency-aware; no static shared secrets in Git.
[ ] Per-cluster version and add-on drift is monitored; cost is attributed per cluster.
[ ] Hub sharding boundaries are decided before the hub is large enough to need them.

Frequently Asked Questions

What is Cluster API and how does it manage multiple Kubernetes clusters?

Cluster API is a Kubernetes SIG Cluster Lifecycle project that manages clusters declaratively. You run controllers on a management cluster and describe each workload cluster as custom resources: a Cluster, a control plane, and MachineDeployments. Provider plugins for AWS, Azure, Google Cloud, vSphere and others create the infrastructure, and controllers continuously reconcile reality to your definition. Scaling, upgrades and repairs become edits to those objects, ideally pull requests in Git.

What is ClusterClass in Cluster API and why use it?

ClusterClass is a reusable template for the shape of a cluster: control-plane class, worker classes, provider templates, typed variables and patches. A Cluster references it and sets a few values, such as version, replica counts and region, and CAPI generates and maintains all underlying objects. It is how you avoid hand-authoring a hundred near-identical clusters, and it lets a single class revision roll out across every cluster that references it.

Do I need Karmada or Open Cluster Management if I use Cluster API?

Not necessarily. Cluster API handles lifecycle: creating, upgrading and repairing clusters. Karmada and OCM handle placement and propagation: deciding which cluster runs which workload and aggregating status. Many fleets use GitOps with per-cluster overlays and need nothing more. Add a placement layer when you want one API that schedules a workload across clusters, fails it over, or applies override policies by cluster labels.

How many Kubernetes clusters can one Cluster API management cluster handle?

There is no published hard number, and I will not invent one. Capacity depends on object count per cluster, controller concurrency, remote API latency and your GitOps load. A planning estimate of about 40 CAPI objects per cluster suggests the limits show up as reconcile load rather than storage. Load-test your own hub and shard by failure domain, such as region or environment, well before you reach that point.

What changed in Cluster API in 2026?

Cluster API v1.12 in January 2026 introduced in-place updates through update extensions and chained upgrades that let managed-topology clusters jump several minor versions with a computed plan. v1.13 followed in April 2026, and v1.14.0 arrived on 11 August 2026 with Go 1.26, a separate API types module and observability improvements. v1beta2 is the supported API, and v1beta1 is slated to stop being served in v1.16.

Argo CD or Flux for a multi-cluster fleet?

Argo CD’s ApplicationSet generators and central UI suit fleets where the hub can reach every cluster. Flux’s in-cluster controllers suit pull-based, firewalled or edge fleets where spokes dial home. Both work with Cluster API, and hybrids are common. Decide on the connectivity model first, then on team familiarity and the UI you need. Our decision record compares them in detail.

References

Cluster API v1.12: Introducing In-place Updates and Chained Upgrades, Kubernetes Blog, 27 January 2026.
Cluster API v1.14.0 release notes and the Cluster API version support page, kubernetes-sigs/cluster-api.
Cluster API documentation.
Kubernetes releases and large-cluster considerations.
Cilium Cluster Mesh setup.
Multi-Cluster Services API concepts, SIG Multicluster, and Archiving KubeFed.
CNCF announces Karmada graduation, September 2026.
Argo CD 3.5 coverage, InfoQ, June 2026, and Flux releases.

Last Updated: October 2026

By Riju — about