GPU Sharing on Kubernetes: MIG vs MPS vs Time-Slicing vs vGPU
A single H100 can cost more per month than a small team’s salary, and the typical inference pod uses a fraction of it. A 3B-parameter classifier that peaks at 12 percent SM occupancy still gets a whole nvidia.com/gpu: 1 from the scheduler, because Kubernetes treats extended resources as indivisible integers. The result is fleets of expensive accelerators idling at single-digit utilization while the queue of teams waiting for capacity grows.
GPU sharing on Kubernetes is the set of techniques that break that integer: MIG partitions silicon, MPS merges CUDA contexts, time-slicing multiplexes in time, and vGPU virtualizes through a hypervisor. They are not interchangeable. Each one makes a different promise about memory, faults, and latency, and picking the wrong one turns a cost saving into a 3 a.m. incident where one tenant’s out-of-memory error kills another tenant’s model server.
This post gives you the mental model, the actual configuration for each mechanism using the NVIDIA GPU Operator and device plugin, the utilization arithmetic, and a decision matrix you can defend in a design review.
What this covers: the isolation model of each approach, operator and device-plugin configuration, MIG profiles and reconfiguration, MPS limits, time-slicing replicas, where Dynamic Resource Allocation fits, noisy-neighbor behavior, and a worked utilization calculation.
Context and Background
Kubernetes schedules GPUs through the device plugin framework. A plugin advertises an extended resource such as nvidia.com/gpu with an integer capacity per node, and the kubelet hands whole devices to containers. There is no native notion of “half a GPU” or “8 GiB of device memory”. That design suited training, where one job wants every SM and every byte of HBM. It suits inference poorly, because production inference is dominated by small and medium models, bursty request arrival, and many independent services.
Three forces made sharing urgent. First, models got smaller where it counts: distilled rerankers, embedding models, speech encoders, and quantized 7B-class LLMs fit comfortably in 10 to 20 GB. Second, GPUs got larger: an 80 GB part is mostly empty when it serves a single small model. Third, platform teams adopted multi-tenant clusters, so several product teams now contend for one pool. For a platform team the question is no longer “can we share” but “which sharing contract can we afford to offer”.
NVIDIA ships four mechanisms, and the first thing to understand is that they sit at different layers of the stack:
- Multi-Instance GPU (MIG) is a hardware feature on Ampere and newer data-center parts. It carves one GPU into as many as seven isolated GPU instances, each with dedicated SMs, L2 cache slices, and memory controllers. NVIDIA’s MIG user guide describes these as isolated instances with dedicated compute and memory and guaranteed performance.
- CUDA Multi-Process Service (MPS) is a user-space server that funnels many client processes into a single GPU context so their kernels run concurrently rather than being time-multiplexed.
- Time-slicing is the default scheduling behavior of the GPU across contexts, exposed in Kubernetes as “replicas” of the same device.
- vGPU is NVIDIA’s virtualization stack, where a hypervisor presents mediated devices to virtual machines, under a commercial license.
If you also run training or batch jobs on the same cluster, sharing interacts with queueing and gang scheduling. Our guide to Slurm vs Kubernetes for AI training with Kueue, Volcano and Slinky covers that side, and it is worth reading before you decide to put fractional inference and whole-GPU training on the same nodes. The newer allocation API that is slowly replacing the device plugin model, Dynamic Resource Allocation, is covered in our DRA GPU scheduling guide and appears again below.
A note on sources. Configuration syntax in this post was checked against NVIDIA’s GPU Operator documentation, the k8s-device-plugin README (latest release shown there: v0.17.1 at the time of writing), the MPS documentation, and the NVIDIA DRA driver guide. Version-specific behavior changes quickly, so treat any flag name as something to confirm against your installed operator version. Numbers marked “illustrative” are worked examples, not measurements.
The Isolation Model: What Each Mechanism Actually Guarantees
The most useful way to compare these techniques is not by features but by the three failure domains you care about in a multi-tenant system: memory, faults, and quality of service (QoS). A mechanism that fails any of the three cannot host untrusted tenants, regardless of how cheap it is.
Direct answer: MIG gives hardware-enforced memory, fault, and QoS isolation; vGPU gives strong isolation at VM granularity; MPS gives concurrency with configurable but softer limits and a shared fault domain; time-slicing gives no isolation beyond process boundaries, no memory limits, and no QoS guarantees. Choose by trust level and latency target first, cost second.
Figure 1: The four GPU sharing mechanisms mapped to the layer where sharing happens and the isolation each one yields.
Figure 1 shows the shape of the trade-off. Moving from MIG toward time-slicing, you gain flexibility and hardware compatibility and lose isolation. The sections below take each failure domain in turn.
Memory isolation
With MIG, each GPU instance owns a slice of the memory system. A process in one instance cannot allocate, read, or exhaust memory belonging to another, and an out-of-memory error stays local. With vGPU, each virtual machine gets a framebuffer of fixed size defined by the vGPU profile, enforced by the vGPU manager.
With time-slicing there is none. The NVIDIA GPU Operator documentation is explicit that time-slicing provides no memory or fault isolation between replicas: every pod on the device sees the whole device memory pool and competes for it. If four replicas each decide to preallocate 70 percent of memory, as many frameworks do by default, three of them will fail at startup or later at a spike.
MPS sits between the two. On Volta and newer GPUs, MPS supports a pinned device memory limit per client through CUDA_MPS_PINNED_DEVICE_MEM_LIMIT, which uses the same device-ordinal-or-UUID syntax as CUDA_VISIBLE_DEVICES, for example 0=2G. The NVIDIA docs note that a client can only tighten the limit set by the control daemon, not raise it. The device plugin’s MPS mode goes further and assigns each replica an equal fraction of device memory: the README says each client is limited to an equal fraction of the total device memory, so ten replicas means a tenth each.
Fault isolation
This is where MPS and time-slicing differ most from MIG. In a time-sliced setup, each pod has its own CUDA context, so a crash of one process does not usually take down another. But a hardware-level error, such as an uncorrectable ECC error or an Xid fault, affects the whole device, and everyone sharing it sees it.
MPS is more fragile because its clients share a server context. The MPS documentation’s description of Volta-and-newer fault handling shows the pattern: when the server handles a fatal GPU error, affected clients are marked with a sticky error and exit, and the server destroys and recreates its contexts on the devices involved. In practice that means a fatal fault caused by one tenant’s kernel can terminate the CUDA state of the other clients attached to that server. Treat MPS as a shared fault domain for design purposes.
MIG isolates faults at the instance level for the memory and compute paths, which is why it is the only fractional option that cloud providers and security reviewers routinely accept for tenant separation. It is not magic: the driver, the PCIe link, and the physical board are still shared, so a driver crash or a GPU reset affects every instance on that GPU.
Quality of service
QoS is about whether tenant A’s load can raise tenant B’s tail latency. MIG partitions are the strongest here because SMs, L2 cache, and memory bandwidth are statically divided. A noisy neighbor in the next instance cannot steal bandwidth.
Time-slicing is the weakest. The device plugin docs state that requesting more than one replica does not grant proportional compute: the GPU time is divided among contexts, and the scheduling quantum, not your request, determines what you get. A latency-sensitive pod sharing with a batch job can wait for the other context’s slice to finish, which adds jitter that shows up directly in p99.
MPS improves throughput because kernels from different clients overlap on the SMs, and its CUDA_MPS_ACTIVE_THREAD_PERCENTAGE setting caps how much of the GPU each client can use. That cap limits the share of SMs a client may occupy, but it does not partition memory bandwidth or L2 cache, so a bandwidth-hungry neighbor can still hurt you.
| Dimension | MIG | MPS | Time-slicing | vGPU |
|---|---|---|---|---|
| Memory isolation | Hardware | Soft limit per client | None | Fixed framebuffer per VM |
| Fault isolation | Per instance | Shared server context | Per context, shared device | Per VM |
| Compute QoS | Static SM partition | Thread percentage cap | None, equal slices | Scheduler policies by profile |
| Bandwidth / L2 isolation | Yes | No | No | Depends on mode |
| Hardware needed | Ampere+ data center | Volta+ for limits | Any CUDA GPU | Supported data-center GPUs |
| Licensing | None beyond driver | None | None | Commercial |
The table is my synthesis of NVIDIA’s documentation; the vGPU row in particular depends on whether you use time-sliced vGPU or MIG-backed vGPU, which changes the QoS story substantially.
Configuring the NVIDIA GPU Operator and Device Plugin
All four mechanisms except vGPU are configured through the same two control points: the GPU Operator’s ClusterPolicy and the NVIDIA device plugin’s configuration file. Understanding the plumbing once makes each mechanism a small variation.
Figure 2: How sharing configuration flows from ConfigMap and node labels through the operator to schedulable resources.
Device plugin configuration plumbing
The device plugin reads a config file with version: v1, an optional flags.migStrategy, and a sharing section. You store named configurations in a ConfigMap in the operator namespace, point the ClusterPolicy at it with spec.devicePlugin.config.name, and optionally set spec.devicePlugin.config.default to the key that applies cluster-wide. For per-node configuration, you omit the default and label nodes with nvidia.com/device-plugin.config=<key>.
One operational detail from the operator documentation is easy to miss: the operator does not watch that ConfigMap. After editing it you must restart the nvidia-device-plugin-daemonset. Running workloads are not affected by the restart, but NVIDIA recommends doing it during a maintenance window because the advertised capacity changes.
apiVersion: v1
kind: ConfigMap
metadata:
name: device-plugin-configs
namespace: gpu-operator
data:
any: |-
version: v1
flags:
migStrategy: none
ts-4: |-
version: v1
flags:
migStrategy: none
sharing:
timeSlicing:
renameByDefault: false
failRequestsGreaterThanOne: true
resources:
- name: nvidia.com/gpu
replicas: 4
mps-4: |-
version: v1
sharing:
mps:
renameByDefault: true
resources:
- name: nvidia.com/gpu
replicas: 4
Apply it to the cluster policy with a merge patch:
kubectl patch clusterpolicies.nvidia.com/cluster-policy \
-n gpu-operator --type merge \
-p '{"spec": {"devicePlugin": {"config": {"name": "device-plugin-configs", "default": "any"}}}}'
# Per-node override: put an inference node on the MPS profile
kubectl label node gpu-node-07 nvidia.com/device-plugin.config=mps-4 --overwrite
kubectl -n gpu-operator rollout restart ds/nvidia-device-plugin-daemonset
The renameByDefault flag decides naming. When true, the shared resource is advertised as nvidia.com/gpu.shared, which lets you keep full-GPU and shared-GPU pools distinguishable in pod specs. When false, the resource name stays nvidia.com/gpu and the node’s nvidia.com/gpu.product label gets a -SHARED suffix, which silently changes what a request for one GPU means. I prefer renaming for production because it makes the sharing contract visible in every manifest and prevents a training job from accidentally landing on a shared slot.
Time-slicing replicas
Time-slicing is the cheapest to turn on and the easiest to misuse. With replicas: 4, a node with 2 physical GPUs advertises 8 units of nvidia.com/gpu. Nothing in the scheduler knows these are fractions; it sees eight integer devices.
apiVersion: apps/v1
kind: Deployment
metadata:
name: reranker
spec:
replicas: 3
selector:
matchLabels: {app: reranker}
template:
metadata:
labels: {app: reranker}
spec:
containers:
- name: server
image: registry.example.com/reranker:1.4
resources:
limits:
nvidia.com/gpu.shared: 1 # one time-sliced replica
Two caveats from the operator docs deserve emphasis. Requesting two replicas does not give you twice the compute; each pod gets an equal share of GPU time regardless of how many replicas it asked for. That is why failRequestsGreaterThanOne: true is worth setting: it makes the plugin reject such requests with an admission error instead of letting a team believe it bought a bigger slice. The second caveat is observability: DCGM-Exporter cannot associate metrics with individual containers when time-slicing is enabled, so per-tenant utilization dashboards go blind.
MPS through the device plugin
MPS support in the device plugin has been marked experimental since v0.15.0 per the README, and it has constraints: it applies to full GPUs only, it supports only nvidia.com/gpu, and it cannot be used on MIG-enabled devices. When enabled, the plugin starts an MPS control daemon on the node, and containers receive a pipe directory mounted into the pod.
Per-client limits are derived from the replica count. With replicas: 4, each client gets one quarter of device memory, and the daemon limits compute accordingly. The README does not spell out the exact thread-percentage computation, so I will not claim one; check the generated daemon environment on your own nodes if you need the precise value.
If you manage MPS yourself outside the plugin, the knobs are environment variables and control commands:
# Start the control daemon (host or sidecar)
export CUDA_MPS_PIPE_DIRECTORY=/tmp/mps
export CUDA_MPS_LOG_DIRECTORY=/var/log/mps
nvidia-smi -i 0 -c EXCLUSIVE_PROCESS
nvidia-cuda-mps-control -d
# Default cap for servers spawned from now on (default is 100)
echo "set_default_active_thread_percentage 25" | nvidia-cuda-mps-control
# In a client container: tighten the limits for this process only
export CUDA_MPS_ACTIVE_THREAD_PERCENTAGE=25
export CUDA_MPS_PINNED_DEVICE_MEM_LIMIT="0=10G"
python serve.py
Per the MPS tools reference, the thread percentage can be set at the daemon, client process, or CUDA context level, and each lower level can only constrain further, never exceed the level above. Putting the GPU in EXCLUSIVE_PROCESS compute mode is the standard prerequisite, and it means the same GPU cannot also be time-sliced.
MIG strategies and the MIG Manager
MIG adds a second controller. The operator’s mig.strategy setting has two values for practical use. single enables MIG on all GPUs of a node with one uniform geometry and still advertises nvidia.com/gpu, which makes pods portable and is ideal for homogeneous inference pools. mixed advertises per-profile resources like nvidia.com/mig-1g.10gb, which lets one node mix profile sizes but requires pods to ask for a specific profile.
MIG Manager watches the node label nvidia.com/mig.config and applies the named configuration with the mig-parted tool. The operator generates a per-node ConfigMap with entries such as all-disabled, all-enabled, all-1g.10gb, all-3g.40gb, and all-balanced, and you can supply a custom ConfigMap with a config.yaml key. A custom geometry looks like this:
version: v1
mig-configs:
inference-mixed:
- devices: [0, 1]
mig-enabled: true
mig-devices:
"1g.10gb": 2
"2g.20gb": 1
"3g.40gb": 1
- devices: [2, 3]
mig-enabled: false
kubectl label node gpu-node-07 nvidia.com/mig.config=inference-mixed --overwrite
kubectl get node gpu-node-07 -L nvidia.com/mig.config.state
The companion label nvidia.com/mig.config.state reports pending, rebooting when a reboot is part of the change, success, or failed. Pods then request profiles under the mixed strategy:
resources:
limits:
nvidia.com/mig-2g.20gb: 1
MIG Profiles, Geometry, and Reconfiguration in Practice
MIG’s strength is the strict geometry, and its weakness is the same geometry. Choosing profiles is a bin-packing problem that you solve up front, and changing the answer later is operationally expensive.
What the profiles look like
From NVIDIA’s supported-profiles tables, an H100 80 GB exposes these profiles: 1g.10gb (up to 7 instances), 1g.10gb+me (1 instance, with media engines), 1g.20gb (up to 4), 2g.20gb (up to 3), 3g.40gb (up to 2), 4g.40gb (1), and 7g.80gb (1). The A100 80 GB uses the same fractions with the same names. The A100 40 GB equivalents are 1g.5gb, 1g.10gb, 2g.10gb, 3g.20gb, 4g.20gb, and 7g.40gb.
Notice the naming: the leading number is the count of compute slices (out of seven) and the suffix is memory. Memory is divided into eighths while compute is in sevenths, which is why you cannot fill a GPU with arbitrary combinations. Seven 1g.10gb instances use 70 GB of the 80 GB, leaving a stranded eighth of memory. Placement constraints also matter: not every combination of profiles can coexist, so a geometry like 3g.40gb plus 4g.40gb works but 3g.40gb plus 3g.40gb plus 1g.10gb needs care. The authoritative check is the supported-configurations table, and the nvidia-smi mig -lgipp command lists valid placements on your device.
Figure 3: The reconfiguration sequence for changing MIG geometry through the GPU Operator.
The reconfiguration cost
Per the operator documentation, when the nvidia.com/mig.config label changes, MIG Manager stops operator-managed GPU pods (device plugin, GPU feature discovery, DCGM exporter), stops any host GPU clients listed in the default-gpu-clients ConfigMap if drivers are preinstalled, enables MIG mode (rebooting the node if required), applies the geometry with mig-parted, restarts the pods, and finally sets the state label. The page states that MIG Manager requires that no user workloads are running on the GPUs being configured, and advises cordoning the node first since a reboot may be needed, as on some cloud providers.
That has direct architectural consequences. You cannot treat MIG geometry as an autoscaler knob that follows request mix minute by minute. Reconfiguring is a node-level maintenance event: cordon, drain, relabel, wait for success, uncordon. A practical pattern is to maintain two or three node pools with fixed geometries, such as a “small” pool of seven 1g.10gb slices and a “medium” pool of 3g.40gb plus 4g.40gb, and let the cluster autoscaler add nodes to whichever pool has demand.
Here is the maintenance runbook as a script:
NODE=gpu-node-07
kubectl cordon $NODE
kubectl drain $NODE --ignore-daemonsets --delete-emptydir-data --timeout=300s
kubectl label node $NODE nvidia.com/mig.config=all-1g.10gb --overwrite
# wait for terminal state
until [[ "$(kubectl get node $NODE -o jsonpath='{.metadata.labels.nvidia\.com/mig\.config\.state}')" =~ ^(success|failed)$ ]]; do
sleep 10
done
kubectl get node $NODE -o jsonpath='{.status.allocatable}' | tr ',' '\n' | grep nvidia
kubectl uncordon $NODE
Sizing a model to a slice
A MIG slice is a hard budget. For inference you need weights plus KV cache or activation workspace plus CUDA context overhead to fit inside the instance. As a rule of thumb, fp16 weights take two bytes per parameter, so a 3B-parameter model needs roughly 6 GB for weights alone (a straightforward calculation, not a measurement). In a 1g.10gb instance that leaves about 4 GB minus overhead for activations and cache, which constrains batch size and context length. If a model fits only barely, a 2g.20gb is the safer choice: the extra memory buys you batch size, and batch size is what turns a slice into useful throughput.
Compute scales roughly with the slice count, but not perfectly: a 1g instance has one seventh of the SMs, so a kernel that saturated a full GPU runs several times slower on it. For latency-bound small models that is fine. For large-batch throughput work, check whether a single full GPU would have finished the work sooner than seven slow instances.
MIG limits worth knowing
- MIG requires supported data-center GPUs (Ampere and later); consumer cards and older Tesla parts cannot use it.
- Peer-to-peer and NVLink between instances are not available in the usual sense, so tensor parallelism across instances is not a use case.
- Inside one instance, multiple processes are time-sliced like on any GPU unless you also run MPS there, so a MIG slice shared by several pods inherits the weaker contract inside that slice. Test this combination on your driver version before relying on it.
- The device plugin’s MPS mode does not support MIG-enabled devices according to the README, so combining them means managing MPS yourself or through another path.
MPS, Time-Slicing, and vGPU in Depth
Why MPS raises utilization
Without MPS, kernels from different processes cannot execute simultaneously on a GPU, because each process has its own context and the hardware time-slices between contexts. A small model that launches short kernels with idle gaps between them leaves most SMs empty during its slice, and the other tenant waits for the slice to end. MPS removes the context boundary: clients connect to a server process that owns one context, and their kernels can occupy different SMs at the same time.
This is the reason MPS often beats time-slicing on aggregate throughput for many small inference services. If four services each use 20 percent of the SMs when running alone, MPS can in principle run them concurrently with little slowdown, while time-slicing serializes them. The gain is real but workload dependent; the shape of the kernels, memory bandwidth demand, and launch patterns decide it, so measure before you commit.
MPS has three limits that matter for design. It is a shared fault domain, as discussed. It has no bandwidth partitioning, so thread percentages bound SM occupancy but not HBM traffic. And its client memory limits are per-client pinned allocations, not a hardware boundary: they stop a well-behaved client from over-allocating, but they are not a security control.
The DRA driver exposes the same knobs declaratively. According to the NVIDIA DRA driver guide, MPS is configured in a GpuConfig under sharing with strategy: MPS and an mpsConfig that can set defaultActiveThreadPercentage (0 to 100), defaultPinnedDeviceMemoryLimit (a quantity such as 4Gi), and per-device overrides. The guide marks the MPSSupport feature gate as alpha and off by default, and notes it cannot be enabled together with DynamicMIG.
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
namespace: team-a
name: mps-quarter
spec:
spec:
devices:
requests:
- name: gpu-share
exactly:
deviceClassName: gpu.nvidia.com
config:
- requests: ["gpu-share"]
opaque:
driver: gpu.nvidia.com
parameters:
apiVersion: resource.nvidia.com/v1beta1
kind: GpuConfig
sharing:
strategy: MPS
mpsConfig:
defaultActiveThreadPercentage: 25
defaultPinnedDeviceMemoryLimit: 4Gi
This example follows the shape published in the driver’s guide. Because the feature is alpha, expect field names and defaults to move between driver releases.
Time-slicing: when the cheap option is correct
Time-slicing deserves more respect than its reputation. It works on every CUDA GPU, including older and consumer-class parts that support neither MIG nor MPS limits. It needs no reconfiguration beyond a plugin restart. And for the right workload it is entirely adequate: development notebooks, CI test runs for model code, demo environments, and internal tools with sparse traffic.
The failure mode is the combination of no memory isolation and framework defaults. PyTorch’s caching allocator holds memory it has once used, and TensorFlow historically grabbed most of the device by default. Four pods on a time-sliced 40 GB GPU can each cache 15 GB at different moments and collide. Mitigate with per-process memory fractions in the application, for example torch.cuda.set_per_process_memory_fraction(0.22), and with admission policy that caps replicas per node. These are conventions, not enforcement, so a misconfigured pod can still break its neighbors.
A second subtlety is scheduling fairness. The scheduler places pods by counting replica resources, not by measuring load. Two heavy pods can land on one GPU while another sits idle, because to Kubernetes the replicas are interchangeable. Pair time-slicing with pod anti-affinity or topology spread constraints on the GPU node name if you care about balance.
vGPU: virtualization with a license
vGPU is a different category. It is not a device plugin option for ordinary containers. According to the GPU Operator’s KubeVirt documentation, a GPU node runs one workload type: containers, passthrough VMs, or vGPU VMs, not a mix. For vGPU nodes the operator deploys a vGPU Manager (which you build from NVIDIA’s vGPU driver package and host in a private registry, since the bundle is not publicly distributed), a vGPU Device Manager, and a sandbox device plugin, driven by the node label nvidia.com/gpu.workload.config=vm-vgpu after enabling sandboxWorkloads. The vGPU Device Manager creates vGPU devices from a named configuration chosen by the nvidia.com/vgpu.config label, and switching configurations requires that no vGPU VMs are running on the node.
So “vGPU Kubernetes” in practice means running VMs through KubeVirt (or a similar layer) that consume mediated devices, with the vGPU type added to the KubeVirt CR under permittedHostDevices.mediatedDevices. That is a good fit when tenants need guest operating systems, strong separation, or Windows and graphics workloads, and when you already pay for NVIDIA’s enterprise licensing. It is a poor fit if your goal is simply to run many container-based model servers cheaply: you add a hypervisor layer, VM start-up time, and license cost to solve a problem MIG solves natively on supported hardware.
The details of vGPU scheduling policy (best effort, equal share, fixed share) and which vGPU profiles are available on a given GPU come from NVIDIA’s vGPU documentation and licensing, and I have not verified them for this post. Check the vGPU release notes for your driver branch before relying on a specific profile or scheduler mode.
Dynamic Resource Allocation and Where It Changes the Picture
The device plugin API was designed for opaque integers. Sharing mechanisms bolted on by renaming resources and multiplying replicas are workarounds for that limit. Dynamic Resource Allocation (DRA) replaces the integer with structured claims: a pod references a ResourceClaim, devices are published as ResourceSlice objects with attributes, and the scheduler can match on device properties, such as memory or MIG profile, and track allocation per device.
For sharing, DRA matters in three ways. It lets a claim describe the sharing strategy explicitly, as the MPS example above shows, instead of encoding it in a node-wide ConfigMap. It lets multiple pods reference the same claim, expressing intentional sharing rather than relying on fake replica counts. And it makes MIG partitions first-class devices that the scheduler can see and allocate, rather than a pre-baked node geometry. The NVIDIA DRA driver also lists a DynamicMIG feature gate in its guides, which points toward on-demand partitioning, though I have not verified its maturity and it is mutually exclusive with MPS support in the guide I checked.
The practical guidance for 2026 is to keep the production sharing path on the device plugin and operator you already operate, and run DRA in a pilot pool where you can learn its failure modes. Our DRA GPU scheduling guide walks through the claim model and driver installation in more detail, and it is the right next read once you have chosen a sharing mechanism.
Noisy Neighbors and Utilization Math
How interference actually happens
Noisy-neighbor effects on a shared GPU come from five contended resources: SM time, memory capacity, memory bandwidth, L2 cache, and copy engines or PCIe bandwidth. Each sharing mechanism partitions a different subset:
- MIG partitions SMs, L2, memory capacity, and memory controllers, so interference through those paths is eliminated. Shared residuals include the PCIe link, host CPU, and the driver.
- MPS partitions SM occupancy only if you set thread percentages, and optionally memory capacity. Bandwidth, L2, and copy engines remain shared.
- Time-slicing partitions time, and nothing else. Within a slice, a tenant has the entire device.
- vGPU depends on mode; time-sliced vGPU shares like time-slicing but with a fixed framebuffer per VM.
Detecting interference requires per-tenant telemetry, which is where time-slicing is weakest because DCGM-Exporter cannot attribute metrics to containers in that mode. With MIG you get per-instance metrics. With MPS you generally rely on application-level metrics such as kernel latency measured in the client, plus server logs.
Worked utilization example (illustrative)
Suppose a platform runs 12 inference services on 12 dedicated H100 GPUs. Measured average SM utilization per service is 9 percent, memory footprint 14 GB each. These are illustrative numbers, but the arithmetic generalizes.
Dedicated: 12 GPUs, aggregate useful work equals 12 x 9 percent = 1.08 GPU-equivalents, so fleet utilization is 1.08 / 12 = 9 percent.
With MIG using 1g.10gb slices, the 14 GB footprint does not fit, so you step up to 2g.20gb (3 per GPU, using 6 of 7 compute slices). Twelve services need 4 GPUs. A 2g instance has 2/7 of the SMs, so a service using 9 percent of a full GPU needs about 31 percent of its slice if work scales linearly, which is a simplifying assumption. Fleet utilization is about 1.08 / 4 = 27 percent, with hard isolation, and the stranded 1/7 of compute per GPU is the price of the geometry.
With time-slicing at 4 replicas per GPU, 12 services need 3 GPUs and 56 GB of memory demand per GPU, which does not fit in 80 GB with 4 x 14 GB plus allocator headroom unless frameworks are bounded. Fleet utilization is 1.08 / 3 = 36 percent, but the p99 impact is unmeasured and memory contention is a hazard.
With MPS at 4 replicas, the same 3 GPUs host the services concurrently with an equal 20 GB memory fraction per client, so the 14 GB models fit, and kernel overlap reduces the serialization penalty. Utilization is again 36 percent, with better tail latency than time-slicing for these small kernels, though without MIG’s isolation.
The lesson is not the specific numbers. It is that utilization gain is bounded by the memory footprint and the geometry before it is bounded by compute, and that every step up in density trades away an isolation guarantee. Compute your own table from real DCGM data: per-service SM activity, memory used, and p99 latency under load, then pick the cheapest mechanism that keeps p99 inside the SLO with the neighbors running.
Also account for cost differently than utilization. If an H100 hour costs C, dedicated serving costs 12C per hour; MIG costs 4C; time-slicing or MPS costs 3C. The relative savings (67 to 75 percent) are the point of the exercise, and they only hold if the packing works on real traffic, which usually has correlated peaks. Provision for the peak overlap, not the average.
A benchmarking protocol you can trust
Do not take published speedups for your workload. Run a controlled test: pin one latency-sensitive service (“victim”) and add a configurable load generator (“aggressor”) on the same GPU under each mechanism. Record victim p50, p99, and p99.9 with the aggressor idle, at 50 percent, and at saturation. Repeat for each mechanism with identical model, batch policy, and clocks. Report the ratio of loaded to idle p99; that single number captures how much isolation you are really buying. Keep the raw data, because the ranking can change with model size and request rate.
Decision Matrix: Choosing a Sharing Mechanism
Start from trust and latency requirements, then hardware, then cost. The flowchart encodes the order I would use in a design review.
Figure 4: A decision flow that starts with tenant trust and SLOs, then hardware support, then workload similarity.
| Scenario | Best fit | Why | Watch out for |
|---|---|---|---|
| External or untrusted tenants on shared cluster | MIG | Hardware isolation of memory, faults, bandwidth | Fixed geometry, drain to reconfigure |
| Strict p99 SLO on small models | MIG | Static partitions remove neighbor jitter | Slice too small for batch size |
| Many tiny services from one team | MPS | Concurrent kernels, simple memory fractions | Shared fault domain, experimental plugin mode |
| Dev, notebooks, CI, demos | Time-slicing | Works on any GPU, trivial to enable | No memory limits, metrics blind |
| VM tenants, guest OS control, existing licenses | vGPU | VM-level separation | License cost, one workload type per node |
| Mixed fleet including pre-Ampere GPUs | Time-slicing or MPS | MIG unavailable | Weakest isolation |
| Large model needing the whole GPU | Dedicated GPU | Sharing adds only overhead | Do not force sharing |
Two hybrid patterns are worth knowing. The first is tiered pools: a MIG pool for production tenants, an MPS or time-sliced pool for internal teams, and dedicated nodes for large models, each tainted and labeled so the pools never mix. The second is MIG for coarse isolation with higher-level sharing inside a slice for a single trusted team, accepting the weaker contract inside the slice.
A Kubernetes-level guardrail for any pool is to make the sharing contract visible. Use a distinct resource name through renameByDefault, add taints so only workloads that opt in can land there, and enforce policy with an admission controller or ResourceQuota on the shared resource name.
apiVersion: v1
kind: ResourceQuota
metadata:
name: shared-gpu-quota
namespace: team-a
spec:
hard:
requests.nvidia.com/gpu.shared: "8"
---
apiVersion: v1
kind: Pod
metadata:
name: embed-server
namespace: team-a
spec:
tolerations:
- key: gpu-pool
value: shared
effect: NoSchedule
nodeSelector:
gpu-pool: shared
containers:
- name: embed
image: registry.example.com/embed:2.1
resources:
limits:
nvidia.com/gpu.shared: 1
env:
- name: PYTORCH_CUDA_ALLOC_CONF
value: "expandable_segments:True"
Spot or preemptible capacity changes the economics again: fractional serving on interruptible nodes needs fast model loading and graceful drain, which is covered in our guide to interruption-tolerant Kubernetes GPU workloads on spot instances.
Trade-offs, Gotchas, and What Goes Wrong
Stranded capacity under MIG. Because compute comes in sevenths and memory in eighths, some geometries waste a slice. If your model mix does not match the profiles, you can pay for hardware isolation and still run below 60 percent of the device. Audit stranded SMs and memory per node, not just allocated slices.
Reconfiguration is an outage for that GPU. MIG Manager stops operator pods and wants the GPUs free of user workloads. Teams that wire it into an autoscaler discover that every geometry change is a drain. Keep geometry static per pool, and treat a change as a release.
Silent semantic changes from renameByDefault: false. Turning on time-slicing without renaming means nvidia.com/gpu: 1 suddenly means “one slice”. A training job that expected a whole GPU lands on a shared one and runs slowly or out of memory. Rename the resource, and set failRequestsGreaterThanOne.
Memory pressure that looks like application bugs. On time-sliced GPUs, out-of-memory errors appear in pods that did nothing wrong, because a neighbor grew. The remedy is operational: per-framework memory caps, replica ceilings, and alerts on device-level memory used rather than pod-level.
MPS blast radius. A fatal fault from one client resets the server’s contexts, and the clients attached to it exit. Run MPS only among tenants who share an on-call rotation and accept shared fate, and keep retry logic in the caller. Also remember that the device plugin’s MPS mode is flagged experimental in its README.
Observability gaps. DCGM-Exporter cannot attribute container metrics under time-slicing. Under MPS, standard GPU metrics aggregate across clients. Build application-level latency and queue-depth metrics so you can see interference even when GPU metrics cannot.
Operator and driver coupling. Behavior changes between operator releases, particularly around MIG Manager’s static versus generated ConfigMaps on older driver branches. Pin versions, test upgrades in a canary pool, and read release notes before bumping.
Security expectations. Only MIG and VM-based vGPU are credible boundaries between mutually distrusting tenants, and even they share a driver and board. If compliance matters, document exactly which resources are shared, and do not describe MPS or time-slicing as isolation to auditors.
Anti-pattern: sharing for its own sake. If a model needs more than half a GPU at its target batch size, sharing introduces jitter for little saving. Consolidate small models into one multi-model server (Triton, vLLM with adapters, or similar) before partitioning hardware, since application-level multiplexing can be cheaper than any of the four mechanisms.
Practical Recommendations
Start with measurement. Collect two weeks of per-service SM activity, framebuffer used, and p99 latency from DCGM on dedicated GPUs. Without it you will pick a mechanism from folklore. Then segment workloads into three groups: those that need isolation (external tenants, strict SLOs), those that tolerate shared fate (one team’s small services), and those that tolerate jitter (dev, batch, CI).
Map groups to pools. Production, multi-tenant inference on A100 or H100 should default to MIG with a small number of fixed geometries. Same-team high-density services can run on an MPS pool once you have tested fault handling and accept the experimental status of the plugin path. Development uses time-slicing with replica ceilings. Reserve vGPU for cases where you already need VMs.
Make the contract explicit in Kubernetes: renamed resources, taints, quotas, and failRequestsGreaterThanOne. Automate the MIG runbook so a geometry change is a reviewed, scripted drain rather than an improvisation. Pilot DRA in one pool to learn claim-based sharing before the device plugin path is retired in your environment.
Checklist:
- [ ] Baseline per-service SM, memory, and p99 on dedicated GPUs.
- [ ] Classify tenants by trust level and SLO.
- [ ] Choose a mechanism per pool using the decision matrix.
- [ ] Use
renameByDefault: trueandfailRequestsGreaterThanOne: truefor time-slicing and MPS. - [ ] Keep MIG geometry static per node pool; script cordon, drain, relabel, verify.
- [ ] Set framework memory caps for every shared pod.
- [ ] Add victim/aggressor tests to CI for each pool.
- [ ] Monitor device-level memory and application p99, not only pod metrics.
- [ ] Pin operator, driver, and device plugin versions; canary upgrades.
- [ ] Document shared fate and fault domains for security review.
Frequently Asked Questions
Is MIG better than MPS for multi-tenant inference?
For tenants that do not trust each other, yes. MIG gives hardware-enforced memory, fault, and bandwidth separation, while MPS shares a server context and a fault domain. MPS can deliver higher aggregate throughput for many small, cooperating services because kernels overlap on the SMs. Choose MIG when isolation or a tight p99 is the requirement, and MPS when density among trusted services is the priority and you accept shared fate.
Can I combine time-slicing with MIG?
The GPU Operator documentation lists MIG strategies and time-slicing as separate configuration areas, and time-slicing supports the MIG resource types from the mixed strategy, so replicas can be applied to MIG devices. You then get hardware isolation between instances but only time-sliced sharing inside each one. It is a reasonable way to host several light pods per slice, provided you accept the weaker contract within the slice.
Does time-slicing give each pod an equal share of GPU memory?
No. Time-slicing divides time, not memory. All replicas see the full device memory pool and compete for it, which is why NVIDIA’s documentation says there is no memory or fault isolation. You must cap memory in each framework, limit replicas per node, and monitor device-level usage. If you need enforced memory fractions, use MPS with limits or MIG instead.
Do I need a license for vGPU on Kubernetes?
vGPU requires NVIDIA’s vGPU software and licensing, and the operator documentation says the vGPU bundle is not publicly distributed, so you build the vGPU Manager image yourself from the downloaded package. It runs VMs through KubeVirt rather than ordinary containers on the same node. MIG, MPS, and time-slicing need no separate sharing license. Confirm current licensing terms with NVIDIA for your hardware and support agreement.
How do I change a MIG profile without downtime?
You cannot change it in place for a GPU in use. MIG Manager needs the target GPUs free of user workloads and may reboot the node. The standard approach is to cordon and drain the node, relabel nvidia.com/mig.config, wait for the state label to read success, and uncordon. For continuity, run multiple nodes per pool and reconfigure them one at a time, using pod disruption budgets to protect service capacity.
Will Dynamic Resource Allocation replace the device plugin for GPU sharing?
DRA is the direction of travel: claims describe devices and sharing strategies declaratively, and the scheduler sees device attributes. The NVIDIA DRA driver already exposes MPS configuration, though the guide marks that support as alpha behind a feature gate. For production in 2026, most teams keep the operator and device plugin path and pilot DRA separately until the features they need are stable in their Kubernetes and driver versions.
Further Reading
- Kubernetes Dynamic Resource Allocation (DRA) GPU scheduling guide
- Slurm vs Kubernetes for AI training: Kueue, Volcano and Slinky decision guide
- Spot and preemptible instance architecture for interruption-tolerant Kubernetes GPU workloads
- NVIDIA GPU Operator: GPU sharing with time-slicing
- NVIDIA GPU Operator: MIG support
- NVIDIA MIG user guide
- NVIDIA MPS documentation
- NVIDIA k8s-device-plugin on GitHub
By Riju — about
