Container Image Lazy Pulling: SOCI, Nydus, eStargz and GKE Compared

Your autoscaler decided to add capacity, the node came up, the scheduler placed the pod, and then nothing happened for a minute. The container was not slow to start. It was still downloading and unpacking a 3 GB image of which it would read a few hundred megabytes. This is the dominant, least-discussed component of many cold starts, and it is exactly the problem container image lazy pulling exists to solve: start the container before the whole image has arrived, and fetch bytes on demand as the process touches them.

It matters now because images keep growing (ML runtimes, CUDA stacks, language toolchains) while scale-out expectations keep shrinking. Four mature approaches exist: AWS’s SOCI, the Nydus image service, eStargz, and managed streaming on GKE and Azure. They differ sharply in what they require from your build pipeline, your runtime and your registry.

What this covers: why pulls dominate cold starts, the OCI layer-format limits that force full downloads, how each lazy-pulling design works internally, containerd configuration, how to measure honestly, the trade-offs that bite in production, and a decision matrix.

Context and Background: Why Image Pull Dominates Cold Start

The most-cited evidence is the FAST ’16 “Slacker” paper (Harter et al.), which studied 57 containerized applications with a benchmark called HelloBench. It reported that about 76% of container start time goes to pulling packages, while only about 6.4% of the pulled data is actually read during startup (USENIX FAST ’16). That study is a decade old and used the Docker of its day, so treat the exact percentages as historical. The structural finding has aged well: start-up reads a small, predictable subset of the image, yet the standard pull path transfers everything.

The standard path has three serial stages for each layer: download the compressed tarball, decompress it, and unpack it into a snapshot directory. Only after all layers are unpacked does the runtime assemble the overlay filesystem and exec the entrypoint. Layer downloads can run in parallel (containerd does this by default), but unpacking is sequential per image because each layer’s snapshot is the parent of the next. For a many-layer image, network bandwidth is rarely the only limit; decompression CPU and disk write throughput are co-bottlenecks.

Cold start is therefore the sum of node provisioning, image pull, runtime setup and application initialisation. If you have already worked through serverless cold starts across Lambda SnapStart, Workers and Cloud Run, you know the platforms attack the same problem with snapshots or isolates. On Kubernetes, you rarely get those. The lever you do control is the image path, and it sits right behind the node-provisioning delay discussed in Karpenter vs Cluster Autoscaler for GPU.

There are older, simpler mitigations and they remain correct first moves: smaller base images, multi-stage builds, pre-pulled images on node AMIs, image pull-through caches, and a registry in the same region as the nodes. Lazy pulling is for the cases where those run out: images that are legitimately large, fleets that scale unpredictably, or nodes that are fresh by design (spot, serverless-style, autoscaled-to-zero).

My thesis, and the angle most vendor posts omit: lazy pulling does not remove the cost of the image, it moves it from a predictable up-front cost to an unpredictable run-time cost. The right question is never “how fast does the container start” but “how fast does the application reach its first useful request, and what is the tail latency of its first minute”. Everything below is judged on that basis.

Reference Architecture: How Lazy Pulling Actually Works

Figure 1: Eager pull versus lazy pull. The lazy path inserts a remote snapshotter that mounts a filesystem backed by registry range reads.

The diagram contrasts the two data paths. In the eager path, containerd’s built-in overlayfs snapshotter receives fully unpacked layers. In the lazy path, a remote snapshotter plugin tells containerd “this layer is already available”, mounts a network-backed filesystem for it, and serves file reads by issuing ranged requests to the registry (or a cache).

The direct answer

Container image lazy pulling is a technique where the container runtime starts a container after fetching only image metadata, then downloads file contents on demand as the process reads them, while the rest of the image streams in the background. It requires a snapshotter that can mount a remote filesystem, an index that maps files to byte ranges, and a registry that supports HTTP range requests.

What the OCI layer format prevents

An OCI layer is a tar archive, usually compressed with gzip. Gzip is a stream format: to read byte N of the uncompressed data you must decompress everything before it. A tar has no central directory either; the header of each entry precedes its data, so finding a file means walking the archive. Together, that means a stock layer blob cannot be random-accessed. A registry can serve a byte range of the compressed blob, but nothing tells you which range holds /usr/lib/libcuda.so.

Every lazy-pulling design is, at heart, a different answer to one question: how do we add random access without breaking the image ecosystem? There are three families of answer.

Add an external index, leave the image untouched. SOCI builds a table of contents and a table of independently decompressible spans, stored as a separate registry artifact next to the image.
Rewrite the layer so it is seekable but still valid OCI. eStargz concatenates many small gzip members, one per file or chunk, and appends a table of contents. A runtime that does not understand it just sees a gzip tar.
Replace the layer format with a purpose-built filesystem image. Nydus converts layers to RAFS, a content-addressable format with separate metadata and chunked data, designed for random access and deduplication from the start.

GKE Image Streaming and Azure Artifact Streaming are managed variants: the provider operates a backend that serves a filesystem view of your image, and you do not choose the format.

The remote snapshotter contract

containerd delegates filesystem preparation to snapshotters. A remote snapshotter participates in the pull: the CRI plugin attaches annotations to each layer, and when the snapshotter returns “already exists” for a layer’s snapshot key, containerd skips downloading and unpacking that layer. The snapshotter has by then mounted something that can serve the layer’s files, typically a FUSE filesystem (or, for Nydus, EROFS over fscache in the kernel). Writable container changes still land in an overlayfs upper directory, so the application sees an ordinary root filesystem.

This is why the runtime configuration looks the same across projects: register a proxy_plugins entry pointing at the snapshotter daemon’s gRPC socket, then select that snapshotter in the CRI config (or per-runtime class). We will see concrete config later.

Where the time goes after the change

Once pulled lazily, startup latency depends on four things: metadata fetch (index and manifests), the number of distinct ranges the process reads before it is ready, the round-trip time to the registry or cache for each, and how aggressively the snapshotter prefetches. A process that reads three shared libraries and a config file touches a handful of ranges. A Python service that imports hundreds of modules, or a JVM that scans a classpath, touches thousands of small files. The design differences among SOCI, Nydus and eStargz are mostly about how well they handle that second pattern.

Deeper Analysis: The Four Designs Compared Mechanically

SOCI: an index beside an untouched image

Seekable OCI (SOCI) is an AWS-originated open specification and the soci-snapshotter containerd plugin. Its defining property is that it loads standard OCI images lazily “without requiring a build-time conversion step” of the image itself; a separate index artifact lives in the registry next to the image (soci-snapshotter on GitHub).

Per AWS’s description, a SOCI index has two kinds of structures. For each layer there is a zTOC, made of a table of contents (every file and its offset in the tarball) and a zInfo table describing spans: chunks of the compressed tarball that can each be fetched with a ranged GET and decompressed independently. One SOCI index manifest per image lists those zTOCs, and its subject field points back to the image manifest by digest. Because the image’s own bytes are not modified, existing signatures and digest verification continue to work (AWS Containers blog).

Figure 2: SOCI runtime sequence. Index and zTOCs are fetched first; file reads translate to span range requests; a background process completes the full pull.

At runtime the snapshotter downloads the index and zTOCs, then creates one FUSE filesystem per image layer. When a process opens a file, the snapshotter looks up the file’s offset in the TOC, maps it to spans through zInfo, and fetches and decompresses only those spans. A local cache ensures each span is fetched and decompressed once. After the container starts, a background process pulls the full image into local storage so that later reads stop crossing the network.

Three operational facts from AWS’s documentation deserve attention:

Size threshold. AWS reports the largest gains for images over 250 MB compressed in its internal testing; below that, the overhead of fetching SOCI artifacts and setting up FUSE may exceed a traditional pull.
Small layers are skipped. Layers under 10 MB are not indexed by default (--min-layer-size) and are downloaded in full at launch. An image whose layers are all under the threshold gets no index. Sidecar images often need the flag tuned.
Workload shape. Gains shrink for workloads that read filesystem metadata heavily or touch most image data soon after start.

Index manifest v1 versus v2. In v1 the index is only loosely associated with the image, so replication tools often skip it and adding or removing an index on a widely deployed image can produce a mix of lazy and fully pulled tasks. SOCI v2 binds both artifacts under an OCI image index with cross-referencing annotations, so the index cannot be silently changed or dropped. The cost is a build-time step (soci convert, requiring soci 0.10.0 or later) that adds an annotation to the image manifest, which changes the image digest while leaving filesystem layers unchanged. On Fargate, AWS states that new SOCI users can only use v2 and images with only a v1 index fall back to a full download (AWS Containers blog on v2). In AWS’s nginx example, a roughly 129 MB image carried a roughly 3.6 MB index, a useful sense of the registry overhead.

The digest change in v2 has a supply-chain implication: if you sign by digest (cosign, Notation), sign the post-conversion digest. Our guide to Trivy, Grype and cosign pipelines covers where in the pipeline signing should happen; conversion belongs before it.

SOCI also ships an opposite mode. Parallel pull-unpack is an upfront option in which layers are downloaded and unpacked concurrently before the container starts, with knobs such as max_concurrent_downloads_per_image (default 3) and max_concurrent_unpacks_per_image (default 1), and a discard_unpacked_layers switch. It is aimed at I/O-bound applications and many-layer images, and an experimental setting can use it as a fallback for images with no SOCI index (parallel mode docs). The existence of this mode is itself telling: even the lazy-loading project acknowledges that some workloads are better served by a faster eager pull.

On AWS, Fargate checks the registry for a SOCI index for each image automatically (Linux platform version 1.4 or later, no task-definition flag), and a task may mix SOCI and non-SOCI containers. For self-managed nodes, you run soci-snapshotter-grpc as a daemon and register it with containerd. I could not verify, in the pages I read, the exact current EKS-managed-node-group support story beyond a linked EKS walkthrough, so check the repository’s eks.md before relying on it.

eStargz: seekable gzip that still looks like gzip

eStargz is the lazily-pullable format of the stargz-snapshotter project, itself a non-core containerd sub-project. It is “compatible to OCI/Docker images”, derived from Google’s earlier stargz format (from the CRFS project) with added runtime optimisation and content verification (stargz-snapshotter).

The trick is gzip’s support for concatenated members. Each file (or chunk of a large file) is compressed as its own gzip member, so a reader can seek to a member boundary and decompress it alone. A table of contents (a JSON file, itself stored as a member) at the end of the archive records each file’s offset. A footer points to the TOC. A runtime unaware of eStargz untars it as a normal layer, which is why such images run on plain Docker.

The second contribution is prefetch. eStargz images can be built with ctr-remote image optimize, which profiles a sample run of the workload, records which files it touches, and places them first in the layer behind a landmark marker. The snapshotter then fetches that prefix in one large request instead of many small ones. This addresses the exact weakness of pure on-demand fetch (thousands of small reads) by converting the hot set into a sequential read. It is the one place where eStargz is stronger than a plain SOCI index, which the SOCI repository describes as not yet implementing “image load order based on your specific workload”.

Practical requirements per the project: containerd above 1.4.2, the stargz snapshotter daemon on each node, and a kubelet pointed at the snapshotter’s image-service endpoint via --image-service-endpoint (supported since v0.19.0). Any OCI-compatible registry works provided the image is already in eStargz form, which means you convert in CI with ctr-remote or BuildKit’s compression options.

A related point: the newer zstd:chunked format from the Podman/CRI-O ecosystem pursues the same idea with zstd and a similar chunk table. I did not verify its current containerd support in this research, so treat it as adjacent and check your runtime’s documentation.

Nydus: a different filesystem, not just a different index

Nydus is the heaviest-weight option and the most ambitious. It is a content-addressable filesystem built on the RAFS format, designed to extend the OCI spec for launch speed, space and bandwidth efficiency, and integrity (dragonflyoss/nydus). Instead of a tar of files, a Nydus image is split into metadata (the merged filesystem tree) and data (chunked blobs). The merged tree means the runtime does not stack and traverse many layers on every lookup, and the chunk granularity lets identical chunks be shared across layers and images.

Figure 3: Nydus architecture. nydusify converts OCI images to RAFS; nydus-snapshotter launches nydusd, which serves the container rootfs via FUSE or kernel EROFS with an optional P2P backend.

The components, per the project: nydus-image converts layers, nydusify pulls an OCI image, converts it and pushes it back, nydusd is the user-space daemon serving fscache or FUSE requests, nydusctl queries it, and nydus-snapshotter is the containerd remote snapshotter that prepares the rootfs. Without Nydus images, the snapshotter behaves like the built-in overlayfs snapshotter, which makes mixed fleets workable.

Kernel integration is the distinguishing feature. Since Linux 5.16 the in-kernel EROFS filesystem supports RAFS v6, and since 5.19 EROFS has a file-based caching (fscache) backend, so compressed RAFS v6 images can be mounted through fscache even when only partially present. The point is to take the user-space FUSE daemon off the hot read path. FUSE incurs a context switch to user space for each uncached read; an in-kernel path avoids that for cached data. The project also supports FUSE for hosts without those kernels, and offers native eStargz support.

Nydus also has a data-path story beyond a single node. Storage backends include registry, OSS, NAS, shared disk and P2P services via Dragonfly, and the project cites production benchmarks claiming Dragonfly integration reduces network latency by more than 80%. I cannot independently verify that figure, and it depends entirely on fleet size and topology, so treat it as a vendor claim. P2P matters when hundreds of nodes pull the same image at once: a registry fan-out of that scale is its own bottleneck, a mirror image of the thundering-herd problems covered in many scale-out postmortems.

The price is operational: a conversion step in CI, an extra daemon (nydusd) per node, a different image format to store and scan, and kernel version requirements to reach the best path. Vulnerability scanners and signing tools that read OCI layers need to scan the source image, not the converted one, or the pipeline must be designed around both.

GKE Image Streaming and Azure Artifact Streaming

Managed streaming trades control for zero pipeline change. On GKE, with an eligible image, the node uses a remote filesystem as the container root. The streaming backend serves files as the workload requests them while GKE downloads the full image to the node in the background and caches it; later reads come from the cache (GKE Image streaming docs).

The documented requirements are specific and worth reading before you assume it will work:

Enable the Container File System API.
Use a containerd node image: COS_CONTAINERD on GKE 1.30.1-gke.1329000 or later (the Autopilot default), or UBUNTU_CONTAINERD on 1.35.0-gke.1403000 or later.
Store images in standard or remote Artifact Registry repositories, or in public Docker Hub.
Private nodes need Private Google Access; VPC Service Controls perimeters need containerfilesystem.googleapis.com added.

Google’s documented example is a 327 MB image that needed about 24 seconds for a full pull versus about 1.5 seconds for the data needed to start with streaming. That is a single vendor example with a small image, not a benchmark of your workload. The page also lists real limitations: images using the V2 manifest schema version 1 are ineligible, images with duplicate or empty layers are pulled without streaming, workloads that read many files at initialisation may start slower because of remote read latency, and the benefit may not appear on the first pull of an image. Streaming nodes also reserve additional memory on a sliding percentage scale (for example 1% of the first 4 GiB), with no extra CPU reserved. Standard clusters enable it per cluster or node pool; Autopilot uses it automatically for eligible images.

Azure Container Registry’s Artifact Streaming was, on the documentation I read, still in preview, Premium SKU only, Linux AMD64 only (no ARM64 or Windows), unsupported with customer-managed-key registries, and requires AKS 1.26 or higher. Microsoft cites time-to-pod-readiness improvements of “over 15%”, particularly helpful for images over 30 GB (Azure docs). Preview status may have changed by the time you read this; verify before planning around it.

Side-by-side summary

Dimension	SOCI	eStargz	Nydus (RAFS)	GKE Image Streaming
Image modified?	No (v1); manifest annotation in v2	Yes, layer rewritten, still valid OCI	Yes, converted to RAFS	No
Build step	Index creation	Convert/optimise	Convert with nydusify	None
Prefetch by workload profile	Not implemented per repo	Yes, via optimize	Yes, access recording	Provider-managed
Data path	FUSE per layer	FUSE	FUSE or kernel EROFS	Google backend
Runs without special runtime	Falls back to normal pull	Yes, plain gzip tar	Needs nydus-snapshotter	Standard clusters
Operator burden	Daemon on self-managed nodes	Daemon plus kubelet flag	Daemon, converter, kernel	Near zero

Configuring containerd and Measuring Honestly

containerd configuration

Self-managed lazy pulling always starts with registering the snapshotter as a containerd proxy plugin. The SOCI getting-started guide shows this form for containerd config version 3:

version = 3

[proxy_plugins]
  [proxy_plugins.soci]
    type = "snapshot"
    address = "/run/soci-snapshotter-grpc/soci-snapshotter-grpc.sock"

The stargz-snapshotter documentation shows the equivalent for its own socket, plus selecting it as the CRI snapshotter:

version = 2

[proxy_plugins]
  [proxy_plugins.stargz]
    type = "snapshot"
    address = "/run/containerd-stargz-grpc/containerd-stargz-grpc.sock"
  [proxy_plugins.stargz.exports]
    root = "/var/lib/containerd-stargz-grpc/"

[plugins."io.containerd.grpc.v1.cri".containerd]
  snapshotter = "stargz"
  disable_snapshot_annotations = false

Note disable_snapshot_annotations = false: the CRI plugin must pass the layer annotations that tell a remote snapshotter what to mount. Forgetting this is the most common reason lazy pulling silently degrades into a normal pull. The containerd config schema changes between major versions (note version 2 versus 3 above), so match the snippet to your installed containerd and consult its docs rather than copying blindly. Our post on containerd 2.3, CRI-O and Podman hardening covers runtime upgrade hygiene that applies here too.

Building a SOCI index for a local test follows the documented flow:

# Convert an image and build its SOCI index (requires containerd + fuse)
sudo soci convert docker.io/library/rabbitmq:latest $REGISTRY/rabbitmq:latest

# Push image and index
sudo nerdctl push $REGISTRY/rabbitmq:latest

# Run the snapshotter and pull lazily
sudo soci-snapshotter-grpc &> ~/soci-snapshotter-logs &
sudo nerdctl pull --snapshotter soci $REGISTRY/rabbitmq:latest

The guide lists containerd 1.4 or later, nerdctl 1.6.0 or later and FUSE as prerequisites, and notes SOCI does not work with rootless nerdctl. For CI where no containerd is running, soci convert --standalone exists. For eStargz, the optimisation step profiles a real run:

ctr-remote image optimize --oci --entrypoint='["/bin/app"]' \
  registry.example.com/app:latest registry.example.com/app:esgz

Treat the flags above as illustrative of the workflow and check ctr-remote image optimize --help for your version.

What to measure

Most published lazy-pulling numbers measure “time until container process starts”. That is the wrong finish line. A useful measurement protocol has these properties:

Define ready. Use your readiness probe passing, or better, the first successful synthetic request. Time to Running flatters lazy pulling because the process may be stalled on remote reads.
Separate cold from warm. Run on a fresh node with an empty content store and snapshotter cache. Record the first pull and the second pull separately; GKE’s own documentation warns the benefit may not appear on the first pull of an image.
Record the stages. Kubernetes events give Pulling and Pulled timestamps; containerd and snapshotter logs give mount timing. For lazy pulls, Pulled arrives early by design, so subtract carefully.
Measure the tail. Capture p50 and p99 request latency for the first 60 seconds after ready. Lazy pulling’s cost shows up as a long tail of slow first requests while uncached ranges are fetched.
Repeat and vary. Run enough trials to see variance, use more than one image shape, and test at your real concurrency, since a registry or cache under simultaneous load behaves differently.

Figure 4: A measurement and decision workflow for container cold start image pull. Compare eager and lazy on time to first successful request and first-minute tail latency, then decide per workload.

A simple harness is a script that creates a pod on a labelled fresh node, records kubectl get events timestamps, hits a health endpoint in a loop with timestamps, and writes JSON for later analysis. Keep it boring and repeatable. I am deliberately not publishing sample numbers here: any figure I printed would be illustrative at best, and the right answer is workload-specific.

Reading the result

If lazy pulling cut time to ready but raised first-minute p99, you have moved cost, not removed it. If both fall, the workload has a small hot set and lazy pulling is a clear win. If time to ready does not move, check three suspects in order: the snapshotter is not actually engaged (look for FUSE mounts per layer, and the node label on GKE), the image is below the size where overhead pays off, or the application reads most of the image at startup.

Trade-offs, Gotchas, and What Goes Wrong

First-read latency is a hidden tax. Every uncached read becomes a network round trip. An application that imports thousands of modules at start can issue thousands of small range requests. Prefetch (eStargz, Nydus access recording) mitigates this; plain on-demand fetch does not. GKE’s documentation says as much: workloads reading many files during initialisation may start more slowly with streaming.

Registry becomes part of the data path. With eager pulls, the registry is needed only at pull time. With lazy pulls, a registry or cache outage during the container’s first minutes can stall running processes mid-read, until the background fetch completes. Co-locate registries with nodes, use pull-through caches, and plan for what happens if range requests fail. FUSE-backed reads that fail surface as I/O errors in your application.

Cache and disk behaviour. Background fetch still writes the full image to local disk, so you gain start speed but not disk savings. Span or chunk caches add memory and disk pressure; GKE documents a per-node memory reservation for streaming. On small nodes this can matter.

FUSE and node-level blast radius. A user-space daemon serving every container’s root filesystem is a single point of failure on that node. If nydusd or the SOCI snapshotter crashes, running containers can lose their filesystems. Monitor the daemon, set restart policies, and test failure injection. This is a different risk profile from the in-kernel overlayfs, and it interacts with the security posture discussed in Kubernetes Pod Security Standards and runtime hardening.

Index and image drift. SOCI v1 indexes can be skipped by replication or removed independently, producing mixed behaviour during rollouts, which is the motivation for v2. For any design with a sidecar artifact, test cross-region replication and image promotion explicitly.

Tooling blind spots. Scanners, signers and SBOM generators expect OCI layers. Converted formats (RAFS, eStargz) and modified manifests (SOCI v2) change what you must scan and sign. Decide which digest is the source of truth before you ship.

Small images and small layers. For images under a few hundred megabytes, the overhead often cancels the benefit; AWS says as much for sub-250 MB images. A smarter fix may be to shrink the image or bake it into the node image.

Not all images qualify. GKE excludes schema-v1 manifests and duplicate or empty layers. Azure’s preview excludes ARM64 and CMK registries. Always check eligibility per image, then verify at runtime that streaming engaged.

Anti-pattern: using lazy pulling to excuse bloat. A 12 GB image that streams quickly is still a 12 GB image that is slow to scan, replicate, cache and patch. Lazy pulling is a mitigation, not a design license.

Decision Matrix and Practical Recommendations

Situation	Best fit	Why
ECS on Fargate, large images	SOCI (v2 index)	Native Fargate support, no task-definition changes
GKE Standard or Autopilot, images in Artifact Registry	GKE Image Streaming	Zero pipeline change, managed cache
Self-managed containerd, willing to convert images in CI	eStargz with optimize, or Nydus	Workload-profiled prefetch reduces small-read penalty
Large fleet, many nodes pulling the same image at once	Nydus plus Dragonfly	P2P distribution addresses registry fan-out
Mixed arch or Windows nodes	Eager pull, parallel pull-unpack	Managed streaming options are often Linux and AMD64 only
Images under about 250 MB	Optimise the image first	Lazy overhead may exceed the saving
Compliance requires strict image provenance	Verify signing flow before adopting	Converted artifacts change digests and scan targets

My recommendations, in order:

Shrink and cache before you stream. Multi-stage builds, distroless or slim bases, layer ordering, and pre-baked node images fix more cold starts per hour of engineering than any snapshotter.
Profile the hot set. Before choosing a technology, trace which files your application reads at startup. If the answer is “most of the image”, skip lazy pulling.
Prefer the managed option on managed platforms. If you are on GKE or Fargate, turn on the provider feature on a canary node pool first. The operational burden is the lowest.
Choose prefetch-capable formats for self-managed fleets. For Python, JVM and Node workloads that touch many small files, profiled prefetch is the difference between a good and a mediocre result.
Instrument the daemon. Treat the snapshotter like a kubelet-grade component: metrics, alerts, restart policy, upgrade plan.
Roll out like a runtime change. Canary a node pool, compare p99 of first-minute requests, and keep an eager-pull rollback path.

A short checklist before enabling it in production:

[ ] Image is larger than about 250 MB or has a clearly small hot set
[ ] Eligibility confirmed (manifest schema, layers, architecture, registry type)
[ ] Indexes or converted images are signed and replicated with the image
[ ] Measured time to first successful request and first-minute p99, cold and warm
[ ] Snapshotter daemon monitored, with a tested failure mode
[ ] Rollback to eager pull verified

For autoscaling context, remember pull time sits inside the larger scale-out loop; the interplay with node provisioning and scale-to-zero is explored in Kubernetes 1.37 HPA scale to zero versus KEDA and Knative, and managed-service differences across clouds in the EKS vs GKE vs AKS comparison.

Frequently Asked Questions

What is container image lazy pulling?

Container image lazy pulling is a technique in which a container starts after downloading only image metadata, and file contents are fetched from the registry on demand as the process reads them. A remote snapshotter mounts a network-backed filesystem for each layer, while the remainder of the image is typically downloaded in the background. The goal is to cut the time between scheduling a pod and the application being ready, especially for large images.

Is SOCI better than eStargz or Nydus?

No single winner exists. SOCI needs no change to the image itself in v1 and is natively supported on AWS Fargate, but the SOCI repository says workload-specific load-order optimisation is not implemented. eStargz offers profiled prefetch and stays valid OCI. Nydus offers a purpose-built format, kernel EROFS paths and P2P integration at the cost of conversion and extra components. Pick based on platform, tolerance for pipeline change, and whether your workload reads many small files.

Does lazy pulling always make containers start faster?

No. Vendors say benefits are largest for big images and for workloads that read a small part of the image at start. AWS notes images under about 250 MB may start slower with SOCI, and GKE notes workloads that read many files during initialisation may start more slowly. Measure time to first successful request, not time to Running, and compare cold and warm runs before adopting it.

Does GKE Image Streaming require changes to my images?

No image conversion is required, but images must be eligible. GKE documents that images must live in standard or remote Artifact Registry repositories or public Docker Hub, use a containerd node image on supported versions, and avoid V2 manifest schema 1, duplicate layers and empty layers. You also enable the Container File System API. On Autopilot it is applied automatically to eligible images.

How does lazy pulling affect image security scanning and signing?

It can change what you scan and sign. SOCI v1 leaves the image digest unchanged, but v2 adds an annotation that changes the digest. eStargz and Nydus rewrite layers, producing new artifacts. Scan the source image, decide which digest your admission policy trusts, and sign after conversion. Nydus advertises end-to-end integrity checks at runtime, which helps detect tampering on the read path.

Should I use lazy pulling or just a smaller image?

Start with a smaller image and cached or pre-baked nodes; they are simpler, cheaper and benefit every pull path, scanning included. Use lazy pulling when images are legitimately large (ML stacks, toolchains), nodes are frequently fresh, and you have measured that pull time dominates. Many teams will end up combining both, since a leaner image also makes lazy-load indexes and caches smaller.

Container Image Lazy Pulling and Cold Starts: SOCI, Nydus, eStargz and Image Streaming Compared (2026)