Apple WWDC 2026 On-Device AI: A Production Analysis Five Months Later

When Apple’s keynote ended on June 8, the commentary was about Siri. Five months and one shipped operating system later, the more useful question is an engineering one: what does Apple’s AI stack look like when real users, real batteries and real regulators hit it? This Apple WWDC on device AI production analysis revisits the pre-keynote speculation, replaces it with what Apple confirmed and shipped, and maps the places where on-device generative AI still bends or breaks.

The headline is that the product is no longer a single on-device model. It is a routed system: a small local model, Apple’s server models on Private Cloud Compute, optional third-party models behind a common protocol, and a new framework for bringing your own weights. The router, the quotas and the fallbacks are now the product surface, and they are where production incidents will come from.

You will leave with a layered mental model of the stack, a request lifecycle, a decision framework for choosing a model path in your own app, the physical limits that cap local inference, and a checklist for shipping features that degrade gracefully. Facts are dated to November 2026, and anything Apple has not confirmed is labelled as reported or illustrative.

What this covers: what was announced versus rumoured, the layered architecture, the request lifecycle, a developer decision framework, memory-bandwidth and thermal limits, privacy and Private Cloud Compute trade-offs, failure modes, and a practical shipping checklist.

Context and Background

A year earlier, at WWDC 2025, Apple opened a roughly three-billion-parameter on-device model to third-party developers through the Foundation Models framework. Apple’s own technical report described that model as optimized for Apple silicon through KV-cache sharing and 2-bit quantization-aware training, paired with a server model built on a Parallel-Track Mixture-of-Experts (PT-MoE) transformer that runs on Private Cloud Compute (PCC). Developers got guided generation, constrained tool calling and LoRA adapter fine-tuning in a few lines of Swift. The source is Apple’s 2025 foundation models tech report.

The gap between that foundation and the product Apple had promised was the story of the following twelve months. Apple demonstrated a context-aware Siri at WWDC 2024 and did not ship it. Press coverage of the 2026 keynote notes that in May 2026 the company sought approval of a US$250 million class-action settlement over the delayed Siri features, with Apple denying wrongdoing. That history explains why the 2026 keynote was so heavily oriented toward delivery rather than vision.

What Apple announced on June 8, 2026 was a software-only event. Siri AI, described on stage as a ground-up rebuild, became the face of Apple Intelligence. Apple said the next generation of Apple Foundation Models was built in collaboration with Google and its Gemini models. The Foundation Models framework grew into what Apple’s developer relations team called a single native Swift API: more powerful on-device models with image input, support for server models, custom skills, and a new language model protocol that lets apps plug in models from other providers.

Two details deserve a caveat. Bloomberg reported that Apple pays roughly one billion dollars a year for a Gemini model with about 1.2 trillion parameters, and The Information reported that the heaviest queries route to Google Cloud because running that model inside PCC was too slow at scale. Apple did not confirm either on stage. This article treats them as credible reporting, not fact, and draws the architecture only from what Apple has stated.

If you build on-device AI outside Apple’s ecosystem, the same constraints apply with different silicon. Our guides to small language models at the edge and ExecuTorch on-device LLM serving cover the Android and embedded side of this problem. The Apple version is distinctive because one company controls the silicon, the OS, the framework, the cloud fallback and the privacy narrative.

What changed since the preview coverage

Pre-keynote analysis, including the earlier version of this article, leaned on leaks about iOS 27 and on speculative silicon numbers. Five months on, three things matter more than any leak. First, iOS 27 shipped on September 14, 2026 as a free update, with Siri AI arriving as an English-language beta and French, Japanese, Korean, Portuguese and Spanish following in October according to Apple’s launch announcement as reported by several outlets. Second, Apple confirmed that Siri AI is not available on iPhone or iPad in the European Union at launch, citing the Digital Markets Act, and is also unavailable in China while regulatory requirements are worked through. Third, a November 3 note on Apple Developer reportedly opened Siri AI integration to all third-party apps in the iOS 27.3 beta, where earlier betas limited deep hooks to a smaller set.

None of those three facts is about model quality. All three are about distribution, regulation and integration surface, which is exactly where production systems tend to fail.

The Reference Architecture: A Routed System, Not a Model

Apple Intelligence in iOS 27 is best understood as four layers: an orchestrator (Siri AI) that interprets requests, a router that chooses where inference runs, several model back ends (on-device, Private Cloud Compute, and developer-selected third-party models), and an action layer built on App Intents that turns model output into app behaviour. Developers touch the stack through the Foundation Models framework and, for custom weights, Core AI.

Figure 1: Routed architecture of Apple’s AI stack as of iOS 27. The partner cloud path is shown because it was reported, not because Apple confirmed it.

The diagram shows the shape. A request enters Siri AI from voice, text or screen context. A router decides whether the on-device model can handle it, whether it needs a larger server model on PCC, or, per reporting, whether the heaviest queries go to a partner cloud. Output flows to App Intents, which is how Siri AI acts inside apps. The Foundation Models framework fronts the on-device and server paths for developers, while Core AI is the lane for models you ship yourself.

Layer 1: Orchestration and context

Siri AI’s pitch is personal context, on-screen awareness and app actions. Apple said the updated App Intents framework lets developers connect their apps to those capabilities. The orchestrator therefore needs three kinds of input: the user’s utterance, a snapshot of what is on screen, and retrieved personal data from mail, messages, files and photos.

That retrieval step is a quiet architectural commitment. Apple said it rebuilt the foundation behind search across Spotlight, Photos and Mail so that content is indexed almost immediately after it changes. Fast, fresh on-device indexing is what makes retrieval-augmented answers about your own data plausible without sending that data off the device. Any assistant is only as good as its retrieval, and Apple’s advantage here is that the index lives where the data lives.

Layer 2: The router and why it is the real product

A router answers one question per request: which model, on which machine, with how much context? The answer trades latency, quality, cost, privacy and battery. A small local model answers instantly and keeps data on the device but has less world knowledge and a shorter effective context. A server model is stronger and slower to reach, and a third-party cloud model is stronger still but changes the privacy contract.

This is the same problem the industry solves for cloud APIs with model cascades, covered in our posts on LLM routing and model cascades and LLM model routing architecture. The difference on a phone is that the routing signal includes battery state, thermal headroom, network quality and a privacy classification, and that the user never sees a model picker. Apple has not published the router’s policy. For developers, the practical consequence is that identical code can take different paths on different days.

Layer 3: The model back ends

There are now three back ends behind one Swift surface. The on-device model is the successor to the roughly three-billion-parameter model of 2025; Apple said the new framework supports “more powerful on-device models with image input”, though it has not, to my knowledge, published a new parameter count for the 2026 generation, so any specific number would be a guess. The server model runs on Private Cloud Compute. The third path is the new language model protocol, which Apple says works with Claude, Gemini and any provider that implements it.

One commercial detail shapes adoption: developers in the App Store Small Business Program with fewer than 2 million total first-time App Store downloads can use the next generation of Apple Foundation Models on PCC at no cloud API cost. That makes Apple’s server model the default cheap choice for small teams and pushes larger publishers toward a build-or-buy decision.

Layer 4: Core AI for bring-your-own-model

Core AI is the new framework for running your own models on device. Apple describes it as an architecture optimized for the unified memory and Neural Engine of Apple silicon, intended to let developers deploy full-scale large language models locally. Coverage in March 2026 framed it as the successor to Core ML, but treat the exact migration path with care: verify against Apple’s current documentation before you plan a port, because API surface and deprecation timelines are the sort of detail that moves between betas.

The architectural point is that Apple now offers two on-device lanes. Foundation Models gives you Apple’s tuned model with guided generation and tool calling, and you inherit Apple’s updates, safety layers and thermal management. Core AI gives you control over weights and quantization, and you inherit the responsibility for memory footprint, load time and fallback behaviour. We compare the quantization choices you would face on that second lane in GGUF vs AWQ vs GPTQ vs FP8.

Walk-through: One Siri AI Request, End to End

A single turn passes through classification, context assembly, a routing decision, generation, and an app action. Each hop has its own latency budget and failure mode, and only the generation hop is what most coverage calls “the AI”. Understanding the full lifecycle is what lets you predict where your own feature will be slow or wrong.

Figure 2: Request lifecycle for a context-aware Siri AI turn. The local model plans first, then either answers or escalates minimal context to Private Cloud Compute.

Step 1: Intent classification and context gathering

The first work is cheap and local. The system classifies the intent, decides which context sources are relevant, and pulls the minimum necessary snippets from the on-device index. Keeping this step local matters for privacy, because the classification result determines what, if anything, leaves the device. It also matters for latency: a deterministic cheap step up front avoids paying cloud round-trip time for requests the local model can finish.

A useful mental model is that the local model is both a worker and a gatekeeper. As a gatekeeper it must be conservative about confidence, because a wrong decision to answer locally produces a confident, poor answer, while a wrong decision to escalate produces a slower but better one. Neither error is visible to the user as a router bug; they surface as “Siri was dumb” or “Siri was slow”.

Step 2: The escalation decision

When the plan requires broad knowledge, long reasoning chains, or large context, the request escalates. Apple’s stated privacy position, from Craig Federighi’s keynote as quoted by press, is that data is used only to execute the request and that outside experts can verify the promise. The mechanism behind that claim is Private Cloud Compute, which Apple designed around stateless processing, no privileged runtime access for operators, and published software images that independent researchers can inspect. The primary source is Apple’s Private Cloud Compute security research post.

What the reporting about a partner cloud for the heaviest queries would change is the threat model for that slice of traffic. If it is accurate, some requests would leave Apple-operated infrastructure. Apple has not confirmed this, and I would not build a compliance argument on either assumption. Treat the Apple documentation as the contract and verify any data-handling statement you make to your own users against current Apple disclosures.

Step 3: Generation and tool calls

Generation can be a plain answer, a structured object, or a tool call. The Foundation Models framework’s guided generation constrains output to a Swift type, which is how apps get reliable structure out of a small model. Tool calling lets the model request your code. Combined with App Intents, this is the path by which Siri AI can, for example, update a weak password through Safari or add a calendar item from a message, both examples Apple showed.

The engineering implication is that your App Intents are now an API consumed by a probabilistic caller. Idempotency, validation and confirmation prompts matter more than they did when intents were invoked by a deterministic shortcut. A model that mis-parses a date will call your intent with a wrong date, confidently.

Step 4: Action, result and presentation

After the app acts, the result returns to the orchestrator, which composes the reply. A less-discussed constraint is usage limits. Reporting from the keynote says some Siri AI features have daily usage limits, with higher limits for iCloud+ subscribers. A limit is a failure mode you must design for: a feature that works in testing may be rate-limited for a heavy user at four in the afternoon.

Where Local Inference Hits Physical Limits

On-device token generation is bounded by memory bandwidth first, thermals second, and compute last. Decoding one token requires streaming essentially all active weights through the processor, so the ceiling on tokens per second is roughly memory bandwidth divided by model bytes. Quantization raises the ceiling by shrinking bytes; it does not change the shape of the constraint.

A first-principles decode ceiling

For a dense model, decode speed in tokens per second is approximately bandwidth in bytes per second divided by the size of the weights in bytes, ignoring KV-cache reads and activations. Apple has not published memory bandwidth for the iPhone 18 generation in the sources I verified, so the table below uses illustrative bandwidth values to show how the formula behaves. They are not measurements of any Apple device.

Illustrative effective bandwidth	3B model at 2-bit (about 0.75 GB)	3B model at 4-bit (about 1.5 GB)	8B model at 4-bit (about 4 GB)
40 GB/s	about 53 tokens/s ceiling	about 27 tokens/s	about 10 tokens/s
60 GB/s	about 80 tokens/s	about 40 tokens/s	about 15 tokens/s
80 GB/s	about 107 tokens/s	about 53 tokens/s	about 20 tokens/s

Treat each cell as a theoretical upper bound. Real throughput is lower because of KV-cache traffic, non-weight tensors, kernel overhead and contention with other processes. The takeaway is structural: halving bytes per weight roughly doubles the ceiling, which is why Apple’s 2025 report emphasises 2-bit quantization-aware training for the on-device model, and why a model four times larger is about four times slower to decode on the same chip.

The same arithmetic explains why server inference is so different. Datacentre accelerators use high-bandwidth memory with bandwidth measured in terabytes per second, and batch many users to amortise weight reads. A phone serves one user and cannot batch, so it is permanently on the unfavourable side of the roofline. Our deep dives on KV-cache compression and speculative decoding show the techniques that attack this limit.

Prefill, context length and the first-token problem

Users feel time to first token, not peak decode speed. Prefill, which processes the prompt, is compute-bound and scales with prompt length. A Siri AI request that carries on-screen text, retrieved emails and instructions can be thousands of tokens before the model produces anything. Longer context also grows the KV cache, which competes for the same limited memory as the weights and for the same bandwidth during decode.

This is why careful context assembly is a feature, not an optimisation. Sending the model only the three relevant emails instead of the whole thread shortens prefill, shrinks the cache, and improves answer quality by removing distractors. Apple’s own index rebuild is partly an investment in this: fast retrieval lets the system keep prompts short.

Thermals, power and sustained load

Phones cool passively. A burst of inference runs at full speed, then sustained load triggers throttling to protect the battery and the user’s hand. The common failure is a feature tested in short bursts on a cool device that behaves differently when a user runs five prompts in a row in a warm car. Apple does not publish its throttling curves, so measure on real devices under your own sustained workload rather than trusting a spec sheet.

A related issue is memory pressure. An on-device model occupies RAM that other apps also want. When the system is under pressure it can evict the model, and the next request pays a cold-load cost. Apple’s Foundation Models framework manages the system model for you, which is a large part of its value; if you ship your own weights through Core AI you own that lifecycle and must handle load, unload and warm-up explicitly.

A Developer Decision Framework

Choosing a model path is a sequence of questions about knowledge, sensitivity, context size and weight control. The framework below turns that into a repeatable decision you can encode in design reviews, with a safe fallback at every leaf.

Figure 3: Choosing a model path for a new AI feature. Every terminal node must have a non-AI fallback.

When the on-device model is the right choice

Choose the system on-device model when the task is narrow, the context is small, the data is sensitive, and the answer does not need broad world knowledge. Classification, extraction, summarisation of a short text, rewriting and structured output from a form are the sweet spots. You get offline operation, near-zero marginal cost, and a privacy story that needs no disclosure beyond the OS.

The cost is capability headroom. A small model hallucinates more on open-world questions and handles long documents poorly. Do not ask it to answer factual questions from memory. Give it the text and ask it to transform that text.

When to escalate to the server model

Choose Apple’s server model on PCC when the task needs better reasoning or knowledge but the data is still sensitive. For small teams, the no-cost tier for the Small Business Program makes this the natural default. You pay in latency and in availability: the feature now needs a network, and Apple’s service has its own capacity and rate limits.

Remember the regional gap. Because Siri AI is not available on iPhone and iPad in the EU at launch, system-level assistant features cannot be your only route to value for European users. Check whether the framework capabilities you depend on are available in a target region before you promise a feature there, since Apple has said some features depend on device, language and region.

When a third-party model makes sense

The language model protocol lets an app use Claude, Gemini or any provider that implements it behind the same Swift API. That is attractive when you need frontier capability, tool use that exceeds the local model, or a model you already evaluate. Google’s Gemini integration is available through the Firebase Apple SDK with Firebase AI Logic and App Check, so apps can switch between on-device and hosted models without building a backend.

The trade-off is that data now goes to a third party under that provider’s terms, and you take on cost, latency and an additional dependency. Put the routing in your own code with explicit consent and a clear user-facing disclosure. For a gateway layer that handles fallbacks and budgets on the server side, see our comparison of LLM gateways.

When Core AI earns its complexity

Choose Core AI when you need a specific open model, a fine-tune that Apple’s adapters cannot express, or deterministic version control of weights. You get predictable behaviour across OS updates, because the model does not change when Apple updates the system model. You pay in app size, download and load time, memory footprint and your own evaluation burden.

A practical rule: start with Foundation Models, measure the quality gap on your real data, and only move to Core AI when the gap is demonstrated and material. Many teams reach for custom weights because it feels more serious, then discover they have taken on a model-ops practice they did not budget for.

Privacy, Verifiability and the Regulatory Edge

Apple’s privacy claim in AI is architectural, not contractual: keep data on the device by default, and when a larger model is needed, run it on servers designed so that Apple cannot see or retain the request. Whether that holds depends on details the marketing leaves out, and those details decide what you can promise your own users.

What Private Cloud Compute actually guarantees

Apple’s published design for PCC rests on a few principles: stateless computation on user data, enforceable guarantees enforced by hardware and software rather than policy, no privileged runtime access that would let an operator inspect requests, non-targetability so an attacker cannot aim at one user, and verifiable transparency through published software images and a research environment. These are Apple’s stated design goals in its PCC security write-up. They are strong claims, and they apply to Apple-operated PCC nodes.

For a developer, the key question is not whether PCC is secure in the abstract, but whether your data ever leaves the PCC boundary. Using Apple’s server model through the Foundation Models framework keeps you inside Apple’s design. Using a third-party model through the language model protocol does not. The code looks nearly identical, which is the risk: a one-line change moves data across a trust boundary. Make the boundary visible in code review, in your privacy manifest and in your user-facing disclosure.

For readers who want the general hardware pattern behind these ideas, our explainers on confidential AI inference with TEEs and remote attestation with confidential VMs cover the mechanisms that make “verifiable” more than a slogan.

The regional availability problem

Regulation is now an engineering input. Apple said Siri AI will not launch on iPhone or iPad in the European Union, blaming the Digital Markets Act, and that there is no timeline; it also remains unavailable in China. EU users still get Siri AI on Mac, Apple Watch and Vision Pro, according to the keynote coverage. Whatever one thinks of the dispute, the consequence for teams is concrete: an assistant-dependent feature can have different availability across a single user’s own devices.

Design for capability detection, not region assumptions. Query what the framework reports as available at runtime, and treat unavailability as a normal state with its own UX. The expensive mistake is shipping a feature whose only path runs through an assistant that part of your audience cannot reach.

Provenance and the AI-content signal

One iOS 27 detail that is easy to miss: Apple introduced Apple Reference Image, which stores an untouched original with verification metadata alongside edited versions, so a photo can be shown to come from a real capture instead of an AI edit. As the Image Playground upgrade now produces photorealistic images, that provenance work matters more. If your app handles user media, review our overview of content provenance with C2PA and watermarking to decide how you will label generated content.

Trade-offs, Gotchas, and What Goes Wrong

Production failures in on-device AI cluster into five families: latency tails, capability cliffs, quota and availability gaps, silent behaviour drift across OS updates, and unsafe action execution. None of them show up in a demo. All of them show up in a one-star review.

Figure 4: Failure paths that converge on the same requirement, a non-AI fallback for every AI feature.

Latency tails, not averages

Median latency is a vanity metric for on-device AI. Thermal throttling, a cold model load after memory eviction, and escalation to the cloud each add seconds to a minority of requests, and users remember the worst one. Track p95 and p99 per path, and record which path served each request. If you cannot attribute a slow response to local, PCC or third-party, you cannot fix it.

A concrete test: run your feature fifty times in a row on a mid-range supported device with the screen on and the device warm, then compare the first five against the last five. If the gap is large, you have a thermal tail. Do the same after backgrounding the app and returning to catch the cold-load path.

Capability cliffs

Small models degrade abruptly rather than gracefully. A prompt that works at 500 tokens may fail at 3,000, and a task that works in English may fall apart in a language added later in the roadmap. Siri AI itself launched English-first with five more languages scheduled for October, which is a reminder that quality is language-specific. Test each supported locale separately; do not infer from English.

Build an evaluation set from real, anonymised inputs and run it against each model path before and after every OS update. If you need a grounding for what a sound harness looks like, see our guide on LLM evaluation harness architecture.

Quotas, availability and the silent off-switch

Daily usage limits for some Siri AI features, regional exclusions and beta status all mean the feature can be unavailable without your code doing anything wrong. The anti-pattern is treating “model unavailable” as an exception. It is a routine state, and the user interface must handle it as calmly as it handles being offline.

Beta status also matters. Siri AI shipped as a beta in English, and Apple had not, in the sources I checked, said when it leaves beta. Betas change behaviour between point releases, so pin your evaluation to OS versions and expect revisions in 27.x updates, including the reported broader third-party integration in the 27.3 beta.

Drift across OS updates

When you use the system model, Apple controls the weights. A system update can change tone, refusal behaviour, structured-output reliability or tool-calling accuracy without any change in your app. This is the price of not owning the model, and it is the strongest argument for Core AI when you need reproducibility. If you stay on the system model, regression-test on every OS beta and keep a canary cohort.

Unsafe action execution

The riskiest new surface is an assistant that acts. A model that can call your App Intents can also call them wrongly or be steered by content it reads, such as an email containing hidden instructions. Treat intent parameters as untrusted input, require confirmation for irreversible actions like payments, deletions or messages, scope intents narrowly, and log what the model asked for versus what ran.

For agentic features in general, our post on AI agent trajectory evaluation covers how to test multi-step behaviour rather than single answers.

Comparing Your Options at a Glance

The table below summarises the four model paths for a typical app feature. The ratings are qualitative judgments from the architecture above, not benchmark results.

Dimension	Foundation Models on device	Apple server model on PCC	Third-party model via protocol	Core AI custom model
Works offline	Yes	No	No	Yes
Data leaves device	No	Yes, to PCC	Yes, to provider	No
Marginal cost	None	Free for eligible small developers, otherwise see Apple terms	Provider pricing	None
Capability ceiling	Lowest	Higher	Highest	Depends on model and device RAM
Weight control	None	None	None	Full
Behaviour stable across OS updates	No	No	Provider dependent	Yes
App size impact	None	None	None	Large
Main risk	Capability cliff	Latency and availability	Privacy and cost	Memory and model-ops burden

The most common production design is not one column but a ladder: try on-device, escalate to PCC with consent where needed, and offer a third-party model only as an explicit user choice. Whichever rung answers, the user needs a stable experience when the rung above is missing.

Cost and capacity thinking

On-device inference moves compute cost to the user’s battery and removes it from your cloud bill, which is a real saving at scale. The server paths reintroduce per-request cost or quota. Model the breakeven honestly: a feature used ten times a day by a million users generates ten million inferences daily, and even a small per-request cost compounds. Do not fabricate a price; use the actual rate card of whichever provider you select and recompute when it changes.

Practical Recommendations

Ship AI features as a ladder with an exit. Start from the narrowest task the small model can do reliably, measure real tail latency on real devices, put every cross-boundary data flow behind explicit user-visible consent, and keep a fully working non-AI path. The teams that suffer are the ones whose feature is only the AI.

A short checklist you can apply in a design review:

Write down, per feature, which model path serves it and what data leaves the device on each path.
Detect capability at runtime and treat unavailability, quota exhaustion and regional gaps as normal states.
Measure p95 and p99 latency per path on warm, sustained-load devices, not just fresh ones.
Maintain a per-locale evaluation set and re-run it on every iOS beta and point release.
Validate and confirm every App Intent parameter that a model can supply, especially irreversible actions.
Keep prompts short: retrieve the minimum context rather than stuffing the window.
Start with Foundation Models; adopt Core AI only when a measured quality or reproducibility gap justifies model-ops cost.
Log which path answered each request so incidents are diagnosable.
Re-check Apple documentation for API names and availability before each release, since several details were still moving between betas.

If you take one idea from this article, take this: the model is the easy part to buy. The router, the quotas and the fallback are the parts you have to engineer. That is also where the next round of incidents will come from.

Frequently Asked Questions

What did Apple actually announce for on-device AI at WWDC 2026?

Apple announced Siri AI, a rebuilt assistant, plus an expanded Foundation Models framework that exposes more powerful on-device models with image input, server models, custom skills and a language model protocol for third-party providers. It also introduced Core AI for running your own models on Apple silicon and Xcode 27 with agentic coding. Apple said the next-generation Apple Foundation Models were built in collaboration with Google and its Gemini models.

Does Siri AI run entirely on the iPhone?

No. Apple describes a mix of on-device processing and Private Cloud Compute, and reporting from Bloomberg and The Information suggests the heaviest queries may route to a partner cloud. Apple has not confirmed that last detail. In practice the router decides per request, so identical prompts can be served from different places depending on complexity, context size and conditions. Plan your privacy disclosures around the Apple documentation, not assumptions.

Is Siri AI available in the EU and in other languages?

At launch it was not available on iPhone or iPad in the European Union, which Apple attributed to the Digital Markets Act, and it is unavailable in China. EU users get it on Mac, Apple Watch and Vision Pro. It launched in English as a beta on September 14, 2026, with French, Japanese, Korean, Portuguese and Spanish scheduled for October. Availability can vary by device, language and region.

Should I use Foundation Models or Core AI in my app?

Start with Foundation Models. It gives you Apple’s tuned model, guided generation, tool calling and system-managed memory with little code. Move to Core AI only if you can show a measured quality gap or need reproducible behaviour across OS updates, because custom weights add app size, load time, memory pressure and an evaluation burden. Many apps can meet their needs entirely with the system model.

Is on-device inference really private?

Local inference keeps data on the device, which is the strongest privacy position. When a request escalates to Private Cloud Compute, Apple’s design goals include stateless processing, no privileged operator access and verifiable transparency. Privacy changes if you route to a third-party model, since data then goes to that provider under its terms. The code change can be one line, so make the boundary explicit and disclose it to users.

What is the biggest production risk with on-device AI?

The biggest risk is not accuracy, it is unplanned unavailability and latency variance: thermal throttling, cold model loads, quotas, regional exclusions and beta changes. Pair that with agentic actions executed through App Intents and the second risk is unsafe automation. Mitigate both with per-path latency metrics, runtime capability checks, confirmation for irreversible actions and a fallback that works without AI.