FPGA Edge Inference vs GPU vs NPU: Latency, Power and HLS in Industrial Machine Vision (2026)

A bottling line rejecting a cracked cap has perhaps four milliseconds between the camera exposure and the air-jet that removes the bad part. A GPU can classify that image in well under a millisecond on average, and still miss the jet, because the average is not what the line experiences. The line experiences the worst frame in a million. That gap between mean speed and bounded speed is the real reason FPGA edge inference still has a seat at the table in 2026, even as GPU modules and NPUs get dramatically faster and easier to program.

This matters now because the choice has become less obvious, not more. NVIDIA’s Jetson Thor pushes server-class throughput into a 130 W module, NPUs now ship inside nearly every application processor, and FPGA vendors have bolted AI engines onto their fabrics. The honest question is no longer “which is fastest” but “which one gives me a latency I can promise, at a power and engineering cost I can afford.”

You will leave with a mechanism-level model of why the three architectures behave differently, the state of the FPGA toolflows (hls4ml, FINN, Vitis AI, Altera’s FPGA AI Suite), worked power-per-inference arithmetic labelled as illustrative, and a decision matrix you can apply to a real inspection station.

What this covers: the three architectures compared on latency determinism and power, how high-level synthesis changes FPGA development, the tool landscape as verified in October 2026, failure modes, and a practical decision framework.

Context and Background

Industrial machine vision is an unusual edge workload. Unlike a phone or a retail kiosk, the camera is often a line-scan or area-scan sensor streaming at a fixed, relentless rate over a specialised link: GigE Vision, CoaXPress, Camera Link, or increasingly 10GigE and USB3 Vision. The inference result drives a physical actuator, so a late answer is a wrong answer. Products run for 10 to 15 years, far beyond the support window of most consumer-derived silicon.

Three silicon families compete here. GPUs, typified by the NVIDIA Jetson line, offer a rich software stack (CUDA, TensorRT) and strong throughput when frames can be batched or at least pipelined. On the NVIDIA side, the Jetson AGX Thor specifications list up to 2,070 FP4 sparse TFLOPS in a configurable 40 to 130 W envelope for the T5000 module, while AGX Orin is listed at 200 to 275 TOPS (precision unstated in the source table I used) across roughly 15 to 60 W. NPUs are fixed-function matrix engines, from microwatt-class designs like the Arm Ethos-U85 paired with Cortex-M85 up to standalone accelerators compared in our Jetson Thor vs Hailo-10H vs Coral piece. FPGAs are arrays of lookup tables, block RAM and DSP slices whose wiring you define.

A fourth category is now blurring the lines: adaptive SoCs that combine FPGA fabric, Arm cores and hardened AI engines. AMD’s Versal AI Edge Series Gen 2 lists AIE-ML v2 tiles delivering between 31 and 184 dense INT8 TOPS depending on the device, alongside up to eight Cortex-A78AE and ten Cortex-R52 cores and a new X5IO block with MIPI C-PHY support (AMD product page; AMD describes the TOPS-per-watt gain as an internal projection). The result is that “FPGA vs NPU” is increasingly a false dichotomy inside a single package.

The framing in most comparison articles is peak TOPS per watt. That is the wrong axis for this domain. TOPS describes what a device could do on a perfectly fed, fully utilised, large-batch workload. A reject gate cares about latency percentiles, jitter, interface compatibility and the ability to still buy the part in 2038. Those are the axes this article uses.

Why peak TOPS misleads at the edge

A datasheet TOPS figure multiplies the number of multiply-accumulate units by clock rate. It assumes every unit is busy every cycle, which holds only when the model is large enough, and the memory system fast enough, to keep them fed. A small convolutional network on a 5-megapixel crop often exposes the opposite: the device idles waiting on memory or on a kernel launch. Utilisation, not peak, sets real throughput, and utilisation is highly architecture-dependent. We return to this with worked numbers later.

Three Architectures, Three Latency Philosophies

Direct answer: A GPU maximises throughput by running thousands of threads over data held in shared memory, so latency depends on scheduling, batching and the operating system. An NPU runs a compiled layer schedule on a fixed array of multiply-accumulate units. An FPGA lays the network out as a physical pipeline, so latency is a fixed number of clock cycles. Only the FPGA makes bounded latency a property of the hardware itself.

Figure 1: A dataflow FPGA pipeline for industrial machine vision. Each layer has its own engine with weights held in on-chip memory, streams connect the stages through FIFOs, and the decision drives hard IO directly.

Figure 1 shows the canonical FPGA structure. Pixels arrive from the camera interface, are debayered and normalised in fabric, and flow through a chain of layer engines. Each engine owns the DSP slices and block RAM for its layer, and neighbouring engines communicate through small FIFO buffers. Because every stage works on a different frame region concurrently, throughput is set by the slowest stage (the initiation interval), and end-to-end latency is the sum of stage depths. Both are known at synthesis time.

The GPU model: massive parallelism, shared resources

A GPU executes a network as a sequence of kernels. Each kernel is launched by the host (or from a CUDA graph), reads activations from DRAM or cache, computes, and writes back. The strengths are obvious: a mature compiler (TensorRT), a huge model zoo, and the ability to change the model by swapping a file. On Jetson-class modules the CPU and GPU share one memory pool, which removes the PCIe copy that plagues discrete cards.

The weakness for hard real-time is that latency emerges from many interacting layers. Kernel launch overhead, the Linux scheduler, DVFS (dynamic voltage and frequency scaling), thermal throttling, memory contention with the camera DMA, and other processes all add variance. NVIDIA’s real-time options exist, but you are bounding a statistical system. For a throughput-oriented workload this is irrelevant. For a 4 ms reject window it means you must measure the 99.99th percentile on the shipped configuration and keep a safety margin.

The NPU model: fixed arrays and compiled schedules

An NPU is closer to a specialised accelerator than a general processor. A compiler converts the network into a static schedule: tile the tensors, place them in on-chip SRAM, and issue them to a systolic or vector array of INT8 multiply-accumulate units. The schedule is deterministic in the sense that the same model produces the same instruction stream. Latency is usually predictable per model, and power is excellent because data movement is minimised.

The constraint is the operator set. NPUs execute what their compiler supports. An unsupported operation (a custom NMS, a deformable convolution, a new activation) falls back to a host CPU, often destroying the latency advantage. NPU vendors also tend to couple the toolchain tightly to a single SoC family, which matters over a 10-year product life.

The FPGA model: you define the machine

An FPGA contains configurable logic blocks, block RAM (BRAM) and UltraRAM, and DSP slices that implement multiply-accumulate in hardware. A dataflow design instantiates exactly the arithmetic the network needs, at exactly the precision it needs. A 4-bit convolution uses 4-bit multipliers built from LUTs, not a padded 8-bit or 16-bit unit. A ternary or binary layer needs no multiplier at all, only XNOR-popcount logic.

This is the architectural source of both the FPGA’s advantages (bounded latency, extreme quantization, custom IO, no OS in the path) and its costs (long build times, scarce engineers, fixed on-chip memory that caps model size). The FPGA does not run a model; it becomes the model.

Why jitter, not mean latency, decides

Consider a line running at 1,000 parts per minute with a reject decision budget of 4 ms from exposure to actuator command. If a GPU pipeline has a mean of 1.2 ms but occasionally spikes to 6 ms once per 200,000 frames, the line sees a missed reject roughly every 3.3 hours of continuous operation at 1,000 frames per minute (200,000 frames divided by 60,000 frames per hour). Whether that is acceptable depends on the product. For a pharmaceutical vial it is not. The figures here are illustrative, not measured, but the arithmetic is the point: rare tail events become routine at industrial duty cycles.

An FPGA pipeline with a fixed 1.1 ms latency has no such tail, because nothing in the path is shared or scheduled. The remaining variance sits upstream (exposure trigger, camera readout) and downstream (the actuator), which a design can also bound because the trigger and output are hardware signals the fabric drives directly.

Where the Milliseconds Go: Latency Budgets and Deterministic Pipelines

Figure 2: Where latency accumulates on a GPU path (copies, queuing, kernel launches, software decision) versus an FPGA path (streaming stages with a fixed cycle count and a hardware output).

Figure 2 contrasts the two paths for the same camera frame. On the GPU path, the frame is captured by a driver, copied or mapped into GPU-visible memory, possibly queued behind earlier frames, executed as a sequence of kernels, copied back, and finally turned into a decision by software that then toggles a GPIO or sends a fieldbus message. Each step is individually fast. The distribution of their sum is what matters, and it has a tail.

The FPGA path eliminates most of these stages rather than speeding them up. Capture, preprocessing and inference are one continuous stream. The output pin is driven by the same fabric. There is no copy because there is no second memory domain, and no queue because the pipeline accepts a new pixel every clock.

Latency, throughput and the batch trap

GPUs reach high utilisation by batching: processing 8, 16 or 32 images together amortises weight reads and keeps the multiprocessors busy. For a camera delivering one image at a time with a hard deadline, batch size one is the only legal operating point, and on large GPUs that is exactly where utilisation collapses. A 275-TOPS device running a 4-GOP-per-frame detector at batch one is not computing at 275 TOPS. It is mostly waiting for kernels to launch and memory to arrive.

The FPGA dataflow architecture does the opposite. It is designed for batch one. Because each layer engine is always resident, a single frame flows through at full speed, and utilisation depends on balancing the stage initiation intervals rather than on queue depth. This is why papers from the high-energy physics community, which use hls4ml for trigger systems with microsecond-scale budgets, treat FPGAs as the only credible option at that scale. Industrial vision is far less extreme (milliseconds, not microseconds), but the same mechanism applies.

A worked latency budget (illustrative)

Take a line-scan inspection station with a 4 ms deadline from trigger to reject command. A plausible budget on an FPGA SoC looks like this. Exposure and sensor readout: 1.5 ms. Link transfer and frame assembly: 0.4 ms. Preprocessing in fabric: 0.1 ms. Inference pipeline at, say, 200 MHz with 150,000 cycles of total depth: 0.75 ms. Decision logic and output: under 0.01 ms. Total: about 2.8 ms, leaving 1.2 ms of margin that is almost entirely fixed.

On a GPU module the exposure, readout and link stages are similar. Inference at batch one may be a mean of 1.0 ms and, in practice, a long tail. Add a Linux userspace capture path, a copy, and a software decision, and a mean near 3 ms is plausible with a p99.99 you must measure. Both designs may meet the deadline. Only one lets you prove it analytically. All numbers in this section are invented to illustrate the structure of the budget, not benchmark results.

Line-scan cameras and the streaming advantage

Line-scan cameras emit one row of pixels at a time. An FPGA can begin processing rows as they arrive, so inference latency overlaps with exposure. A frame-based GPU path typically waits for a complete image (or a complete tile) before starting. For web inspection of continuous material such as film, steel or textile, streaming convolution with line buffers removes the “wait for the full frame” term entirely. This is one of the few cases where the FPGA’s advantage is structural and not merely a matter of tuning.

Power Per Inference: Worked Arithmetic

Power comparisons are where marketing and engineering diverge most. Three effects dominate: data movement energy, precision, and utilisation. Moving a 32-bit word from off-chip DRAM costs orders of magnitude more energy than an arithmetic operation on it, a ratio reported widely in the architecture literature (the classic figures come from Mark Horowitz’s ISSCC 2014 keynote; I have not re-verified the exact picojoule values for this post, so treat the ratio qualitatively). An architecture that keeps weights and activations on chip wins on energy, which is precisely what a dataflow FPGA with weights in BRAM and an NPU with large SRAM both do, and what a GPU running a small network from DRAM often does not.

The utilisation correction

Energy per inference equals average power divided by inference rate. A module that draws 30 W while delivering 400 frames per second spends 75 mJ per frame. If it is configured to a 15 W mode and delivers 250 frames per second, it spends 60 mJ per frame. Neither number is on the datasheet. The only reliable method is to run your network at your batch size and measure wall power with a meter.

Here is an illustrative comparison for a 2 GOP-per-frame quantized detector running at batch one, 300 frames per second, on three hypothetical configurations. These are modelling assumptions, not measurements of any specific product.

Platform class	Assumed average power	Assumed fps	Energy per frame	Notes
GPU module, 30 W mode	22 W	300	73 mJ	Batch one underuses the GPU; host CPU and DRAM add to the draw
Edge NPU accelerator	5 W	300	17 mJ	INT8 compiled schedule; host overhead extra
Mid-size FPGA, 4-bit dataflow	9 W	300	30 mJ	Fabric plus IO; includes camera interface logic

The point of the table is the structure. The NPU usually wins on pure inference energy for a fixed, supported CNN. The FPGA can beat the GPU comfortably at batch one, and its advantage grows if the network tolerates 4-bit or lower precision, because LUT-based multipliers shrink with bit width. The FPGA also absorbs functions (debayering, lens correction, trigger generation, fieldbus) that otherwise need a second chip, so system power can favour the FPGA even when inference-only energy does not.

Reading vendor power claims

Vendor TOPS-per-watt figures usually use a specific data type, sparsity assumption, and workload. NVIDIA’s Thor figure of 2,070 TFLOPS is listed as FP4 with sparsity, which is not comparable to an INT8 dense figure. AMD’s Versal Gen 2 page lists both dense and max-sparsity INT8 TOPS and a separate MX6 figure; it also states its “up to 3X TOPS/watt” claim compares Gen 2 MX6 against Gen 1 INT8 and rests on internal projections. Altera’s FPGA AI Suite page cites a theoretical 88.5 INT8 TOPS on an Agilex 7 M-Series device, about 3,679 ResNet-50 frames per second at 90 percent utilisation. Each is honest within its own footnote. None is directly comparable to another. Always normalise to the same network, precision, batch size and measurement boundary.

Thermal envelope and the fanless cabinet

Industrial enclosures are frequently sealed, fanless, and sit near ovens or motors. A 130 W module needs serious heat sinking. A 9 W FPGA SoC can run passively in a 60 degree ambient. For many deployments the thermal design power, not energy per frame, is the binding constraint, and this favours both NPUs and mid-range FPGAs over the largest GPU modules. It also has a reliability angle: lower junction temperatures materially extend electronics life, a real consideration against a 10-year service life.

High-Level Synthesis: Changing Who Can Program an FPGA

Historically the FPGA’s weakness was the toolflow: write RTL in VHDL or Verilog, simulate, synthesise, place and route, close timing. A small inference engine could take a hardware team months. High-level synthesis (HLS) compiles a C, C++ or similar description into RTL, letting software engineers express an algorithm as loops and arrays while the tool schedules operations, allocates DSP slices and builds pipelines. The designer steers with pragmas: unroll this loop, pipeline that one at an initiation interval of one, partition this array across BRAM banks.

HLS does not remove the need to understand hardware. It moves the work from writing registers to shaping a compiler’s decisions. A loop with a carried dependency will not pipeline at II=1 regardless of how politely you ask. An array accessed in a pattern the memory cannot serve will stall. But for neural networks, whose structure is regular, HLS is effective, and domain tools generate the HLS code for you.

Figure 3: The FPGA inference toolflow. Quantization comes first, then a choice between generating a custom dataflow design (hls4ml, FINN) or compiling to a prebuilt NPU-like overlay (Vitis AI, FPGA AI Suite).

Figure 3 captures the key fork. The left branch generates a bespoke accelerator for your exact network through HLS and full place-and-route. The right branch compiles your model onto a programmable engine instantiated once in the fabric, which behaves like an NPU you happen to have built from LUTs. The first gives maximum efficiency and determinism; the second gives fast iteration and the ability to change models without a new bitstream.

hls4ml: the research-grown generator

hls4ml, born at CERN and Fermilab for trigger-level physics, converts trained models into HLS projects. As of the documentation I fetched in October 2026, the stable PyPI release is 1.3.0, with 1.4.0 development builds on the main branch. Frontends include Keras v2 and v3 (with QKeras, HGQ and HGQ2 quantization), PyTorch and ONNX (including QONNX). Backends listed are Vivado HLS (older, tested with 2020.1), Vitis HLS (tested 2022.2 to 2024.1), Catapult HLS (Siemens), and an Altera backend described as beta against version 2026.1. Intel HLS is marked deprecated and XLS is experimental. The tool runs on Linux with Python 3.10 or later.

Its control knobs map directly to hardware. The reuse factor sets how many times each multiplier is reused: a weight matrix of M by N entries with reuse factor RF uses at most M times N divided by RF multiplications per clock. RF of one is fully parallel and lowest latency; higher values trade latency for fewer DSP slices. The io_parallel mode wires one layer directly into the next and suits small models. The io_stream mode places FIFO buffers between layers and suits larger models, with the tool able to size FIFOs from RTL co-simulation.

The 2025 hls4ml paper reports, citing HGQ work, LUT reductions of 50 to 95 percent, DSP reductions of 50 to 100 percent, and latency reductions up to 80 percent with no reported accuracy loss. These come from the paper’s cited results on its own benchmarks, mostly physics models; they will not transfer unchanged to a ResNet-class vision backbone. The tool shines for small, heavily quantized networks. For large backbones, expect long synthesis times and difficulty fitting weights on chip.

FINN: streaming dataflow for quantized networks

FINN, from AMD’s research group, targets very low precision. Its documented flow: train a quantized network in Brevitas with quantization-aware training, export to QONNX, convert to FINN-ONNX, then run build_dataflow to generate the accelerator. The documentation describes FINN as more compiler infrastructure than finished compiler and notes it is still under development. It requires Vitis and Vivado 2024.2 or later, recommends Docker, and calls for at least 8 GB of RAM for Zynq-class targets and 64 GB for Alveo cards. Supported boards include Kria SOM, ZCU102, ZCU104, Ultra96 and AUP-ZU3, and Versal V80 via a separate package. Pynq-Z1 and Pynq-Z2 are no longer supported.

FINN generates a stitched IP block with AXI-stream interfaces, which is exactly what an industrial design wants: you drop it between your camera capture block and your output logic. Its design philosophy is full on-chip residency: every layer has its own engine and weights live in BRAM or LUTs. That yields spectacular efficiency for networks that fit, and a hard wall for those that do not.

Vitis AI: from DPU to NPU, and what changed

Vitis AI is the cautionary tale of toolchain drift, and a reason to read vendor documentation before committing. Earlier versions (up to the 3.x series, per the GitHub release notes) shipped fixed DPU overlays: DPUCZDX8G for Zynq UltraScale+, DPUCVDX8G for Versal, and others. The release notes show that DPUCZDX8G was not updated in the 4.0 or 5.0 releases, with updates focusing on Versal AIE DPUs; Vitis AI Library was noted as unavailable in 5.0. The current documentation I fetched presents Vitis AI as a platform for Versal AI Edge Series and Gen 2, with a compiler, NPU IP, runtime, the AMD Quark quantizer, and reference designs for the VEK280 and VEK385 kits. I could not confirm the exact release date or full feature list of the 6.x line; the page states a last-updated date but not a release date.

The practical lesson: if you have a Zynq UltraScale+ design on a Vitis AI DPU, plan for the toolchain to be frozen. Verify the current support matrix for your exact device before starting. For new designs on Versal AI Edge, the NPU overlay path is the vendor-supported direction.

Altera FPGA AI Suite and OpenVINO

The Intel-origin FPGA business operates as Altera, and its FPGA AI Suite takes the OpenVINO route. Models are converted with OpenVINO’s tools into intermediate representation files, the AI Suite compiler partitions the graph between FPGA and CPU, and the OpenVINO runtime API schedules inference across both. Per the product page, supported devices include Agilex 5, Stratix 10, Arria 10 GX and Cyclone 10, with bit-accurate software emulation through an OpenVINO plugin stated as available for Agilex 5 only. The page lists TensorFlow, PyTorch, Caffe, MXNet, Keras and ONNX as front ends and says most Open Model Zoo models are supported.

This is the lowest-friction path for teams already on OpenVINO, since the same application code can target CPU or FPGA. The trade-off is the same as any overlay: you get a parameterised engine, not a hand-fitted pipeline. For hard real-time with sub-millisecond bounds, check the achievable latency on your model before assuming the overlay matches a custom dataflow.

Quantization is the real gate

Every FPGA toolflow assumes a quantized model. Fixed-point weights of 8, 6, 4 or even 2 bits, and activations of similar width, are the norm. hls4ml supports fixed-point, power-of-two, ternary and binary types, with both post-training quantization and quantization-aware training. Post-training quantization to INT8 usually costs little accuracy on standard vision models. Below 8 bits, quantization-aware training is typically needed, and accuracy recovery becomes a project in itself.

Industrial datasets help here. A binary good-or-defective classifier on a controlled, well-lit part often tolerates 4-bit weights because the decision boundary is simple and the imaging is constrained. An open-world detector with subtle small defects is much less forgiving. Budget accuracy validation on a held-out set of real production defects, including rare ones, before deciding on bit width.

Custom IO, Legacy Interfaces and Lifecycle

The most underrated FPGA advantage is not inference at all. It is the ability to speak whatever protocol the factory uses, directly, with deterministic timing. Machine vision interfaces vary widely. According to Basler’s interface comparison, GigE Vision (including 5GigE) reaches up to 5 Gbps over cables up to 100 m, while CoaXPress 2.0 (CXP-12) reaches 12.5 Gbps per channel over up to 40 m. Camera Link, the older parallel-style standard still found on installed line-scan systems, needs a dedicated frame grabber. I did not verify Camera Link bandwidth figures for this post.

On a GPU module, a CoaXPress or Camera Link camera requires a frame grabber card, a driver, and a DMA path into system memory, and the Jetson or x86 host must have a slot and a vendor driver for your OS version. On an FPGA, the receiving logic can live in the same device as the inference engine. Pixels never touch a CPU. The same applies to hardware triggers, encoder inputs for line-scan synchronisation, strobe lighting control and reject-gate outputs, all of which need microsecond-level timing that a general-purpose OS does not give you cleanly.

The lifecycle argument

Industrial equipment is specified in decades. FPGA vendors publish long-term supply commitments for many device families, and a design in HDL or IP-block form can be re-targeted within a family. Consumer-derived GPU and NPU modules follow shorter cycles, though NVIDIA does offer industrial-grade Jetson variants (the AGX Orin Industrial is listed at 248 TOPS and up to 75 W in the comparison table I fetched) with longer support. The specific longevity commitments change often; get the current product-lifecycle document from the vendor rather than trusting a blog, including this one.

There is a counterweight. FPGA toolchains also age. A design built with Vivado HLS 2020.1 may not rebuild in a modern Vitis release, as hls4ml’s own backend notes (Vivado HLS is discontinued) imply. Archive the exact tool versions in a container or VM alongside the source, which is cheap insurance.

Functional safety and certification

For safety-related functions, the certification story differs. AMD’s Versal Gen 2 page advertises Cortex-R52 processors with up to 100k DMIPS offered for ASIL D and SIL 3 applications. That is a hardware capability claim; achieving a certified system still requires a safety case, qualified tools and process evidence. FPGA logic that is deterministic and has no operating system under the inference path is often easier to argue about than a driver stack, but nothing is automatic.

Development Cost, Time and Skills

Honest comparisons of FPGA versus GPU must include engineering effort, because it frequently dominates silicon cost at industrial volumes of hundreds to a few thousand units per year.

Dimension	GPU (Jetson class)	NPU / accelerator	FPGA with HLS toolflow
Time to first working inference	Hours to days	Days	Weeks
Iteration per model change	Minutes (retrain, export, TensorRT build)	Minutes to hours (compile)	Hours (HLS) to a day or more (place and route)
Skills needed	Python, CUDA/TensorRT	Python, vendor compiler	Python plus hardware timing, HLS, Vivado or Quartus
Operator coverage	Very broad	Vendor-limited	Whatever you can build, at cost
Latency bound	Statistical	Usually predictable per model	Cycle-exact
Custom IO	Via cards and drivers	Limited	Native
Per-unit silicon cost at volume	Moderate to high	Low	Wide range, often moderate

The time rows are my own experience-based estimates, not benchmarks. A realistic FPGA project for a team new to the platform runs three to six months to a production-quality inference path, including quantization-aware retraining, HLS tuning, integration with the camera interface and timing closure. A GPU project can reach a demo in a week. If the product has uncertain requirements or the model will change monthly, the iteration cost alone can eliminate the FPGA.

Build times and timing closure

Full place-and-route for a mid-size accelerator can take from tens of minutes to several hours, and a design that nearly fills the device can fail timing, requiring floorplanning or restructuring. The hls4ml paper reports that splitting a quantized ResNet into subgraphs cut synthesis from about seven hours to three on average, a reminder that even tuned flows measure in hours. Plan CI around this: run HLS C-simulation and quick estimates on every commit, and full builds nightly.

Verification differs

A GPU model’s output is checked against a reference in Python. An FPGA design needs bit-accurate agreement between the software model, the HLS C-simulation, the RTL co-simulation and the hardware. Fixed-point rounding and overflow behaviour create subtle mismatches. Tools help: Altera’s suite offers bit-accurate emulation on Agilex 5, and hls4ml supports co-simulation. Build a golden test set of real production images, including known defects, and diff outputs at every stage.

Trade-offs, Gotchas, and What Goes Wrong

On-chip memory caps model size. A dataflow design with weights in BRAM and UltraRAM cannot exceed the device’s memory. A modest detector with tens of millions of parameters at 8 bits needs tens of megabytes, which exceeds the on-chip memory of many mid-range FPGAs. Options: aggressive quantization and pruning, folding to reuse engines at higher latency, or spilling to external DDR, which brings back memory-bandwidth limits and jitter. The deterministic story weakens the moment DRAM enters the path, though DRAM controllers with fixed access patterns are still far more predictable than a shared GPU memory system.

HLS surprises. A pragma that works at one size fails to meet timing at another. Pipelined loops can silently fall back to a higher initiation interval. Resource estimates from HLS reports are often optimistic compared with post-implementation numbers. Always check the post-route report, not the synthesis estimate.

Model churn. Vision research moves quickly. Transformers and attention (hls4ml lists multi-head attention as experimental) are far harder to map than CNNs. If your accuracy plan depends on adopting next year’s architecture, an FPGA dataflow lock-in is a risk. An overlay-based flow (Vitis AI NPU, FPGA AI Suite) reduces that risk at the cost of efficiency.

Toolchain and licence drift. The Vitis AI history above shows the pattern: DPU targets got less attention while the vendor shifted to newer devices. Pin tool versions, and check licence costs for non-free device tiers (FINN’s documentation notes a Vivado/Vitis licence may be needed for non-WebPack parts).

GPU anti-patterns. Using a big GPU module at batch one and calling it fast; ignoring thermal throttling during summer; running other containers on the same module; relying on mean latency in a test lab with a cool, idle system. Test with the full application, at ambient temperature extremes, for hours.

NPU anti-patterns. Choosing an NPU on a single benchmark model and discovering the production model has an unsupported operator; assuming INT8 accuracy matches FP32 without checking; neglecting that host preprocessing (resize, colour conversion) may now dominate latency.

The hybrid trap. Splitting work across an FPGA front end and a GPU back end sounds like the best of both worlds. It often is, but the interface between them (DMA, PCIe, a shared buffer) reintroduces variance, and two toolchains double the maintenance. Use hybrids when the FPGA’s job is clearly bounded: capture, preprocessing, triggering. Keep the hard-real-time decision on the side that can bound it.

Practical Recommendations

Start from the requirement, not the silicon. Write down the latency bound including percentile (for example, 4 ms at p99.999), the maximum model size and operator list, the camera interface, the thermal envelope, the product lifetime and the team’s skills. Most decisions fall out of those six items.

Figure 4: A decision flow for edge inference silicon. Hard real-time bounds and custom IO point to FPGA; frequent model change points to GPU; fixed CNNs under tight power point to an NPU.

Figure 4 encodes a simplified heuristic. It is a starting point for discussion, not a substitute for measurement.

Scenario	Best fit	Why
Line-scan web inspection, CoaXPress or Camera Link, hard reject deadline	FPGA or SoC FPGA	Streaming, native IO, bounded latency
Multi-camera robot cell, changing models, relaxed deadline of tens of ms	GPU module	Flexibility, fast iteration, CUDA ecosystem
Battery or fanless sensor node, fixed small CNN	NPU	Lowest energy per inference
Safety-adjacent vision with long lifecycle and deterministic proof	FPGA SoC with Cortex-R cores	Cycle-exact path, hardware safety islands
Research prototype or proof of concept	GPU first	Time to result; port later if requirements demand
Transformer or large vision-language model at the edge	GPU (Thor class)	Memory capacity and operator coverage

A checklist before you commit

Measure p99.99 latency on the real application, not mean latency on a benchmark.
Normalise power claims to the same network, precision and batch size; measure wall power.
Confirm the target network quantizes to the planned bit width with acceptable accuracy on real defect data.
Check operator support against the vendor toolchain version you will actually ship.
Pin and archive toolchain versions, containers and licences.
Get the vendor’s current lifecycle and long-term supply statement in writing.
Prototype on an evaluation board (a Kria or Versal kit, a Jetson dev kit) before designing a custom carrier.
Plan a fallback: keep a GPU reference implementation to validate FPGA output bit for bit.

A sensible 2026 strategy for many teams is a two-step path. Prove the model and accuracy on a GPU or generic NPU. If the latency bound or IO demands it, port the stable model to an FPGA toolflow, using the GPU version as the golden reference. This captures the speed of GPU iteration and the determinism of the FPGA where it earns its cost.

Frequently Asked Questions

Is an FPGA faster than a GPU for edge inference?

It depends on what you measure. A large GPU has higher peak throughput, especially with batching. An FPGA dataflow design usually wins on worst-case latency at batch size one, because the pipeline is fixed and nothing is shared or scheduled. For small, heavily quantized networks it can also win on energy per frame. For large models or fast-changing architectures, the GPU typically wins on practicality and often on raw speed.

Can FPGAs run deep learning without writing VHDL or Verilog?

Yes. High-level synthesis tools such as hls4ml generate FPGA designs from Keras, PyTorch or ONNX models, and FINN builds dataflow accelerators from Brevitas-trained networks. Overlay-based flows such as Vitis AI for Versal AI Edge or Altera’s FPGA AI Suite with OpenVINO compile models onto prebuilt engines. You still need to understand quantization, resource limits and timing, but not register-level RTL for most standard layers.

What is the difference between an NPU and an FPGA for vision?

An NPU is a fixed array of multiply-accumulate units driven by a compiler schedule, optimised for supported operators at low power. An FPGA is reconfigurable logic you wire into a custom pipeline. NPUs are easier to use and often more energy-efficient for standard CNNs. FPGAs offer cycle-exact latency, arbitrary precision and custom IO. Many modern adaptive SoCs combine both, with hardened AI engines beside programmable logic.

Is Vitis AI still supported in 2026?

Vitis AI is still documented, but its scope has shifted. The current documentation describes it as a platform for Versal AI Edge Series and Gen 2 devices, with an NPU IP, compiler, runtime and the Quark quantizer. GitHub release notes show the Zynq UltraScale+ DPU not updated in versions 4.0 and 5.0. Confirm the support matrix for your exact device and tool version before starting a design.

How much does FPGA development cost compared to GPU?

Expect higher engineering cost and longer schedules for the FPGA, commonly months rather than days for the first production-quality inference path, based on the author’s estimate rather than a published benchmark. Silicon cost can be competitive at moderate volumes. The deciding factors are model stability, volume, and whether determinism or custom IO is required. If neither is required, GPU or NPU development is almost always cheaper overall.

Which is best for industrial machine vision in 2026?

No single answer wins. Choose an FPGA or SoC FPGA for hard real-time rejects, line-scan streaming, legacy interfaces and long lifecycles. Choose a Jetson-class GPU for flexible multi-model cells and large networks. Choose an NPU for fixed CNNs under tight power and cost. Prototype on a GPU first, measure the tail latency, and move to an FPGA only when the measurements demand it.