GPU Selection for Edge AI Inference: TGP, TDP and Enclosure Thermal Limits
Choosing a GPU for a fanless edge AI box PC is a thermal decision before it is a compute decision: TGP must fit the chassis cooling budget and the DC power budget. This guide walks through the numbers that actually gate the choice.
Quick answer
For a fanless box PC, pick the GPU by TGP (total graphics power) first: the chassis cooling design caps what can be passively dissipated (practically up to ~50–75 W), and the DC input budget caps total system draw. Compute classes (TOPS, VRAM) come second. A model that cannot dump its heat into the fins at 50°C ambient will throttle no matter how good its benchmarks are.
The three budgets to check
| Budget | What it limits | Typical fanless ceiling |
|---|---|---|
| Chassis thermal (passive) | GPU TGP + CPU TDP the fins can dissipate | ~50–75 W GPU class |
| DC input | Total system power on 9–36 V input | Size PSU for CPU+GPU+peripherals +30% |
| VRAM / workload | Model size at batch size you actually run | 6–16 GB workstation class fits most vision inference |
GPU classes that fit fanless designs
- Integrated graphics (Intel UHD/Iris, AMD Radeon iGPU): 0 extra TGP; fine for light vision preprocessing, video decode (quick sync), small ONNX models.
- NVIDIA low-profile workstation GPUs (A2000/A4000/A4500 class): 50–130 W — the lower end works in hybrid-cooled (passive+small fan) designs; full fanless usually tops out near the 50–75 W range.
- NVIDIA Jetson modules (Orin NX/Nano): 10–60 W mode-configurable, excellent TOPS-per-watt, native support in the Jetson software stack — the default choice for strict fanless designs.
- Consumer gaming GPUs: 150–450 W TGP — not viable passively; require a fan-cooled IPC with filtered airflow.
Design notes
- Request the thermal validation report: sustained inference load at rated ambient, GPU junction temperature logged. A 30-minute demo is not a thermal test.
- On DC-powered vehicles/AGVs, include inrush and transient margins; GPU power spikes are fast and deep.
- For headless inference, install the
nvidia-headlessdriver stack (Linux) to skip display init and save RAM. - Peak TOPS marketing numbers assume sparsity and specific precisions; benchmark your model at your batch size.
Frequently Asked Questions
How much GPU power can a fanless box PC dissipate?
Practically, passive cooling supports GPUs in the 50-75 W TGP range with a well-designed finned chassis at moderate ambient temperatures, sometimes more with hybrid cooling that adds a small temperature-controlled fan. Beyond that, heat cannot leave the enclosure fast enough and the GPU throttles.
Should I use a Jetson module or a workstation GPU for edge AI?
Jetson modules win on power efficiency (10-60 W, high TOPS-per-watt) and are the default for strict fanless and battery-powered designs. Workstation GPUs (A2000/A4000 class) win on raw throughput and the full CUDA ecosystem for larger models, but need more thermal headroom.
Why does my GPU inference slow down after a few minutes?
That is thermal throttling: the GPU reduced clocks because junction temperature hit its limit. In a fanless enclosure it means the sustained TGP exceeds what the chassis can dissipate at your ambient temperature — the fix is hardware selection (lower-TGP GPU or better thermal design), not software tuning.