AI infrastructure · 2026

Where should industrial AI run? The cycle time decides, not the architecture diagram

Cloud or edge is usually debated as a platform preference. On a production line it is not a preference — it is arithmetic between the response time you can get and the cycle time you have.

Latency is a constraint, not a preference

A cloud inference round trip typically lands somewhere between 200 and 2,000 milliseconds once network, queueing and the model itself are counted. Edge inference on local hardware typically returns in 10 to 100 milliseconds. If a decision has to be made inside a machine cycle — reject this part, adjust this parameter, stop this axis — the cloud number does not fit and no amount of architecture will make it fit. If the decision changes a plan rather than a part, it fits comfortably.

200–2,000 ms

typical cloud inference round trip

10–100 ms

typical edge inference response

~2/3

share of AI compute now going to inference rather than training

The cost curve points the same way

For high-volume, continuous inference the economics follow the physics. Total-cost analyses put edge deployment 30–50% cheaper than cloud over a five-year horizon for exactly this profile — a camera or sensor stream that never stops. Per-inference costs have fallen roughly an order of magnitude on local hardware over the past few years. A line inspecting every part around the clock is the worst possible fit for per-call cloud pricing and the best possible fit for a device you buy once.

The split that works

In practice the question is not cloud or edge but which half of the problem goes where. The division below is boring, which is why it survives contact with a factory.

1. 01

   Train centrally

   Where the data, the accelerators and the version history live. Training is not latency-sensitive.

2. 02

   Infer at the edge

   Where the cycle time, the safety case and the network reality demand an answer now.

3. 03

   Aggregate back

   Drift monitoring, retraining sets and fleet-wide comparison need the central view.

4. 04

   Fail safe locally

   The line keeps a defined, documented behaviour when the link is down — because it will be.

What actually breaks

The failures in edge deployments are rarely about model accuracy. They are about fleet management: forty devices running three model versions with no record of which; hardware that was current at pilot and unobtainable at rollout; no rollback path when a new model performs worse on one plant’s lighting; and a connectivity assumption nobody wrote down. The discipline this requires is closer to industrial maintenance than to machine-learning research — versioning, spares, documented rollback, and someone who owns the fleet.

A rule you can use in a meeting

If the decision changes something inside one machine cycle, it belongs at the edge. If it changes something within a shift, a day or a week — scheduling, maintenance priorities, energy and cost calls — the cloud is not just acceptable, it is the better place for it. Most real systems need both, and the useful conversation is about the boundary, not the platform.

Before choosing hardware, write down the cycle time the decision has to fit inside. That single number eliminates most of the options.