Local language models in production: what the machine is allowed to do
A language model in a browser is allowed to be wrong. The user reads the answer, frowns, asks again. Production has no such corrective: there, a wrong answer becomes a wrongly encoded label, a misconfigured reader, a batch of scrap. Anyone letting language models near production processes should therefore answer an architecture question first, and it is not the one usually asked.
The wrong question: how good is the model?
Benchmarks are the currency of the AI industry, and on the shop floor they are almost beside the point. A model with 99 percent accuracy sounds impressive, but across 10,000 labels it produces a calculated 100 faulty ones if its output lands on the chip unchecked. No serious operation would accept a machine on those terms. The question is not how often the model is right. The question is what happens when it is wrong.
That turns a quality question into a permissions question: what is the model allowed to do at all?
Two lanes: understanding and execution
The answer we consider sound is a strict separation into two lanes.
The data path is classic, deterministic software: vetted recipes, a builder for the data structures, writing with a full read-back, an audit log. No language model exists here. Every step is reproducible and testable like any other piece of industrial software.
The language model sits in front of it and does exactly two things: it recognises which job pattern is present, and it extracts values from the job text. What matters is the form of its output: no free text, but structured fields, and every adopted value carries the literal quote from the source as evidence. Anything unevidenced is flagged as a suggestion or stays empty. The job is assembled by the data path, never by the model.
In between stands the human, with a display that makes the difference visible: evidenced, suggestion, missing. And as the final gate, a first-piece check in which the first physical part is read back and confirmed in plain text before the batch runs.
In one sentence: the AI never writes. If the model is wrong, the job dies at validation or at the first-piece check. The error never reaches the workpiece.
The vocabulary already exists, just elsewhere
The vocabulary for this principle comes not from mechanical engineering but from IT security. The OWASP Top 10 for LLM applications list "Improper Output Handling" (never pass model output unchecked to downstream systems) and "Excessive Agency" (needlessly granting models execution rights) as risk classes of their own. NVIDIA's NeMo Guardrails dedicate a rail class of their own to this, "execution rails", gating tool and action calls. And constrained decoding, enforcing a schema at token selection time, hardens the extraction further.
The closest relative in research is the Fraunhofer IOSB project LLM4OPC, which makes industrial machines addressable in natural language. There, a tool agent executes after human approval. The two-lane principle is the stricter variant: the model does not execute after approval, it never executes.
Why local rather than cloud
Two arguments favour running the model on the device rather than in the cloud, one technical and one practical.
The technical one: constrained decoding needs access to the model's token probabilities. Those who run the model themselves, locally with open weights, control this layer directly and can verify that control. Against a cloud API it remains an assurance given by the provider.
The practical one: data sovereignty is no niche concern in Germany. In Bitkom's March 2026 survey, 77 percent of companies named data protection requirements as an obstacle to digitisation, more often than anything else; in a separate 2025 Bitkom survey, 70 percent said they had stopped innovation plans at least once because of data protection rules. And since NIS2 came into force in Germany, affected companies pass their security requirements down the supply chain via supplier questionnaires. A device whose language model runs locally and which operates without a cloud connection answers most of those questions by itself: the job data never leaves the building.
There is also a European dimension, and it is shifting right now. With projects like Soofi, the Sovereign Open Source Foundation Models, large open models are emerging that are developed and trained entirely in Germany: funded by the Federal Ministry for Economic Affairs and Energy, coordinated by the German AI Association (KI Bundesverband); the first model generation, Soofi S, is approaching its open release, announced for this year. For operators this means two things. A real European alternative to non-European models is within sight. And only an open-weights architecture can adopt it: those who build locally with interchangeable models today keep the path to European models open without rebuilding the application. A contract with a cloud API outside Europe does the opposite. Sovereignty is not decided at training time alone, but every day where the model runs and where the job data flows.
What the EU AI Act says
The European AI Act is not an obstacle to this architecture principle but a tailwind. Following the Digital Omnibus of July 2026, the high-risk obligations of Annex III apply from late 2027, and shop-floor assistance is not among the areas listed there in the first place. What already applies today is the transparency obligation of Article 50: the operator must be able to tell that they are interacting with an AI system. That is a notice in the interface, not a hurdle.
One detail from the European Commission's guidelines helps further: systems based exclusively on rules defined by humans are not AI systems within the meaning of the regulation. A cleanly separated deterministic data path thus remains what it is, classic industrial software. What gets regulated is exactly the component that is not allowed to write anyway. One more reason the separation pays off.
What this means for Auto-ID
In our industry the bottleneck is rarely the hardware; it is the expertise in the operating workflow. Setting up RFID or NFC encoding means knowing memory banks and lock modes, choosing an EPC scheme with partition and filter values, managing access and kill passwords and, for NFC, mastering NDEF record structure. Established label software mirrors exactly that knowledge, in dialogs built for specialists.
A language model changes none of it, and that is the point: encoding itself is deterministic and stays that way. What can change is the access. If a production worker can describe the job the way it reads on the job ticket, and a system turns that into evidenced, checkable suggestions executed by deterministic code, a specialist process becomes operable without the specialist knowledge living in the operator's head. It lives in vetted recipes, a chip database and a verification workflow instead.
We are currently building a device along this pattern for a specific encoding process at label manufacturers. More here once it carries its name. If you want to go deeper before then: we provide vendor-independent consulting on RFID and NFC projects, from chip selection to encoding — talk to us.
Sources: OWASP Top 10 for LLM Applications 2025; NVIDIA NeMo Guardrails documentation; Fraunhofer IOSB, LLM4OPC project; Regulation (EU) 2026/1744 (Digital Omnibus on AI); Bitkom press releases of 11 March 2026 and 23 May 2025; German NIS2 implementation act (NIS2UmsuCG); Soofi project information, soofi.info (as of August 2026).
