A Local LLM in a Factory, With Nothing Leaving the Building

Ask a plant manager whether the maintenance team can paste a fault code and a photo of a manual page into a public chatbot, and you will usually get one of two answers: "of course" or "absolutely not." Both are usually wrong in the same way. Nobody has decided which information is which.

Running a language model on a server inside your own building is no longer the hard part. A mid-range GPU can serve a capable open-weight model, and the tooling (Ollama, llama.cpp, vLLM and others) is mature. The hard part is three questions:

1. Which factory data must never leave, and which can?
2. Where does the model sit relative to the plant network?
3. How do you prove nothing leaves, to yourself and to an auditing customer?

This post is a design guide for those three questions. For GPU and memory sizing, we already have a separate hardware sizing guide (Thai version), so we will not repeat it.

1. Sort your data into three buckets first

Before choosing a model, classify what the model will see. A workable split:

Bucket	Examples	Where it can be processed
Free to send	Public datasheets, generic maintenance know-how, marketing copy	A shared cloud API is acceptable
Only under contract	Internal SOPs, non-critical reports	A cloud service with a written no-training commitment, ideally inside your own cloud account
Never leaves	Customer drawings, process recipes, measurement and quality data, tooling details, anything under a customer NDA	Your own infrastructure only

The third bucket is where factories differ most from offices. A precision-parts maker’s drawings, tolerances and process parameters are the product. For suppliers to Japanese customers, there is an additional reason to be careful: under Japan’s trade-secret framework, handing information to a third party, such as a generative-AI provider, can in some cases undermine the "secrecy management" a trade secret depends on. One Japanese commentary also notes that June 2026 guidelines from Japanese competition and SME authorities say a party holding another’s confidential information should not disclose it to third parties without the other side’s prior consent, so it is worth checking whether your NDAs treat an AI service as a third party. (This is a secondary reading of those guidelines, not legal advice; confirm with counsel and the primary text.)

The practical ladder, from loosest to tightest control:

1. A shared external API
2. A shared API with a no-training contract
3. A managed deployment inside your own cloud subscription
4. Your own hardware, optionally air-gapped

Bucket three needs level 4. That is the case for a local LLM.

2. What a shop-floor model is actually good for

Be modest. A local model of roughly 8B–30B parameters is good at language work grounded in your documents, and not good at being an oracle.

Good fits

• Maintenance Q&A with citations. "What is the torque spec for this spindle cover?" answered from your manuals and cited back to the page.
• Fault-code and alarm explanation drawn from your own history of past work orders.
• Drafting. Shift handover notes, structured work orders from a technician’s free-text, incident summaries.
• Search across messy documents in Thai, Japanese, Chinese and English, where keyword search fails.

Poor fits

• Anything that writes to a PLC or control system. Keep the model read-only and require a human to approve any action.
• Numeric reasoning you cannot verify. Calculations belong in code, not in a model’s head.
• Authoritative answers without a source. If the model cannot cite a document, treat the answer as a suggestion.

3. Where the model sits: keep it out of the control network

flowchart TD
  A["Control network PLC and machines"] --> B["Read only data gateway"]
  B --> C["Plant data zone historian and MES"]
  C --> D["AI zone LLM server and document index"]
  E["Manuals SOPs and work orders"] --> D
  D --> F["Users on the office or shop floor network"]
  D --> G["Audit log of every query and answer"]
  H["Internet"] -.->|"blocked by default"| D

Design rules that matter more than the model choice:

• Read-only from below. The AI zone can read from the plant data zone through a gateway. Nothing flows back into the control network.
• Default-deny outbound. The LLM server should have no route to the internet. Model weights and updates arrive through a controlled, offline import.
• Log every query and answer, with the document passages used. This is your audit trail.
• Humans approve actions. The model recommends; a person decides.

4. How to prove nothing leaves

"Nothing leaves the building" is a claim. Customers and auditors will want evidence. Four checks:

1. Firewall evidence. A default-deny egress rule for the AI zone, with the rule set exported and reviewed.
2. Observed traffic. Capture traffic from the LLM server for a representative week and show there are no outbound connections to unexpected destinations. Repeat after every software upgrade.
3. Offline supply chain. Record where model weights came from, check their hashes on import, and pin versions. Frameworks change defaults between releases, so retest after upgrades.
4. No hidden telemetry. Review the runtime’s and every library’s configuration for update checks and usage reporting, and turn them off or block them at the firewall.

5. Sizing in one paragraph

For interactive use by a handful of people, a single 24 GB GPU runs 7B–14B models comfortably and 27B–32B models at 4-bit quantization if you leave room for context (plan for 20–30% VRAM headroom beyond the weights). The limit is usually concurrency and context length, not model quality. If twenty technicians ask questions at the same time, you need either a serving stack built for batching or more hardware. Our sizing guide has the tiers and the memory arithmetic.

6. What we have shipped, and what is project work

To keep this honest:

• Shipped in simpliSOC: a local LLM that reads qualifying security alerts and writes a plain-language explanation and a true/false-positive assessment. It works with any OpenAI-compatible server (Ollama, llama.cpp, vLLM, LocalAI). The model runs on-premise and alert content stays on your network. It assists triage only: it never closes, merges or suppresses an alert, and an analyst stays in the loop. Details are on the simpliSOC page. This is a security-operations tool, not a shop-floor assistant.
• Project work, scoped per customer: an on-premise LLM service for manufacturers (model selection, an OpenAI-compatible API layer, a RAG pipeline over manuals and production records, guardrails and audit logging, and integration with ERP or MES). See simpliLLM. It is delivered as an engagement, not a boxed product, so scope and timing depend on your documents and hardware.

For the retrieval side, see our post on production RAG for Thai and Japanese business documents, and for the risks of putting a model into production, AI security in production.

7. FAQ

Is a local model as good as a cloud model?
Not at the top end. For document-grounded tasks such as manual lookup and drafting, a well-fed smaller model is often sufficient, and the right test is your own documents and questions, not a public benchmark.

Does "local" mean "secure"?
No. It removes one risk (data sent to a third party). You still need access control, patching, segmentation and logging.

Can the model control machines?
We recommend against it. Keep it read-only and require human approval for any action.

Do we need a GPU?
For more than light use, yes. CPU-only inference works for small models and a single user, but it is slow.

How do we handle Thai, Japanese and Chinese documents?
Quality varies by model and by document type (scans, tables and drawings are harder). Evaluate on a sample of your real documents before committing to hardware.

Is this legal advice for NDAs and trade secrets?
No. Check your contracts and consult counsel.

Sources

• Trade secrets and generative AI for engineering drawings, Timewell (summary of Japanese guidance)
• iFactory: on-premise LLM architecture for factories (vendor blog; used for use-case ideas only)
• IntuitionLabs: local LLM deployment on 24 GB GPUs

---

Simplico Co., Ltd. builds on-premise AI, SOC and manufacturing systems for companies in Thailand, Japan and across Asia. Write to hello@simplico.net for a readiness assessment.