Air-Gapped AI for Government and Defense: Deploying LLMs in Disconnected Environments

The Hardest AI Deployment Problem Nobody Talks About

Every AI vendor’s sales deck assumes internet connectivity. Model APIs, cloud inference, SaaS dashboards, telemetry — the entire commercial AI ecosystem is built on the assumption that your network talks to theirs.

For government agencies, defense contractors, intelligence organizations, and critical infrastructure operators, that assumption is wrong. Many of the environments that need AI most — classified networks, SCIF facilities, OT/ICS networks, forward-deployed systems — are partially or fully disconnected from the internet by design.

Air-gapped AI isn’t a niche requirement. It’s where the highest-value use cases live: intelligence analysis, logistics planning, maintenance prediction, document processing at scale, and decision support where latency to a cloud endpoint is unacceptable.

What “Air-Gapped” Actually Means for AI

A true air-gapped deployment has zero network connectivity to the outside world. That creates specific constraints:

What You Can’t Do

Call any external API (OpenAI, Anthropic, Google, etc.)
Pull model updates from the internet
Use cloud-based RAG or embedding services
Send telemetry or crash reports
Access package repositories for live updates

What You Need Instead

Self-contained model weights transferred via approved media (USB, optical, cross-domain solution)
All dependencies pre-bundled — no pip install or npm install at runtime
Local embedding models for RAG/vector search
Offline documentation for operators
Update mechanism through your existing cross-domain transfer process

Reference Architecture

┌─────────────────────────────────────────────┐
│              Air-Gapped Network             │
│                                             │
│  ┌─────────┐    ┌──────────┐    ┌────────┐ │
│  │  Users   │───▶│ Open     │───▶│ vLLM   │ │
│  │ (Browser)│    │ WebUI    │    │ (Local) │ │
│  └─────────┘    └────┬─────┘    └────────┘ │
│                      │                      │
│              ┌───────┴────────┐             │
│              │ Safety Filters │             │
│              │ • Injection    │             │
│              │ • Content      │             │
│              │ • Policy       │             │
│              └───────┬────────┘             │
│                      │                      │
│         ┌────────────┼────────────┐         │
│         │            │            │         │
│    ┌────┴────┐  ┌────┴────┐  ┌───┴──────┐  │
│    │PostgreSQL│  │ Chroma/ │  │  Audit   │  │
│    │ (State)  │  │pgvector │  │  Logs    │  │
│    └─────────┘  └─────────┘  └──────────┘  │
│                                             │
└─────────────────────────────────────────────┘

Every component runs locally. The entire stack is containerized (Docker/Podman) and can be deployed from a pre-built image transferred through your approved media process.

Model Selection for Air-Gapped Environments

Not all open-weight models are equal in disconnected settings. Key selection criteria:

Factor	Why It Matters
License	Must permit government/defense use. Llama 3, Mistral, Gemma — all permissive.
Size vs. hardware	Match model parameters to available GPU memory. No cloud overflow option.
Quantization support	GPTQ/AWQ/GGUF lets you run larger models on smaller hardware.
Embedding model	You need a local embedding model for RAG — not just the chat model.
Tokenizer	Must be bundled with the weights, not downloaded at runtime.

Recommended starting points:

General purpose: Llama 3 70B (2x A100) or Llama 3 8B (single RTX 4090)
Code/technical: CodeLlama 34B or DeepSeek Coder
Embedding: BGE-large-en-v1.5 or E5-large-v2 (both run on CPU)

Operational Considerations

Updates and Patching

Model updates, security patches, and configuration changes follow your existing cross-domain transfer process. Pre-build the update package on an internet-connected staging system, validate checksums, transfer via approved media.

User Management

Open WebUI supports local authentication or integration with on-premise directory services (LDAP/Active Directory). No external identity provider required.

Monitoring Without Telemetry

Usage tracking runs entirely locally — per-user token counts, model utilization, safety filter triggers — all stored in your PostgreSQL instance. Export reports on your schedule through approved channels.

Multi-Classification Support

For environments spanning multiple classification levels, deploy separate instances per network with no cross-connects. Each instance is self-contained.

SAM.gov and Federal Procurement

For federal agencies exploring AI capabilities, I also maintain a SAM.gov contract opportunity search tool that integrates directly into the AI chat interface — useful for acquisition teams evaluating AI-related contracts and vendors.

Deployment Approach

I deploy air-gapped AI stacks as pre-built, transferable packages:

Staging build — full stack assembled and tested on internet-connected hardware
Validation — security scan, dependency audit, checksum generation
Transfer — pre-built container images + model weights via approved media
Deployment — docker compose up on target hardware, configure auth, load policies
Acceptance — functional test, safety filter validation, user acceptance

No internet required at the deployment site. No ongoing external dependencies.

All infrastructure tooling is open source. The only proprietary component is the model weights themselves — and those are open-weight with permissive licenses.

Book an intake call →

Air-Gapped Government Defense FedRAMP SCIF Private LLM