Internet-in-a-Box

USER MANUAL: Off-Grid Knowledge Node & Local AI Deployment

Target Hardware: 1 TB Portable Solid-State Drive (SSD)

Supported OS: Windows 10/11, macOS (Apple Silicon / Intel), Linux (Ubuntu/Debian-based)

Revision: 1.0

1. System Requirements & External SSD Formatting

Before setting up your offline node, you must format the external drive correctly. Modern offline archives (such as full Wikipedia .zim files) frequently exceed 4 GB in single-file size, making legacy file systems like FAT32 unusable.

1.1 Minimum Specifications

• External Storage: 1 TB Solid-State Drive (SSD) via USB 3.0 or higher.

  
  

• System Memory (RAM): 16 GB minimum (32 GB recommended for smooth LLM inference).

  
  

• Processor: Multi-core CPU; dedicated GPU with $\ge 8\text{ GB}$ VRAM (NVIDIA RTX series or Apple M-series) highly recommended for AI performance.

  
  

1.2 Format the Drive (exFAT)

exFAT is required for cross-platform support across Windows, macOS, and Linux without file-size limitations.

• Windows: Open File Explorer $\rightarrow$ right-click your external drive $\rightarrow$ Format... $\rightarrow$ File System: exFAT $\rightarrow$ Allocation Unit Size: Default $\rightarrow$ Click Start.

  
  

• macOS: Open Disk Utility $\rightarrow$ select the external physical drive $\rightarrow$ click Erase $\rightarrow$ Name: OfflineNode $\rightarrow$ Format: exFAT $\rightarrow$ Scheme: GUID Partition Map $\rightarrow$ Click Erase.

  
  

• Linux: Open terminal and run:

  
  

  Bash

  sudo mkfs.exfat -n OfflineNode /dev/sdX1

  (Replace /dev/sdX1 with your actual drive identifier).

  
  

2. Standard Directory Layout Setup

To ensure software components find their respective files, establish the following folder structure on the root of your newly formatted external SSD.

1.Create Root Directories:

Open your external drive root directory (OfflineNode or assigned drive letter like E:\) and create a primary folder named Data.

2.Create Data Sub-Folders:

Inside Data/, create three distinct directories:

• Data/ZIM_Library/ — Stores all offline .zim encyclopedia and book archives.

  
  

• Data/Ollama_Models/ — Stores model weight binaries, manifests, and blobs.

  
  

• Data/AnythingLLM_Storage/ — Stores AnythingLLM vector caches, workspace configurations, and local document databases.

  
  

3.Create Installer Folder (Optional):

On the root of the SSD, create a folder named Installers/. Save copies of the executable installation files for Kiwix, Ollama, and AnythingLLM here so you can deploy the suite to other host computers without internet access.

3. Kiwix Setup & Library Mounting

3.1 Software Installation

1. Download Kiwix Desktop for your operating system from the official Kiwix distribution page.

   
   

2. Run the installer or place the portable executable into your drive's Installers/ directory.

   
   

3.2 Download Popular .zim Archives

Download your preferred .zim packages directly into your external drive directory: <DRIVE_ROOT>/Data/ZIM_Library/.

Dataset Name	Description	Approximate Size	Recommended Download
Wikipedia (Full)	Complete English Wikipedia with all images	$\sim 100\text{ GB}$	wikipedia_en_all_maxi.zim
Wikipedia (Mini/No Images)	Complete text of Wikipedia without images	$\sim 15\text{ GB}$	wikipedia_en_all_nopic.zim
Project Gutenberg	Entire collection of over 60,000 public domain ebooks	$\sim 65\text{ GB}$	gutenberg_en_all.zim
Stack Overflow	Complete developer Q&A platform archive	$\sim 50\text{ GB}$	stackoverflow_en_all.zim
Wikimed	Comprehensive medical & healthcare encyclopedia	$\sim 20\text{ GB}$	wikimed_en_all.zim

3.3 Link Kiwix to the External Drive Directory

1. Open Kiwix Desktop.

   
   

2. Navigate to Settings (gear icon) or select Local Books in the navigation menu.

   
   

3. Select Add Storage / Add Local File or change the default library directory to:

   
   

   • Windows: E:\Data\ZIM_Library\

     
     

   • macOS: /Volumes/OfflineNode/Data/ZIM_Library/

     
     

   • Linux: /media/<username>/OfflineNode/Data/ZIM_Library/

     
     

4. Click Open Archive and choose any .zim file from that directory. Kiwix will automatically index and display all ZIM files contained in the designated folder.

   
   

4. Redirecting Ollama Model Repository to the SSD

By default, Ollama attempts to write large Large Language Model files to your OS system drive (C:\Users\<User>\.ollama or ~/.ollama). You must configure system environment variables or directory links to point directly to the external SSD.

Option A: System Environment Variable Method (Recommended)

On Windows:

1. Press Win + R, type sysdm.cpl, and press Enter (opens System Properties).

   
   

2. Go to the Advanced tab and click Environment Variables.

   
   

3. Under User variables, click New...:

   
   

   • Variable name: OLLAMA_MODELS

     
     

   • Variable value: E:\Data\Ollama_Models (Replace E: with your SSD drive letter).

     
     

4. Click OK on all prompts to confirm.

   
   

5. Restart your terminal or command prompt.

   
   

On macOS / Linux:

1. Open your terminal configuration file (~/.zshrc or ~/.bashrc) in a text editor:

   
   

   Bash

   nano ~/.zshrc

2. Add the following environment variable declaration at the bottom of the file:

   
   

   Bash

   export OLLAMA_MODELS="/Volumes/OfflineNode/Data/Ollama_Models"

   (On Linux, adjust the path to your mount point, e.g., /media/$USER/OfflineNode/Data/Ollama_Models).

   
   

3. Save the file (Ctrl + O, Enter, Ctrl + X) and reload the environment:

   
   

   Bash

   source ~/.zshrc

Option B: Directory Junction / Symbolic Link Method

If environment variables are reset by system updates, create a symbolic link directly from the default folder path to the drive path.

• Windows (Command Prompt as Administrator):

  
  

  DOS

  rmdir /S /Q "%USERPROFILE%\.ollama\models" mklink /J "%USERPROFILE%\.ollama\models" "E:\Data\Ollama_Models"

• macOS / Linux (Terminal):

  
  

  Bash

  rm -rf ~/.ollama/models ln -s /Volumes/OfflineNode/Data/Ollama_Models ~/.ollama/models

4.1 Download Local LLMs to the External SSD

With Ollama installed and the environmental link set, run the following commands in your terminal to download models directly to the SSD:

Bash

# General multi-purpose model (8B parameter)
ollama run llama3.2

# Coding and technical query focus model
ollama run qwen2.5-coder

Verify that the downloaded assets populate inside <DRIVE_ROOT>/Data/Ollama_Models/blobs/.

5. AnythingLLM Installation & Configuration

5.1 Storage Redirection for AnythingLLM

To keep all local user databases, uploaded document indexes, and vector embeddings on the SSD:

On Windows:

Create a junction link from the default desktop application storage directory to your SSD:

DOS

mklink /J "%APPDATA%\anythingllm-desktop\storage" "E:\Data\AnythingLLM_Storage"

On macOS:

Bash

ln -s "/Users/$USER/Library/Application Support/anythingllm-desktop/storage" "/Volumes/OfflineNode/Data/AnythingLLM_Storage"

5.2 Application Setup & Integration

1. Launch AnythingLLM Desktop.

   
   

2. Complete the initial onboarding wizard:

   
   

   • LLM Provider Selection: Select Ollama.

     
     

   • Ollama Base URL: Ensure it points to [http://127.0.0.1:11434](http://127.0.0.1:11434).

     
     

   • Model Selection: Select llama3.2:latest (or your preferred downloaded model) from the drop-down menu.

     
     

   • Vector Database: Select LanceDB (default lightweight local engine).

     
     

   • Embedding Engine: Select AnythingLLM Embedder or Ollama (e.g., nomic-embed-text).

     
     

3. Click Save & Continue to complete workspace setup.

   
   

6. Air-Gap Testing & Verification Protocol

To verify complete offline operation without dependence on external internet connections, perform this step-by-step test procedure.

1.Disconnect Host Machine Network Interfaces:

Turn off Wi-Fi on the host computer and disconnect all Ethernet cables. Ensure the machine shows no active network links.

2.Test Offline Kiwix Reader Access:

Open Kiwix Desktop. Search for a specific article (e.g., "Quantum Mechanics" in Wikipedia or "Pride and Prejudice" in Gutenberg). Confirm that images and full text load instantaneously from the SSD without throwing network timeout errors.

3.Verify Ollama Standalone Execution:

Open a command terminal while offline and execute:

Bash

ollama run llama3.2 "Explain the basic principles of an electric motor."

Verify that the model streams responses locally without making network calls.

4.Verify AnythingLLM RAG Workflow:

1. Open AnythingLLM Desktop.

   
   

2. Create a new workspace named Offline Documentation.

   
   

3. Upload a local text file or PDF into the workspace document manager.

   
   

4. Click Move to Workspace, then click Save and Embed Workspace.

   
   

5. Query the workspace regarding the specific contents of your uploaded document. Confirm the system generates answers using the local embedding model and Ollama LLM backend.

   
   

7. Troubleshooting & Maintenance

• Issue: Ollama keeps downloading models to the internal C: or System Drive.

  
  

  • Solution: Confirm the daemon process was restarted after configuring system variables. On Windows, right-click the Ollama tray icon, click Quit, and relaunch Ollama from the Start Menu.

    
    

• Issue: Kiwix fails to locate ZIM files after moving to another PC.

  
  

  • Solution: Drive letters may change across different host Windows systems (e.g., E: changing to F:). Update the Kiwix library path via Settings $\rightarrow$ Add Local Directory to match the new drive letter assignment.

    
    

• Issue: LLM performance is extremely slow.

  
  

  • Solution: In AnythingLLM settings, check system resource usage. If running purely on CPU, lower the model quantization level or switch to a smaller parameter model (e.g., llama3.2:1b or phi3:mini).