ODS, the Osmantic Deployment System, installs and wires a local AI server onto a Windows, macOS, or Linux machine, so the pieces of a self-hosted stack arrive pre-connected rather than assembled by hand. One installer command detects the GPU, picks a model for the detected memory envelope, generates credentials, starts the services, and opens a local web UI.

The stack covers local inference, a chat interface, a control dashboard, voice input and output, agents and workflow automation, retrieval and search, image generation, and privacy and observability tooling. Local mode is the default; cloud and hybrid API modes are optional, and every port is configurable through environment variables.

Hardware detection assigns a deterministic tier and then a catalog selector reads a model library to choose an installable GGUF file, writing the choice into .env. Bootstrap mode downloads a small model first so a chat session works within minutes while the full model downloads in the background, then hot-swaps; if a new model fails to load, the previous model is restored.

Every service is an extension: a folder with a manifest and a Compose fragment that the dashboard, CLI, health checks, and Compose stack discover automatically. The ods CLI manages status, logs, restarts, mode switching between local, cloud, and hybrid, model swaps, extension enable and disable, and configuration presets.

Features

One-command install: GPU detection, model selection, credential generation, and service startup in a single step on Linux, macOS, and Windows
Bootstrap mode: a small model serves chat immediately while the selected model downloads, then swaps with no downtime
Tiered model catalog: per-tier default picks and context windows for NVIDIA, AMD Strix Halo, Apple Silicon, and Intel Arc envelopes
Extension system: manifest plus Compose fragment folders that the dashboard, CLI, and stack discover automatically
Mode switching: ods mode local, cloud, and hybrid route inference through local hardware, hosted APIs, or both
Voice and image: speech-to-text, text-to-speech, and node-based image generation run as part of the stack
Retrieval and search: a vector database, an embedding service, self-hosted web search, and a research engine
Privacy and ops: a scrubbing proxy for API calls, a token usage monitor, optional tracing, and a dashboard with GPU metrics and service health
Rollback: a model that fails to load reverts to the previously selected one

Tier / envelope	Default catalog pick	Context
NVIDIA 8 GB CPU fallback	Qwen3.5 2B (Q4KM)	8K
NVIDIA 8 GB discrete VRAM	Qwen3.5 9B (Q4KM)	32K
NVIDIA 24 GB discrete VRAM	Qwen3.5 27B (Q4KM)	32K
NVIDIA 48 GB discrete VRAM	DeepSeek R1 Distill Llama 70B (Q4KM)	32K
AMD Strix Halo 64 GB unified	Qwen3.6 35B-A3B (UD-Q4KM)	128K
Apple Silicon 8 GB unified	Phi-4 Mini (Q4KM)	128K
Apple Silicon 64+ GB unified	Qwen3.6 35B-A3B (UD-Q4KM)	128K
Intel Arc 16 GB discrete VRAM	Phi-4 14B (Q4KM)	16K

Claim ODS

Verify write access with GitHub