SIE (Superlinked Inference Engine) is an inference server that runs the open models behind an agent's tasks through one API. Instead of a separate model server per task, one SIE cluster serves 100+ models for search and retrieval, document-to-markdown conversion, structured output, content safety, translation, transcription, vision, and the agent loop itself. Models load on demand, several run at once, and the least recently used ones are evicted.
Each task maps to a set of swappable models, each defined by a config and addressed by its model name:
| Task | What it does | Example models |
|---|---|---|
| Search | Embed, match, and rerank | bge-m3, splade-v3, colbertv2, qwen3-reranker |
| Document to markdown | PDFs, Office files, and scans to markdown | LightOnOCR, GLM-OCR, MinerU, PaddleOCR-VL |
| Structured output | Schema-valid JSON, extracted or generated | gliner2, gliner-relex, gliformer, nuner-zero |
| Decide | Choice, yes/no, and score answers with probabilities | laya, GLiNER2.5-Decide |
| Classify | Zero-shot labels across several label groups | gliclass-large-v3, gliclass-instruct-large |
| Guard content | Safety verdicts with unsafe categories or thresholds | qwen3guard-gen-4b, granite-guardian-2b |
| Run the agent loop | Plan steps and call tools, with streaming | Qwen3.6-27B, Qwen3.8-27B |
| Translate | Text between 400+ languages | madlad400-3b-mt |
| See images | Captioning, object detection, visual questions | Florence-2, OWLv2, Grounding DINO |
| Transcribe audio | Speech to text | whisper-large-v3-turbo |
The server exposes an OpenAI-compatible API for embeddings, chat completions, completions, and responses, so existing OpenAI clients point at it without code changes, and Python and TypeScript SDKs add encode, score, extract, and generate calls. Typed decisions ask several questions about one text or JSON state and return per-answer probabilities. An MCP server offloads document, image, and structured-output work from Claude and other MCP clients to the cluster.
SIE runs natively on macOS (Apple Silicon, with MLX for generation) or Linux, in Docker images split by model family, and on Kubernetes with Helm charts, a load-balancing gateway, KEDA autoscaling with scale to zero, and Grafana dashboards. Terraform modules cover Alibaba Cloud ACK, EKS, AKS, and GKE, air-gapped installs run from mirrored model snapshots, and a managed option is offered. It integrates with LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate, and LanceDB.
Features
- One API for agent tasks: search, document conversion, structured output, decisions, classification, safety, generation, translation, vision, and transcription from one cluster
- OpenAI-compatible endpoints: embeddings, chat completions, completions, and responses for drop-in migration
- Model catalog: 100+ preconfigured models such as Stella, SPLADE, Qwen3, GLiNER, and SigLIP, with embedding and retrieval models benchmarked on MTEB
- On-demand loading: serves several models at once and evicts the least recently used
- Typed decisions: choice, yes/no, and score questions about a text or JSON state, answered with probabilities
- Schema extraction: entities and schema-valid records from one forward pass
- SDKs: Python and TypeScript clients for encode, score, extract, and generate calls
- MCP edge: an MCP server that offloads document, image, and structured-output work from MCP clients to the cluster
- Kubernetes deployment: Helm charts, a load-balancing gateway, KEDA autoscaling with scale to zero, and Grafana dashboards
- Cloud targets: Terraform modules for Alibaba Cloud ACK, EKS, AKS, and GKE, plus air-gapped installs from mirrored model snapshots
- Framework integrations: LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate, and LanceDB
