The Superlinked Inference Engine (SIE) serves the open models behind agent tasks from one self-hosted cluster: search and retrieval, document-to-markdown conversion, structured output, content safety, and the agent loop itself. Instead of a separate model server per task, one system serves over 100 models, loading each on demand and evicting least-recently-used models to free capacity.
The API is OpenAI-compatible - /v1/embeddings, /v1/chat/completions, /v1/completions, and /v1/responses - so existing clients can point at the cluster by changing a base URL. Python and TypeScript SDKs wrap the same endpoints with typed calls for encoding, scoring, and entity extraction, and every model in the catalogue is a config file addressed by its Hugging Face ID.
Tasks map to sets of swappable models: bge-m3, splade-v3, colbertv2, and qwen3-reranker for embedding, matching, and reranking; lightonocr, glm-ocr, mineru, paddleocr-vl, and docling for turning PDFs, Office files, and scans into markdown; gliner2, nuner-zero, and qwen3.6-27b for schema-valid JSON; granite-guardian-2b for content-safety verdicts; and qwen3.6-27b for planning steps and calling tools. Embedding and retrieval models are benchmarked on MTEB.
Deployment covers a laptop and a production cluster with the same code. Docker images are bundle-specific so dependency-incompatible model families stay isolated, with separate CPU, CUDA, Transformers 5 OCR, and SGLang generation images; on Apple Silicon a native install serves models locally and MLX handles generation. For clusters, SIE ships a load-balancing gateway, Helm charts, KEDA autoscaling with scale to zero, and Grafana dashboards, with separately maintained Terraform modules for Alibaba Cloud ACK, EKS, AKS, and GKE.
Anonymous usage data - version, operating system, architecture, and GPU type - is collected by default and can be disabled with SIETELEMETRYDISABLED=1 or DONOTTRACK=1; no IP addresses, hostnames, or request data are collected.
Features
- One cluster, many models: over 100 models served simultaneously with on-demand loading and LRU eviction
- OpenAI-compatible API: embeddings, chat completions, completions, and responses endpoints
- Task coverage: search and rerank, document-to-markdown, structured output, content safety, and the agent loop
- Pre-configured catalogue: Stella, SPLADE, Qwen3, GLiNER, SigLIP, and more, with retrieval models benchmarked on MTEB
- Python and TypeScript SDKs: sie-sdk and @superlinked/sie-sdk for encode, score, extract, and generate calls
- Kubernetes surface: load-balancing gateway, Helm charts, KEDA autoscaling to zero, and Grafana dashboards
- Cloud modules: separately maintained Terraform for Alibaba Cloud ACK, EKS, AKS, and GKE
- Framework integrations: LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate, and LanceDB
- MCP edge: an MCP package offloads document work from MCP clients to the cluster
- Hardware options: CPU, NVIDIA CUDA, and Apple Silicon images, with MLX for generation on macOS
