ContextGem is a Python framework for pulling structured data out of a single document with a large language model. You describe in natural language what you want extracted, and the framework generates the prompts, builds the validation models, maps each result back to the text it came from, orchestrates multi-step pipelines, and tracks usage and cost.
There are two extraction primitives. Aspects capture document segments — clauses, sections, topics, themes — and can nest into sub-aspects. Concepts infer specific data points from a document or from an aspect: strings, JSON objects, numbers, dates, booleans, ratings, labels, entities, and assessments. Extracted items carry paragraph- or sentence-level references and generated justifications, and a set of aspects and concepts can be saved as a reusable extraction pipeline that runs consistently across many documents.
A Document holds text, images, or both. Models are configured through DocumentLLM, which is backed by LiteLLM, so cloud providers such as OpenAI, Anthropic, Google, Azure OpenAI, and xAI and local runtimes such as Ollama and LM Studio share one interface with fallback models and role-based routing. An LLM Group sends different aspects or concepts to different models — a small one for plain extraction, a reasoning model for inference. Documents, pipelines, and model configurations serialize, so results can be stored, transferred, and reloaded instead of re-running calls.
The design targets in-depth analysis of one document at a time using long context windows rather than retrieval over a corpus, which means cross-document querying and corpus-wide retrieval are out of scope; RAG frameworks such as LlamaIndex and Haystack are named for those cases. For reliable structured extraction the project recommends models at or above the capability of gpt-4o-mini; around 8B-parameter models may not follow the detailed extraction instructions. Documentation covers tuning for accuracy, speed, cost, and long documents.
Features
- Automated prompts: extraction prompts are generated from the natural-language description of an aspect or concept
- Automated data modelling: validation models are built for you rather than hand-written per extraction
- Reference mapping: each extracted item points back to the paragraphs or sentences it came from
- Built-in justifications: the model's reasoning for an item is captured alongside the value
- Nested extraction: aspects contain sub-aspects and concepts for hierarchical document analysis
- Declarative pipelines: a saved set of aspects and concepts reused across documents
- Text and vision: extraction runs over document text, images, or both
- Provider flexibility: cloud and local models through a LiteLLM-backed unified interface
- LLM groups: role-based routing of tasks to specialised models, with fallbacks
- Serialization: documents, pipelines, and model configurations save and load to avoid repeat calls
- Cost estimation: token usage and pricing tracked per run
