Easy Dataset is an application for building datasets used to fine-tune, evaluate, and ground large language models. It parses domain documents, splits them into chunks, generates questions and answers with an LLM, organizes the results under a domain label tree, and exports them in training formats. Each project runs the full pipeline from document processing to export and evaluation.
Documents in PDF, Markdown, DOCX, TXT, EPUB, and other formats are chunked with selectable strategies (Markdown structure, recursive separators, fixed length, code-aware) and can be segmented visually. From chunks the tool generates questions, answers, and chain-of-thought text, and it can distill label trees and questions directly from a topic without documents. An evaluation module produces test sets, runs judge-model scoring, and supports double-blind human comparison of two models.
The tool works with any OpenAI-format API and providers such as OpenAI, MiniMax, Ollama, Zhipu AI, Alibaba Bailian, and OpenRouter, while vision models such as Gemini and Claude handle PDF parsing and image QA. It runs as a web app, in Docker, or as desktop clients for Windows, macOS, and Linux, with Chinese, English, Turkish, and Portuguese interfaces.
Features
- Document processing: PDF, Markdown, DOCX, TXT, and EPUB inputs with several chunking algorithms
- Question and answer generation: templated batch question generation, LLM answers, and chain-of-thought output
- Domain label tree: auto-built two-level label hierarchy with auto-tagging
- Dataset types: single-turn QA, multi-turn dialogue, image QA, and distilled datasets
- Evaluation: true/false, single-choice, multiple-choice, short-answer, and open-ended test sets with judge-model scoring and a human blind arena
- Custom prompts: project-level overrides for question, answer, and cleaning prompt templates
- Task center and monitoring: background batch jobs with interruption, plus token and API-call dashboards
- Export and integration: Alpaca, ShareGPT, and Multilingual-Thinking formats, balanced export per tag, LLaMA Factory config generation, and Hugging Face Hub upload
- Model playground: side-by-side comparison of up to three models
