Firecrawl is a web-data API that turns arbitrary web pages into clean, LLM-ready output such as Markdown, structured JSON, or screenshots. It is aimed at developers building AI applications and agents that need reliable web content without managing the mechanics of scraping themselves, handling rotating proxies, orchestration, rate limits, and JavaScript-rendered pages behind a single API. It also parses web-hosted documents like PDFs and DOCX. Firecrawl is open source under AGPL-3.0 and also offered as a hosted service, with a self-host option.
- Scrape: convert a single URL into markdown, HTML, structured JSON, or a screenshot
- Search: query the web and return full page content for the results
- Crawl: fetch content from all pages of a website via one asynchronous job
- Map: quickly discover all URLs on a site, optionally ranked by a search term
- Batch Scrape: scrape many URLs asynchronously
- Interact: after scraping a page, drive it with AI prompts or code (click, scroll, type, wait) before extracting
- Agent: describe the data you want in natural language and let it search, navigate, and retrieve without supplying URLs, optionally against a schema for structured output
For extraction the Agent endpoint offers two models (a cheaper default and a pro model for complex, multi-site research). Official SDKs are available for Python, Node.js, Java, Elixir, and Rust (plus a community Go SDK), and the SDKs handle polling for asynchronous jobs automatically. Firecrawl can also be wired into agents through a CLI-installable skill and an MCP server, letting MCP-compatible clients such as coding agents access live web data. The maintainers note that respecting site policies and robots.txt when scraping is the users responsibility; Firecrawl respects robots.txt by default.
