Product Roadmap
All
AllIn ProgressPlannedShipped
In Progress
Q1 2026
Windows support
Bring the CLI and desktop app to Windows with full installer support.
Core Team
In Progress
Q1 2026
Crawl diffing
Compare two .crawl files to detect added, removed, and changed pages between crawl runs.
Core Team
Planned
Q2 2026
Custom crawl rules
Define include/exclude URL patterns, respect robots.txt, and configure custom request headers.
Core Team
Planned
Q3 2026
API server mode
Run crawler.sh as a local HTTP server with REST endpoints for triggering crawls and retrieving results programmatically.
Core Team
Shipped
April 2026
JavaScript rendering
Built-in JS rendering engine with auto-detection. Samples pages and enables rendering when client-side frameworks are detected. Three modes: Auto, Always, and Never.
Core Team
Shipped
February 2026
SEO analysis CLI
The crawler seo subcommand with 23 check categories, CSV and TXT export, and canonical-aware duplicate detection.
Core Team
Shipped
Feb 2026
Content extraction
Content extraction with Markdown output, word count, byline, and excerpt for every crawled page.
Core Team
Shipped
Feb 2026
Desktop app v0.1
Desktop app with 8 dashboard cards, real-time crawl feed, SEO analysis panel, status code charts, and multi-format export.
Core Team
Shipped
Feb 2026
CLI v0.1
Initial release with crawl, info, and export subcommands. NDJSON, JSON, and Sitemap XML output formats. Configurable concurrency, depth, and delay.
Core Team
Community engagement
We build in public. Want to request something or upvote a feature? Join the conversation on: