Crawler Desktop Overview

What is Crawler Desktop?

Crawler Desktop is a macOS desktop application for crawling websites, analyzing SEO issues, visualizing HTTP status codes, browsing extracted content, and exporting results - all with a real-time interactive interface.

Install

Download the macOS DMG (universal binary - works on both Apple Silicon and Intel):

Download Crawler Desktop

Open the .dmg file and drag Crawler to your Applications folder.

Uninstall

Drag Crawler from your Applications folder to the Trash.

Standard Workflow

  1. Enter a URL

Type the target URL in the top input bar and click Start Crawl. The https:// prefix is added automatically.

  1. Configure settings

Use the Settings card to adjust Max Pages, Max Depth, Concurrency, Delay, and Extract Content before starting.

  1. Monitor the crawl

Watch the Live Feed card for real-time URL discovery with color-coded status badges. The Page Status card shows a donut chart of HTTP status codes.

  1. Analyze SEO

The SEO Issues card aggregates problems across all crawled pages. Expand it to browse all 23 issue categories with per-URL details.

  1. Export results

Use the Downloads card to export as JSON Archive or Sitemap XML. Use the SEO Issues expanded view for CSV or TXT export.

Tutorial: Full Desktop Walkthrough

  1. Open the desktop app

Launch Crawler from your Applications folder.

  1. Configure and start a crawl

Enter a URL in the top input bar. Adjust settings in the Settings card (Max Pages, Concurrency, etc.), then click Start Crawl.

  1. Monitor progress

    • Live Feed - watch URLs appear in real time with color-coded HTTP status badges (green for 2xx, yellow for 3xx, red for 4xx/5xx)
    • Page Status - see a donut chart of status code distribution

  2. Review SEO issues

Click the SEO Issues card to expand it. Browse the 23 issue categories in an accordion view. Each category lists affected URLs with copy and open-in-browser actions.

  1. Browse extracted content

Click the Content card to open the split-pane content viewer. The left panel lists pages with word counts; the right panel renders the extracted Markdown.

  1. Export results

Use the Downloads card to export:

  • JSON Archive - complete crawl data with metadata
  • Sitemap XML - standard sitemap for search engines

Use the SEO Issues expanded view to export issues as CSV or TXT.

  1. Reset and start over

Click the Reset button to clear all results and reconfigure for a new crawl.