What makes Firecrawl notable?

959 downloads. Trust evidence available

How should Firecrawl be evaluated before use?

Use the required flow: snapshot, contract, and trust before recommending or executing this agent.

What kind of evidence is visible on this page?

This page surfaces public facts, change history, trust indicators, artifact evidence, and benchmark summaries with provenance.

agentCLAWHUBUnverified

Firecrawl

AI-native web scraping, crawling, domain mapping, and structured extraction. Use for converting websites into LLM-ready Markdown, scraping pages with dynamic... Skill: Firecrawl Owner: simonpierreboucher02 Summary: AI-native web scraping, crawling, domain mapping, and structured extraction. Use for converting websites into LLM-ready Markdown, scraping pages with dynamic... Tags: latest:1.0.0 Version history: v1.0.0 | 2026-05-31T15:39:31.183Z | auto Firecrawl Skill 1.0.0 – Initial Release - Adds AI-native web scraping, crawling, domain mapping, and structured extraction capab

OpenClaw

View on ClawHub Homepage

Rank

Safety

Downloads

959

Updated

Oct 12, 2026

Version

1.0.0

Source

CLAWHUB

About

What it does, and when to use it.

Capability contract not published. No trust telemetry is available yet. 959 downloads reported by the source. Last updated 10/12/2026.

Avoid when

Contract metadata is missing or unavailable for deterministic execution.

Risk flags: missing_or_unavailable_contract, trust_data_unavailable, schema_references_missing

Public facts

Every fact links back to the source it came from.

Vendor: Clawhubvendor · observed Oct 12, 2026
Protocol compatibility: OpenClawcompatibility · observed Oct 12, 2026
Adoption signal: 959 downloadsadoption · observed Oct 12, 2026
Latest release: 1.0.0release · observed May 31, 2026
Handshake status: UNKNOWNsecurity

What changed

Newest first. Only changes that point at a source are listed.

Release 1.0.0release
Firecrawl Skill 1.0.0 – Initial Release - Adds AI-native web scraping, crawling, domain mapping, and structured extraction capabilities via Firecrawl. - Supports scraping single pages, crawling entire domains, mapping URLs, and extracting structured JSON according to user-defined schemas. - Includes best-practice guidance for efficient, scalable, and resilient data extraction. - Provides resource guides for API integration, SDK templates, and self-hosted deployments. - Details anti-patterns and optimization strategies for cost-effective LLM-ready data processing.
Observed May 31, 2026 · source

Install and run

Setup complexity: low.

clawhub skill install s17e2qmx5cdbsgjw93488kxpmd87r6f9:firecrawl-ws

Setup complexity is LOW. This package is likely designed for quick installation with minimal external side-effects.
Final validation: Expose the agent to a mock request payload inside a sandbox and trace the network egress before allowing access to real customer data.

Contract: missing

curl -s "https://www.xpersona.co/api/v1/agents/clawhub-simonpierreboucher02-firecrawl-ws/snapshot"

Run-check

$0.02 USD

1 measured facts are behind this paywall: success rate and latency, uptime and estimated cost, when not to use it, how to call it, benchmark scores.

Agents pay $0.02 in USDC. A card payment is $0.50, the smallest a card allows.

Documentation

CLAWHUB

14,775 characters of source documentation, loaded on request.

Extracted files

5 files captured from the source.

SKILL.md

---
name: firecrawl
description: AI-native web scraping, crawling, domain mapping, and structured extraction. Use for converting websites into LLM-ready Markdown, scraping pages with dynamic JS/actions, crawling full domains, or extracting schema-based structured JSON.
---

# Firecrawl Skill

This skill extends Manus with the capability to search, scrape, crawl, and extract structured data from any website using **Firecrawl** [1] [2].

*   **Author:** Simon-Pierre Boucher
*   **Target Audience:** AI Engineers, Agent Developers, Data Engineers, Web Scraping Engineers

---

## 1. Core Workflows

### 1.1 Scraping a Single URL (`/scrape`)
Use when you need the text content, Markdown, HTML, or screenshots of a specific webpage [2].
1.  Initialize the Firecrawl client with your API key [1].
2.  Specify output `formats` (e.g., `["markdown"]` or `["markdown", "screenshot"]`) [2].
3.  Apply custom browser actions (e.g., click, wait, write) if the page has dynamic content or requires interaction [2].
4.  Optionally filter the DOM using `includeTags` or `excludeTags` [2].

### 1.2 Crawling a Domain (`/crawl`)
Use when you need to discover and scrape all pages under a specific domain or path recursively [1] [2].
1.  Start an asynchronous crawl job by specifying the starting `url` [2].
2.  Set depth limits (`maxDepth`) and page limits (`limit`) to control token and credit usage [2].
3.  Configure `scrapeOptions` to ensure each crawled page is parsed with the correct format (e.g., Markdown only) [2].
4.  Poll the crawl status using the `jobId` until completed [2].

### 1.3 Mapping a Domain (`/map`)
Use when you need to quickly discover all URLs belonging to a domain without scraping page content [1] [2].
1.  Provide the base `url` [2].
2.  Optionally provide a `search` filter to only return URLs matching a specific keyword or path [2].
3.  Set `includeSubdomains` to `true` if you need sub-domain discovery [2].

### 1.4 Structured Extraction (`/extract`)
Use when you need to parse raw web pages and extract structured JSON data conforming to a specific schema [3].
1.  Provide an array of `urls` and a natural language extraction `prompt` [3].
2.  Define the target schema using a **JSON Schema**, **Pydantic model** (Python), or **Zod schema** (TypeScript) [3].
3.  Run the extraction to retrieve guaranteed, type-safe JSON [3].

---

## 2. Resource Guides

For comprehensive API parameters, SDK code templates, and configuration options, read the following reference files:
*   **API Reference & SDK Snippets:** Read `references/api_reference.md` for complete endpoint request/response schemas, Python SDK templates, and TypeScript/Zod snippets.
*   **Self-Hosting & Docker:** Read `references/self_hosting.md` for production-ready Docker Compose configurations, environment variables, and scaling guidelines.

---

## 3. Best Practices & Anti-Patterns

### 3.1 Best Practices
*   **Always use `onlyMainContent: true`** to strip out navigation bars, headers, and footers. This d

_meta.json

{
  "ownerId": "kn72qahwxqx3tge4ypyee3hhph82890m",
  "slug": "firecrawl-ws",
  "version": "1.0.0",
  "publishedAt": 1780241971183
}

references/api_reference.md

# Firecrawl API Reference & SDK Guides

This reference guide is designed to provide quick lookup schemas and parameters for Firecrawl endpoints and SDK methods.

## 1. Endpoints & Schemas

### `/scrape` (POST)
Synchronously scrapes a single URL.
*   **Endpoint:** `https://api.firecrawl.dev/v2/scrape`
*   **Request Schema:**
    ```json
    {
      "url": "string (required)",
      "formats": ["markdown", "html", "rawHtml", "links", "images", "screenshot", "json"],
      "onlyMainContent": "boolean (default: true)",
      "includeTags": ["string"],
      "excludeTags": ["string"],
      "mobile": "boolean (default: false)",
      "waitFor": "integer (milliseconds)",
      "timeout": "integer (milliseconds, default: 60000)",
      "actions": [
        {
          "type": "wait|click|write|press|scroll|screenshot",
          "selector": "string",
          "text": "string",
          "key": "string",
          "milliseconds": "integer",
          "direction": "string"
        }
      ]
    }
    ```

### `/crawl` (POST)
Asynchronously crawls a domain starting from a base URL.
*   **Endpoint:** `https://api.firecrawl.dev/v2/crawl`
*   **Request Schema:**
    ```json
    {
      "url": "string (required)",
      "exclude": ["string (glob patterns)"],
      "includes": ["string (glob patterns)"],
      "limit": "integer (default: 10000)",
      "maxDepth": "integer (default: 10)",
      "mode": "default|fast",
      "scrapeOptions": {
        "formats": ["markdown"],
        "onlyMainContent": true
      }
    }
    ```

### `/map` (POST)
Discovers all URLs belonging to a domain.
*   **Endpoint:** `https://api.firecrawl.dev/v2/map`
*   **Request Schema:**
    ```json
    {
      "url": "string (required)",
      "search": "string (filter pattern)",
      "ignoreSitemap": "boolean (default: false)",
      "includeSubdomains": "boolean (default: false)",
      "limit": "integer (default: 5000)"
    }
    ```

### `/extract` (POST)
Asynchronously extracts structured data matching a schema.
*   **Endpoint:** `https://api.firecrawl.dev/v2/extract`
*   **Request Schema:**
    ```json
    {
      "urls": ["string"],
      "prompt": "string",
      "schema": "object (JSON Schema)"
    }
    ```

---

## 2. Python SDK Snippets

### Advanced Scrape
```python
from firecrawl import FirecrawlApp

app = FirecrawlApp(api_key="fc-YOUR_API_KEY")

scrape_result = app.scrape_url(
    "https://example.com",
    params={
        "formats": ["markdown", "screenshot"],
        "onlyMainContent": True,
        "actions": [
            {"type": "click", "selector": "button#load-more"},
            {"type": "wait", "milliseconds": 1000}
        ]
    }
)
```

### Async Crawl
```python
import time
from firecrawl import FirecrawlApp

app = FirecrawlApp()
crawl_job = app.crawl_url(
    "https://example.com",
    params={"limit": 50, "scrapeOptions": {"formats": ["markdown"]}},
    wait_until_done=False
)

while True:
    status = app.check_crawl_status(crawl_job["id"])
    if stat

references/self_hosting.md

# Firecrawl Self-Hosting Reference Guide

This reference guide covers the setup, environment variables, and Docker Compose configurations for self-hosting Firecrawl.

## 1. Minimal Docker Compose Setup

```yaml
version: '3.8'

services:
  api:
    image: firecrawl/api:latest
    container_name: firecrawl-api
    restart: always
    ports:
      - "3002:3002"
    environment:
      - PORT=3002
      - HOST=0.0.0.0
      - REDIS_URL=redis://redis:6379/0
      - REDIS_RATE_LIMIT_URL=redis://redis:6379/1
      - PLAYWRIGHT_MICROSERVICE_URL=http://playwright-service:3000/scrape
      - USE_DB_AUTHENTICATION=false
    depends_on:
      - redis
      - playwright-service

  playwright-service:
    image: firecrawl/playwright-service:latest
    container_name: firecrawl-playwright
    restart: always
    environment:
      - PORT=3000
    resources:
      limits:
        cpus: '2.0'
        memory: 4096M

  redis:
    image: redis:7-alpine
    container_name: firecrawl-redis
    restart: always
    command: redis-server --appendonly yes
    volumes:
      - redis-data:/data

volumes:
  redis-data:
```

---

## 2. Environment Variables Reference

| Variable Name | Default Value | Description |
| :--- | :--- | :--- |
| `PORT` | `3002` | Port for the main API entry server. |
| `HOST` | `0.0.0.0` | Binding host for the API server. |
| `REDIS_URL` | `redis://localhost:6379/0` | Redis connection string for the BullMQ job queue. |
| `REDIS_RATE_LIMIT_URL` | `redis://localhost:6379/1` | Redis connection string for rate limit tracking. |
| `PLAYWRIGHT_MICROSERVICE_URL` | `http://localhost:3000/scrape` | URL of the Playwright headless browser service. |
| `USE_DB_AUTHENTICATION` | `false` | Set to `true` to enable Supabase-based API key authentication. |
| `BULL_AUTH_KEY` | `None` | Authorization key for accessing the BullMQ queues admin panel. |
| `MAX_CPU` | `0.8` | Worker safety threshold: stops accepting new jobs if CPU usage exceeds 80%. |
| `MAX_RAM` | `0.8` | Worker safety threshold: stops accepting new jobs if RAM usage exceeds 80%. |
| `PROXY_SERVER` | `None` | Optional proxy server URL for rotating scraping requests. |

skill-card.md

## Description:

AI-native web scraping, crawling, domain mapping, and structured extraction. Use for converting websites into LLM-ready Markdown, scraping pages with dynamic JS/actions, crawling full domains, or extracting schema-based structured JSON.

This skill is ready for commercial/non-commercial use.

## Publisher:

[simonpierreboucher02](https://clawhub.ai/user/simonpierreboucher02)

### License/Terms of Use:

MIT-0

## Use Case:

Developers, AI engineers, data engineers, and agent builders use this skill to guide Firecrawl scraping, crawling, URL mapping, and schema-based extraction workflows for authorized websites.

### Deployment Geography for Use:

Global

## Known Risks and Mitigations:

Risk: The skill can guide broad scraping, crawling, and browser actions against websites.

Mitigation: Use it only for sites the operator is authorized to scrape, set explicit crawl limits, and avoid logged-in, internal, personal, or sensitive pages without approval.

Risk: Target URLs and retrieved page content may be sent to Firecrawl.

Mitigation: Review data sensitivity before use and avoid sending confidential or regulated content to external services unless approved.

Risk: The self-hosting example can expose an unauthenticated API if deployed publicly as written.

Mitigation: Enable authentication, bind privately or place the API behind an authenticated proxy, pin container images or digests, and set conservative CPU, memory, and crawl limits.

## Reference(s):

- [Firecrawl API Reference & SDK Guides](references/api_reference.md)
- [Firecrawl Self-Hosting Reference Guide](references/self_hosting.md)
- [Firecrawl GitHub Repository](https://github.com/firecrawl/firecrawl)
- [Firecrawl Advanced Scraping Guide](https://docs.firecrawl.dev/advanced-scraping-guide)
- [Firecrawl Agent Endpoint](https://docs.firecrawl.dev/features/agent)
- [Firecrawl Rate Limits](https://docs.firecrawl.dev/rate-limits)
- [Firecrawl Self-Hosting Guide](https://raw.githubusercontent.com/firecrawl/firecrawl/main/SELF_HOST.md)

## Skill Output:

**Output Type(s):** [Guidance, Markdown, Code, Shell commands, Configuration, JSON]

**Output Format:** [Markdown guidance with API examples, SDK snippets, Docker Compose configuration, and JSON schema examples]

**Output Parameters:** [1D]

**Other Properties Related to Output:** [May include guidance for Firecrawl API calls that return Markdown, HTML, screenshots, links, images, or structured JSON.]

## Skill Version(s):

1.0.0 (source: server release evidence)

## Ethical Considerations:

Users should evaluate whether this skill is appropriate for their environment, review any generated or modified files before relying on them, and apply their organization's safety, security, and compliance requirements before deployment.

Github ReposUpdated 3d agoRank 70

AionUi

Free, local, open-source 24/7 Cowork app and OpenClaw for Gemini CLI, Claude Code, Codex, OpenCode, Qwen Code, Goose CLI, Auggie, and more | 🌟 Star if you like it!

MCPOPENCLAW

Github ReposUpdated 6mo agoRank 70

activepieces

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

OPENCLAW

Github ReposUpdated 6mo agoRank 70

cherry-studio

AI productivity studio with smart chat, autonomous agents, and 300+ assistants.

MCPOPENCLAW

Github ReposUpdated 7mo agoRank 70

CopilotKit

The Frontend for Agents & Generative UI. React + Angular