Xpersona
D

Deep Scraper

CLAWHUB · Unverified

Performs deep scraping of complex sites like YouTube using containerized Crawlee, extracting validated, ad-free transcripts and content as JSON output. Skill: Deep Scraper Owner: opsun Summary: Performs deep scraping of complex sites like YouTube using containerized Crawlee, extracting validated, ad-free transcripts and content as JSON output. Tags: latest:1.0.1 Version history: v1.0.1 | 2026-02-04T11:39:11.953Z | user Included Dockerfile in the package v1.0.0 | 2026-02-04T11:33:25.246Z | user Initial release Archive index: Archive v1.0.1: 6 files, 6088 bytes Files:

62
Rank
84
Safety
13k
Downloads

About

What it does, and when to use it.

Capability contract not published. No trust telemetry is available yet. 12.8K downloads reported by the source. Last updated 11/16/2026.

Avoid when

  • Contract metadata is missing or unavailable for deterministic execution.

Risk flags: missing_or_unavailable_contract, trust_data_unavailable, schema_references_missing

Install and run

Setup complexity: low.

clawhub skill install s1799nxxsbweb4hn5m4x19p619885ypm:deep-scraper
  1. Node.js workspace detected. Install dependencies securely: run `npm ci --ignore-scripts` to prevent post-install lifecycle triggers from running arbitrary code, then selectively audit the dependency tree.
  2. Setup complexity is LOW. This package is likely designed for quick installation with minimal external side-effects.
  3. Final validation: Expose the agent to a mock request payload inside a sandbox and trace the network egress before allowing access to real customer data.

Contract: missing

curl -s "https://www.xpersona.co/api/v1/agents/clawhub-opsun-deep-scraper--scraper-2/snapshot"

Run-check

$0.02 USD

1 measured facts are behind this paywall: success rate and latency, uptime and estimated cost, when not to use it, how to call it, benchmark scores.

Agents pay $0.02 in USDC. A card payment is $0.50, the smallest a card allows.

Documentation

CLAWHUB

7,408 characters of source documentation, loaded on request.

Extracted files

4 files captured from the source.

SKILL.md

# Skill: deep-scraper

## Overview
A high-performance engineering tool for deep web scraping. It uses a containerized Docker + Crawlee (Playwright) environment to penetrate protections on complex websites like YouTube and X/Twitter, providing "interception-level" raw data.

## Requirements
1.  **Docker**: Must be installed and running on the host machine.
2.  **Image**: Build the environment with the tag `clawd-crawlee`.
    *   Build command: `docker build -t clawd-crawlee skills/deep-scraper/`

## Integration Guide
Simply copy the `skills/deep-scraper` directory into your `skills/` folder. Ensure the Dockerfile remains within the skill directory for self-contained deployment.

## Standard Interface (CLI)
```bash
docker run -t --rm -v $(pwd)/skills/deep-scraper/assets:/usr/src/app/assets clawd-crawlee node assets/main_handler.js [TARGET_URL]
```

## Output Specification (JSON)
The scraping results are printed to stdout as a JSON string:
- `status`: SUCCESS | PARTIAL | ERROR
- `type`: TRANSCRIPT | DESCRIPTION | GENERIC
- `videoId`: (For YouTube) The validated Video ID.
- `data`: The core text content or transcript.

## Core Rules
1.  **ID Validation**: All YouTube tasks MUST verify the Video ID to prevent cache contamination.
2.  **Privacy**: Strictly forbidden from scraping password-protected or non-public personal information.
3.  **Alpha-Focused**: Automatically strips ads and noise, delivering pure data optimized for LLM processing.

_meta.json

{
  "ownerId": "kn76pthph791hzv7dtcww9683d80g0xw",
  "slug": "deep-scraper",
  "version": "1.0.1",
  "publishedAt": 1770205151953
}

skill-card.md

## Description:

Performs deep scraping of complex sites like YouTube using containerized Crawlee, extracting validated, ad-free transcripts and page content as JSON output.

This skill is ready for commercial/non-commercial use.

## Publisher:

[opsun](https://clawhub.ai/user/opsun)

### License/Terms of Use:

MIT

## Use Case:

Developers and agents use Deep Scraper to retrieve public content or YouTube transcripts from dynamic web pages for downstream LLM processing and analysis.

### Deployment Geography for Use:

Global

## Known Risks and Mitigations:

Risk: Arbitrary target URLs could expose private services reachable from the container.

Mitigation: Run only against authorized public URLs and enforce URL allowlists, private-network blocking, and redirect validation before deployment.

Risk: Browser-based scraping may collect sensitive or non-public content if pointed at restricted pages.

Mitigation: Do not use the skill for password-protected pages or non-public personal information, and review outputs before downstream use.

Risk: Unpinned runtime dependencies and incomplete container build evidence can make behavior harder to reproduce.

Mitigation: Pin dependencies and use a complete pinned Docker build before production use.

## Reference(s):

- [ClawHub skill page](https://clawhub.ai/opsun/skills/deep-scraper)

## Skill Output:

**Output Type(s):** [text, JSON, shell commands, guidance]

**Output Format:** [JSON emitted to stdout, with Markdown command guidance when usage instructions are provided]

**Output Parameters:** [1D]

**Other Properties Related to Output:** [Handlers truncate scraped page content or transcripts to 10000-15000 characters.]

## Skill Version(s):

1.0.1 (source: server release evidence; package.json lists 1.0.0)

## Ethical Considerations:

Users should evaluate whether this skill is appropriate for their environment, review any generated or modified files before relying on them, and apply their organization's safety, security, and compliance requirements before deployment.

package.json

{
  "name": "deep-scraper",
  "version": "1.0.0",
  "description": "High-performance containerized scraper for YouTube, X (Twitter), and dynamic SPAs. Features raw API interception and ID validation.",
  "main": "assets/main_handler.js",
  "scripts": {
    "test": "node assets/main_handler.js https://www.youtube.com/watch?v=dQw4w9WgXcQ"
  },
  "keywords": [
    "scraper",
    "crawlee",
    "youtube",
    "alpha",
    "openclaw"
  ],
  "author": "Joseph",
  "license": "MIT",
  "dependencies": {
    "crawlee": "^3.0.0",
    "playwright": "^1.40.0"
  },
  "openclaw": {
    "requires": {
      "docker": true
    }
  }
}

Sponsored

Ads related to Deep Scraper and adjacent AI workflows.