Xpersona
agentCLAWHUBUnverified

Doorstep Screen Control - 屏幕操控

屏幕控制技能 - 通过OpenClaw Node + pyautogui实现电脑屏幕识别和鼠标键盘操控。功能：(1) 截图获取屏幕画面，(2) OCR文字识别定位，(3) 图片匹配定位，(4) 鼠标移动/点击/拖拽，(5) 键盘输入/快捷键，(6) 基于视觉信息的自动化操作。Use when: (1) 需要远程操... Skill: Doorstep Screen Control - 屏幕操控 Owner: ncsimok Summary: 屏幕控制技能 - 通过OpenClaw Node + pyautogui实现电脑屏幕识别和鼠标键盘操控。功能：(1) 截图获取屏幕画面，(2) OCR文字识别定位，(3) 图片匹配定位，(4) 鼠标移动/点击/拖拽，(5) 键盘输入/快捷键，(6) 基于视觉信息的自动化操作。Use when: (1) 需要远程操... Tags: latest:1.0.0 Version history: v1.0.0 | 2026-04-29T02:37:31.778Z | user 初版：基于OpenClaw Node + pyautogui的屏幕识别与鼠标键盘操控技能，支持截图/OCR/图片匹配/鼠标点击/键盘输入 Archive index: Archive v1.0.0: 6 files, 7868 bytes

OpenClaw
View on ClawHubHomepage

Rank

62

Safety

84

Downloads

1.1k

Updated

Oct 11, 2026

Version

1.0.0

Source

CLAWHUB

About

What it does, and when to use it.

Capability contract not published. No trust telemetry is available yet. 1.1K downloads reported by the source. Last updated 10/11/2026.

Avoid when

  • Contract metadata is missing or unavailable for deterministic execution.

Risk flags: missing_or_unavailable_contract, trust_data_unavailable, schema_references_missing

Public facts

Every fact links back to the source it came from.

Vendor
Clawhubvendor · observed Oct 11, 2026
Protocol compatibility
OpenClawcompatibility · observed Oct 11, 2026
Adoption signal
1.1K downloadsadoption · observed Oct 11, 2026
Latest release
1.0.0release · observed Apr 29, 2026
Handshake status
UNKNOWNsecurity

Install and run

Setup complexity: low.

clawhub skill install s17c962yfsevawab1qf1f7z8v984g5tn:doorstep-screen-control
  1. Setup complexity is LOW. This package is likely designed for quick installation with minimal external side-effects.
  2. Final validation: Expose the agent to a mock request payload inside a sandbox and trace the network egress before allowing access to real customer data.

Contract: missing

curl -s "https://www.xpersona.co/api/v1/agents/clawhub-ncsimok-doorstep-screen-control/snapshot"

Documentation

CLAWHUB

6,668 characters of source documentation, loaded on request.

Extracted files

4 files captured from the source.

SKILL.md

---
name: screen-control
description: "屏幕控制技能 - 通过OpenClaw Node + pyautogui实现电脑屏幕识别和鼠标键盘操控。功能：(1) 截图获取屏幕画面，(2) OCR文字识别定位，(3) 图片匹配定位，(4) 鼠标移动/点击/拖拽，(5) 键盘输入/快捷键，(6) 基于视觉信息的自动化操作。Use when: (1) 需要远程操电脑屏幕，(2) 需要自动操作桌面应用（非浏览器），(3) 需要批量处理电脑上的文件或软件，(4) 需要通过视觉识别自动化操作。Triggers: '操作电脑', '屏幕控制', '自动点击', '截图识别', '桌面自动化', '远程操控', '操控鼠标', '屏幕识别'。"
---

# Screen Control — 屏幕操控技能

## 概述

通过 OpenClaw Node + pyautogui + OCR，实现对电脑屏幕的识别和鼠标键盘操控。

## 架构

```
用户指令 → Agent → screen_control.py (pyautogui + OCR) → 屏幕操作
                    ↕
              OpenClaw Node (截图/远程执行)
```

## 前置条件

1. **OpenClaw Node** — 需配对成功（见 references/setup-guide.md）
2. **Python依赖** — pyautogui, pillow, mss, pytesseract（如需OCR）
3. **Tesseract-OCR** — 如需文字识别功能（安装指引见 setup-guide.md）

## 工作流程

当用户要求操控电脑时：

### Step 1：确认屏幕状态
```bash
python {baseDir}/scripts/screen_control.py screenshot
python {baseDir}/scripts/screen_control.py size
```
获取当前屏幕画面和分辨率，AI识别画面内容。

### Step 2：分析画面 + 定位目标
AI分析截图，确定需要操作的坐标或内容。

- 如果目标基于**图片**：`locate <image_path>`
- 如果目标基于**文字**：`clicktext "确定"`
- 如果目标基于**坐标**：`click 500 300`

### Step 3：执行操作
```python
# 打开浏览器
press_key("win") + type_text("chrome") + press_key("enter")

# 打开网页
type_text("douyin.com\n")  # \n = enter

# 拖动
mouse_drag(100, 100, 500, 500)

# 复制粘贴
hotkey("ctrl", "c")
hotkey("ctrl", "v")
```

### Step 4：验证结果
再次截图确认操作结果。

## 脚本命令速查

| 命令 | 参数 | 说明 |
|------|------|------|
| screenshot | 无 | 截图保存到文件 |
| size | 无 | 获取屏幕分辨率 |
| move | x y | 移动鼠标 |
| click | x y [button] | 点击（按钮: left/right/middle） |
| doubleclick | x y | 双击 |
| text | "文字" | 键盘输入文字 |
| key | key_name | 按键（enter/esc/tab等） |
| locate | image_path | 查找图片位置 |
| findtext | "文字" | 查找文字位置 |
| clicktext | "文字" | 查找并点击文字 |
| scroll | clicks | 滚动（正=上，负=下） |
| color | x y | 获取像素颜色 |

## 常用场景

### 1. 自动登录平台后台
```
截图 → 定位登录框坐标 → 输入账号密码 → 点击登录按钮
```

### 2. 批量操作抖音/小红书后台
```
截图 → 找到发布按钮 → 点开 → 上传视频/图片 → 填写标题 → 发布
```

### 3. 打开特定软件/文件
```
按Win键 → 搜索程序名 → 回车打开
```

### 4. 文件整理/批量重命名
```
打开文件夹 → 选中文件 → 执行操作
```

## 安全注意

1. 操作前先截图给用户确认
2. 任何破坏性操作（删除文件、修改配置）需用户确认
3. 使用 pyautogui.FAILSAFE=True（左上角急停）
4. 不要在用户未授权的情况下操作浏览器密码/支付页面

_meta.json

{
  "ownerId": "kn7ehnh2e0mad8tnvhmjgj3kyd82q5xj",
  "slug": "doorstep-screen-control",
  "version": "1.0.0",
  "publishedAt": 1777430251778
}

references/setup-guide.md

# Screen Control Node屏幕操控
# Node配对配置参考

## 配对步骤

### 方式1：命令行配对（推荐）
```bash
# 终端1 - 启动Gateway（如未运行）
openclaw gateway start

# 终端2 - 启动Node客户端
openclaw node run --host 127.0.0.1 --port 18789

# 终端3 - 查看并批准配对请求
openclaw nodes pending
openclaw nodes approve <requestId>
```

### 方式2：以管理员身份安装为服务
```powershell
# 以管理员身份运行PowerShell
openclaw node install --host 127.0.0.1 --port 18789
```

## 配对后的操作

配对成功后，Agent可以通过以下方式操控电脑：

1. **截图** — 获取屏幕实时画面
2. **pyautogui** — 移动鼠标、点击、键盘输入
3. **exec** — 运行本地脚本（文件操作、打开应用等）

## 屏幕控制脚本用法

```bash
# 截图
python scripts/screen_control.py screenshot

# 获取屏幕尺寸
python scripts/screen_control.py size

# 移动鼠标到坐标(500, 300)
python scripts/screen_control.py move 500 300

# 点击(500, 300)位置
python scripts/screen_control.py click 500 300 left

# 双击
python scripts/screen_control.py doubleclick 500 300

# 输入文字
python scripts/screen_control.py text "你好世界"

# 按键
python scripts/screen_control.py key enter
python scripts/screen_control.py key escape

# 组合键（使用pyautogui.hotkey）
python -c "import pyautogui; pyautogui.hotkey('ctrl', 'c')"

# 查找图片并点击
python scripts/screen_control.py locate button.png

# 查找文字位置
python scripts/screen_control.py findtext "确认"

# 查找并点击文字
python scripts/screen_control.py clicktext "提交"

# 滚动
python scripts/screen_control.py scroll -3

# 获取像素颜色
python scripts/screen_control.py color 100 200
```

## 注意事项

1. **Failsafe** — 将鼠标移到屏幕左上角可紧急停止
2. **权限** — 脚本需要管理员权限才能操作某些窗口
3. **Tesseract** — OCR功能需要安装Tesseract-OCR
   - 下载: https://github.com/UB-Mannheim/tesseract/wiki
   - 设置环境变量: TESSERACT_PATH
4. **分辨率** — 使用前先确认屏幕分辨率

skill-card.md

## Description:

Enables an agent to use OpenClaw Node and local Python tools to capture screenshots, locate text or images, and control mouse and keyboard actions on a desktop.

This skill is ready for commercial/non-commercial use.

## Publisher:

[ncsimok](https://clawhub.ai/user/ncsimok)

### License/Terms of Use:

MIT-0

## Use Case:

Developers and operators use this skill when an agent needs controlled desktop automation for non-browser applications, file workflows, or visual UI tasks that require screenshots, OCR, image matching, mouse actions, and keyboard input.

### Deployment Geography for Use:

Global

## Known Risks and Mitigations:

Risk: The skill can let an agent view the screen and control mouse and keyboard input.

Mitigation: Use it only on systems where this access is intentional, review screenshots before actions, and require user confirmation for destructive or sensitive workflows.

Risk: Persistent node installation and local script execution can broaden access on sensitive systems.

Mitigation: Prefer the foreground loopback node run mode, avoid administrator service installation unless operationally required, and confirm how to stop the node, revoke pairings, and restrict local exec.

## Reference(s):

- [Screen Control Node setup guide](references/setup-guide.md)
- [Tesseract OCR installation guide](https://github.com/UB-Mannheim/tesseract/wiki)
- [ClawHub skill page](https://clawhub.ai/ncsimok/skills/doorstep-screen-control)

## Skill Output:

**Output Type(s):** [Text, Markdown, Code, Shell commands, Configuration]

**Output Format:** [Markdown guidance with shell commands and JSON command outputs]

**Output Parameters:** [1D]

**Other Properties Related to Output:** [May produce screenshots or local screenshot file paths when the screen control script is run.]

## Skill Version(s):

1.0.0 (source: server release metadata)

## Ethical Considerations:

Users should evaluate whether this skill is appropriate for their environment, review any generated or modified files before relying on them, and apply their organization's safety, security, and compliance requirements before deployment.

Related agents

Same protocol, similar job.

Github ReposUpdated 2d agoRank 70

AionUi

Free, local, open-source 24/7 Cowork app and OpenClaw for Gemini CLI, Claude Code, Codex, OpenCode, Qwen Code, Goose CLI, Auggie, and more | 🌟 Star if you like it!

MCPOPENCLAW
Github ReposUpdated 6mo agoRank 70

activepieces

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

OPENCLAW
Github ReposUpdated 6mo agoRank 70

cherry-studio

AI productivity studio with smart chat, autonomous agents, and 300+ assistants.

MCPOPENCLAW
Github ReposUpdated 7mo agoRank 70

CopilotKit

The Frontend for Agents & Generative UI. React + Angular

OPENCLAW

Machine-readable data

The same record, as JSON, for agents and crawlers.

{
  "facts": [
    {
      "factKey": "vendor",
      "category": "vendor",
      "label": "Vendor",
      "value": "Clawhub",
      "href": "https://clawhub.ai/ncsimok/skills/doorstep-screen-control",
      "sourceUrl": "https://clawhub.ai/ncsimok/skills/doorstep-screen-control",
      "sourceType": "profile",
      "confidence": "medium",
      "observedAt": "2026-10-11T11:49:44.487Z",
      "isPublic": true
    },
    {
      "factKey": "protocols",
      "category": "compatibility",
      "label": "Protocol compatibility",
      "value": "OpenClaw",
      "href": "https://www.xpersona.co/api/v1/agents/clawhub-ncsimok-doorstep-screen-control/contract",
      "sourceUrl": "https://www.xpersona.co/api/v1/agents/clawhub-ncsimok-doorstep-screen-control/contract",
      "sourceType": "contract",
      "confidence": "medium",
      "observedAt": "2026-10-11T11:49:44.487Z",
      "isPublic": true
    },
    {
      "factKey": "traction",
      "category": "adoption",
      "label": "Adoption signal",
      "value": "1.1K downloads",
      "href": "https://clawhub.ai/ncsimok/doorstep-screen-control",
      "sourceUrl": "https://clawhub.ai/ncsimok/doorstep-screen-control",
      "sourceType": "profile",
      "confidence": "medium",
      "observedAt": "2026-10-11T11:49:44.487Z",
      "isPublic": true
    },
    {
      "factKey": "latest_release",
      "category": "release",
      "label": "Latest release",
      "value": "1.0.0",
      "href": "https://clawhub.ai/ncsimok/doorstep-screen-control",
      "sourceUrl": "https://clawhub.ai/ncsimok/doorstep-screen-control",
      "sourceType": "release",
      "confidence": "medium",
      "observedAt": "2026-04-29T02:37:31.778Z",
      "isPublic": true
    },
    {
      "factKey": "handshake_status",
      "category": "security",
      "label": "Handshake status",
      "value": "UNKNOWN",
      "href": "https://www.xpersona.co/api/v1/agents/clawhub-ncsimok-doorstep-screen-control/trust",
      "sourceUrl": "https://www.xpersona.co/api/v1/agents/clawhub-ncsimok-doorstep-screen-control/trust",
      "sourceType": "trust",
      "confidence": "medium",
      "observedAt": null,
      "isPublic": true
    }
  ],
  "events": [
    {
      "eventType": "release",
      "title": "Release 1.0.0",
      "description": "初版：基于OpenClaw Node + pyautogui的屏幕识别与鼠标键盘操控技能，支持截图/OCR/图片匹配/鼠标点击/键盘输入",
      "href": "https://clawhub.ai/ncsimok/doorstep-screen-control",
      "sourceUrl": "https://clawhub.ai/ncsimok/doorstep-screen-control",
      "sourceType": "release",
      "confidence": "medium",
      "observedAt": "2026-04-29T02:37:31.778Z",
      "isPublic": true
    }
  ]
}

Record generated Oct 11, 2026.

Sponsored

Ads related to Doorstep Screen Control - 屏幕操控 and adjacent AI workflows.