What makes Desktop Control notable?

54.9K downloads. Trust evidence available

How should Desktop Control be evaluated before use?

Use the required flow: snapshot, contract, and trust before recommending or executing this agent.

What kind of evidence is visible on this page?

This page surfaces public facts, change history, trust indicators, artifact evidence, and benchmark summaries with provenance.

agentCLAWHUBUnverified

Desktop Control

Advanced desktop automation with mouse, keyboard, and screen control Skill: Desktop Control Owner: matagul Summary: Advanced desktop automation with mouse, keyboard, and screen control Tags: latest:1.0.0 Version history: v1.0.0 | 2026-02-05T01:33:20.863Z | auto Version 1.0.0 - Initial release of the Desktop Control skill for OpenClaw. - Provides advanced automation: mouse movement/clicks, keyboard input, hotkeys, and typing speed control. - Supports screen capture, region-based screen

OpenClaw

View on ClawHub

Rank

Safety

Downloads

55k

Updated

May 21, 2026

Version

1.0.0

Source

CLAWHUB

About

What it does, and when to use it.

Capability contract not published. No trust telemetry is available yet. 54.9K downloads reported by the source. Last updated 5/21/2026.

Avoid when

Contract metadata is missing or unavailable for deterministic execution.

Risk flags: missing_or_unavailable_contract, trust_data_unavailable, schema_references_missing

Public facts

Every fact links back to the source it came from.

Adoption signal: 54.9K downloadsadoption · observed May 21, 2026
Vendor: Clawhubvendor · observed May 18, 2026
Protocol compatibility: OpenClawcompatibility · observed May 18, 2026
Adoption signal: 54.4K downloadsadoption · observed May 18, 2026
Latest release: 1.0.0release · observed Feb 5, 2026
Handshake status: UNKNOWNsecurity

Install and run

Setup complexity: low.

clawhub skill install publishers:matagul:desktop-control

Setup complexity is LOW. This package is likely designed for quick installation with minimal external side-effects.
Final validation: Expose the agent to a mock request payload inside a sandbox and trace the network egress before allowing access to real customer data.

Contract: missing

curl -s "https://www.xpersona.co/api/v1/agents/clawhub-matagul-desktop-control-2/snapshot"

Documentation

CLAWHUB

31,634 characters of source documentation, loaded on request.

Extracted files

4 files captured from the source.

SKILL.md

---
description: Advanced desktop automation with mouse, keyboard, and screen control
---

# Desktop Control Skill

**The most advanced desktop automation skill for OpenClaw.** Provides pixel-perfect mouse control, lightning-fast keyboard input, screen capture, window management, and clipboard operations.

## 🎯 Features

### Mouse Control
- ✅ **Absolute positioning** - Move to exact coordinates
- ✅ **Relative movement** - Move from current position
- ✅ **Smooth movement** - Natural, human-like mouse paths
- ✅ **Click types** - Left, right, middle, double, triple clicks
- ✅ **Drag & drop** - Drag from point A to point B
- ✅ **Scroll** - Vertical and horizontal scrolling
- ✅ **Position tracking** - Get current mouse coordinates

### Keyboard Control
- ✅ **Text typing** - Fast, accurate text input
- ✅ **Hotkeys** - Execute keyboard shortcuts (Ctrl+C, Win+R, etc.)
- ✅ **Special keys** - Enter, Tab, Escape, Arrow keys, F-keys
- ✅ **Key combinations** - Multi-key press combinations
- ✅ **Hold & release** - Manual key state control
- ✅ **Typing speed** - Configurable WPM (instant to human-like)

### Screen Operations
- ✅ **Screenshot** - Capture entire screen or regions
- ✅ **Image recognition** - Find elements on screen (via OpenCV)
- ✅ **Color detection** - Get pixel colors at coordinates
- ✅ **Multi-monitor** - Support for multiple displays

### Window Management
- ✅ **Window list** - Get all open windows
- ✅ **Activate window** - Bring window to front
- ✅ **Window info** - Get position, size, title
- ✅ **Minimize/Maximize** - Control window states

### Safety Features
- ✅ **Failsafe** - Move mouse to corner to abort
- ✅ **Pause control** - Emergency stop mechanism
- ✅ **Approval mode** - Require confirmation for actions
- ✅ **Bounds checking** - Prevent out-of-screen operations
- ✅ **Logging** - Track all automation actions

---

## 🚀 Quick Start

### Installation

First, install required dependencies:

```bash
pip install pyautogui pillow opencv-python pygetwindow
```

### Basic Usage

```python
from skills.desktop_control import DesktopController

# Initialize controller
dc = DesktopController(failsafe=True)

# Mouse operations
dc.move_mouse(500, 300)  # Move to coordinates
dc.click()  # Left click at current position
dc.click(100, 200, button="right")  # Right click at position

# Keyboard operations
dc.type_text("Hello from OpenClaw!")
dc.hotkey("ctrl", "c")  # Copy
dc.press("enter")

# Screen operations
screenshot = dc.screenshot()
position = dc.get_mouse_position()
```

---

## 📋 Complete API Reference

### Mouse Functions

#### `move_mouse(x, y, duration=0, smooth=True)`
Move mouse to absolute screen coordinates.

**Parameters:**
- `x` (int): X coordinate (pixels from left)
- `y` (int): Y coordinate (pixels from top)
- `duration` (float): Movement time in seconds (0 = instant, 0.5 = smooth)
- `smooth` (bool): Use bezier curve for natural movement

_meta.json

{
  "ownerId": "kn7ag28ra4hhta8bx2k2j1kpv180kqbk",
  "slug": "desktop-control",
  "version": "1.0.0",
  "publishedAt": 1770255200863
}

AI_AGENT_GUIDE.md

# AI Desktop Agent - Cognitive Automation Guide

## 🤖 What Is This?

The **AI Desktop Agent** is an intelligent layer on top of the basic desktop control that **understands** what you want and figures out how to do it autonomously.

Unlike basic automation that requires exact instructions, the AI Agent:
- **Understands natural language** ("Draw a cat in Paint")
- **Plans the steps** automatically
- **Executes autonomously** 
- **Adapts** based on what it sees

---

## 🎯 What Can It Do?

### ✅ Autonomous Drawing
```python
from skills.desktop_control.ai_agent import AIDesktopAgent

agent = AIDesktopAgent()

# Just describe what you want!
agent.execute_task("Draw a circle in Paint")
agent.execute_task("Draw a star in MS Paint")
agent.execute_task("Draw a house with a sun")
```

**What it does:**
1. Opens MS Paint
2. Selects pencil tool
3. Figures out how to draw the requested shape
4. Draws it autonomously
5. Takes a screenshot of the result

### ✅ Autonomous Text Entry
```python
# It figures out where to type
agent.execute_task("Type 'Hello World' in Notepad")
agent.execute_task("Write an email saying thank you")
```

**What it does:**
1. Opens Notepad (or finds active text editor)
2. Types the text naturally
3. Formats if needed

### ✅ Autonomous Application Control
```python
# It knows how to open apps
agent.execute_task("Open Calculator")
agent.execute_task("Launch Microsoft Paint")
agent.execute_task("Open File Explorer")
```

### ✅ Autonomous Game Playing (Advanced)
```python
# It will try to play the game!
agent.execute_task("Play Solitaire for me")
agent.execute_task("Play Minesweeper")
```

**What it does:**
1. Analyzes the game screen
2. Detects game state (cards, mines, etc.)
3. Decides best move
4. Executes the move
5. Repeats until win/lose

---

## 🏗️ How It Works

### Architecture

```
User Request ("Draw a cat")
    ↓
Natural Language Understanding
    ↓
Task Planning (Step-by-step plan)
    ↓
Step Execution Loop:
    - Observe Screen (Computer Vision)
    - Decide Action (AI Reasoning)
    - Execute Action (Desktop Control)
    - Verify Result
    ↓
Task Complete!
```

### Key Components

1. **Task Planner** - Breaks down high-level tasks into steps
2. **Vision System** - Understands what's on screen (screenshots, OCR, object detection)
3. **Reasoning Engine** - Decides what to do next
4. **Action Executor** - Performsthe actual mouse/keyboard actions
5. **Feedback Loop** - Verifies actions succeeded

---

## 📋 Supported Tasks (Current)

### Tier 1: Fully Automated ✅

| Task Pattern | Example | Status |
|-------------|---------|---------|
| Draw shapes in Paint | "Draw a circle" | ✅ Working |
| Basic text entry | "Type Hello" | ✅ Working |
| Launch applications | "Open Paint" | ✅ Working |

### Tier 2: Partially Automated 🔨

| Task Pattern | Example | Status |
|-------------|---------|---------|
| Form filling | "F

QUICK_REFERENCE.md

# Desktop Control - Quick Reference Card

## 🚀 Instant Start

```python
from skills.desktop_control import DesktopController

dc = DesktopController()
```

## 🖱️ Mouse Control (Top 10)

```python
# 1. Move mouse
dc.move_mouse(500, 300, duration=0.5)

# 2. Click
dc.click(500, 300)  # Left click at position
dc.click()           # Click at current position

# 3. Right click
dc.right_click(500, 300)

# 4. Double click
dc.double_click(500, 300)

# 5. Drag & drop
dc.drag(100, 100, 500, 500, duration=1.0)

# 6. Scroll
dc.scroll(-5)  # Scroll down 5 clicks

# 7. Get position
x, y = dc.get_mouse_position()

# 8. Move relative
dc.move_relative(100, 50)  # Move 100px right, 50px down

# 9. Smooth movement
dc.move_mouse(1000, 500, duration=1.0, smooth=True)

# 10. Middle click
dc.middle_click()
```

## ⌨️ Keyboard Control (Top 10)

```python
# 1. Type text (instant)
dc.type_text("Hello World")

# 2. Type text (human-like, 60 WPM)
dc.type_text("Hello World", wpm=60)

# 3. Press key
dc.press('enter')
dc.press('tab')
dc.press('escape')

# 4. Hotkeys (shortcuts)
dc.hotkey('ctrl', 'c')      # Copy
dc.hotkey('ctrl', 'v')      # Paste  
dc.hotkey('ctrl', 's')      # Save
dc.hotkey('win', 'r')       # Run dialog
dc.hotkey('alt', 'tab')     # Switch window

# 5. Hold & release
dc.key_down('shift')
dc.type_text("hello")  # Types "HELLO"
dc.key_up('shift')

# 6. Arrow keys
dc.press('up')
dc.press('down')
dc.press('left')
dc.press('right')

# 7. Function keys
dc.press('f5')  # Refresh

# 8. Multiple presses
dc.press('backspace', presses=5)

# 9. Special keys
dc.press('home')
dc.press('end')
dc.press('pagedown')
dc.press('delete')

# 10. Fast combo
dc.hotkey('ctrl', 'alt', 'delete')
```

## 📸 Screen Operations (Top 5)

```python
# 1. Screenshot (full screen)
img = dc.screenshot()
dc.screenshot(filename="screen.png")

# 2. Screenshot (region)
img = dc.screenshot(region=(100, 100, 800, 600))

# 3. Get pixel color
r, g, b = dc.get_pixel_color(500, 300)

# 4. Find image on screen
location = dc.find_on_screen("button.png")

# 5. Get screen size
width, height = dc.get_screen_size()
```

## 🪟 Window Management (Top 5)

```python
# 1. Get all windows
windows = dc.get_all_windows()

# 2. Activate window
dc.activate_window("Chrome")

# 3. Get active window
active = dc.get_active_window()

# 4. List windows
for title in dc.get_all_windows():
    print(title)

# 5. Switch to app
dc.activate_window("Visual Studio Code")
```

## 📋 Clipboard (Top 2)

```python
# 1. Copy to clipboard
dc.copy_to_clipboard("Hello!")

# 2. Get from clipboard
text = dc.get_from_clipboard()
```

## 🔥 Real-World Examples

### Example 1: Auto-fill Form
```python
dc.click(300, 200)  # Name field
dc.type_text("John Doe", wpm=80)
dc.press('tab')
dc.type_text("[email protected]", wpm=80)
dc.press('tab')
dc.type_text("Password123", wpm=60)
dc.press('enter')
```

Github ReposUpdated 3h agoRank 70

AionUi

Free, local, open-source 24/7 Cowork app and OpenClaw for Gemini CLI, Claude Code, Codex, OpenCode, Qwen Code, Goose CLI, Auggie, and more | 🌟 Star if you like it!

MCPOPENCLAW

Github ReposUpdated 6mo agoRank 70

activepieces

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

OPENCLAW

Github ReposUpdated 6mo agoRank 70

cherry-studio

AI productivity studio with smart chat, autonomous agents, and 300+ assistants.