Xpersona
L

Linux GUI Control

CLAWHUB · Unverified

Control the Linux desktop GUI using xdotool, wmctrl, and dogtail. Use when you need to interact with non-browser applications, simulate mouse/keyboard input, manage windows, or inspect the UI hierarchy of applications on X11/GNOME. Supports: (1) Clicking/typing in apps, (2) Resizing/moving windows, (3) Extracting text-based UI trees from apps (A11y), (4) Taking screenshots for visual analysis. Skill: Linux GUI Control Owner: dreamtraveler13 Summary: Control the Linux desktop GUI using xdotool, wmctrl, and dogtail. Use when you need to interact with non-browser applications, simulate mouse/keyboard input, manage windows, or inspect the UI hierarchy of applications on X11/GNOME. Supports: (1) Clicking/typing in apps, (2) Resizing/moving windows, (3) Extracting text-based UI trees from apps (A11y), (4) Taking

62
Rank
84
Safety
13k
Downloads

About

What it does, and when to use it.

Capability contract not published. No trust telemetry is available yet. 12.7K downloads reported by the source. Last updated 11/16/2026.

Avoid when

  • Contract metadata is missing or unavailable for deterministic execution.

Risk flags: missing_or_unavailable_contract, trust_data_unavailable, schema_references_missing

Install and run

Setup complexity: low.

clawhub skill install s17bxsj8479c90rxv90k3hc84x885tz9:guicountrol
  1. Setup complexity is LOW. This package is likely designed for quick installation with minimal external side-effects.
  2. Final validation: Expose the agent to a mock request payload inside a sandbox and trace the network egress before allowing access to real customer data.

Contract: missing

curl -s "https://www.xpersona.co/api/v1/agents/clawhub-dreamtraveler13-guicountrol-countrol-2/snapshot"

Run-check

$0.02 USD

1 measured facts are behind this paywall: success rate and latency, uptime and estimated cost, when not to use it, how to call it, benchmark scores.

Agents pay $0.02 in USDC. A card payment is $0.50, the smallest a card allows.

Documentation

CLAWHUB

5,826 characters of source documentation, loaded on request.

Extracted files

3 files captured from the source.

SKILL.md

---
name: linux-gui-control
description: "Control the Linux desktop GUI using xdotool, wmctrl, and dogtail. Use when you need to interact with non-browser applications, simulate mouse/keyboard input, manage windows, or inspect the UI hierarchy of applications on X11/GNOME. Supports: (1) Clicking/typing in apps, (2) Resizing/moving windows, (3) Extracting text-based UI trees from apps (A11y), (4) Taking screenshots for visual analysis."
---

# Linux GUI Control

This skill provides tools and procedures for automating interactions with the Linux desktop environment.

## Quick Start

### 1. Identify Target Window
Use `wmctrl` to find the exact name of the window you want to control.
```bash
wmctrl -l
```

### 2. Inspect UI Hierarchy
For apps supporting accessibility (GNOME apps, Electron apps with `--force-renderer-accessibility`), use the inspection script to find button names without taking screenshots.
```bash
python3 scripts/inspect_ui.py "<app_name>"
```

### 3. Perform Actions
Use `xdotool` via the helper script for common actions.
```bash
# Activate window
./scripts/gui_action.sh activate "<window_name>"

# Click coordinates
./scripts/gui_action.sh click 500 500

# Type text
./scripts/gui_action.sh type "Hello World"

# Press a key
./scripts/gui_action.sh key "Return"
```

## Workflows

### Operating an App via Text UI
1. List windows with `wmctrl -l`.
2. Activate the target window.
3. Run `scripts/inspect_ui.py` to get the list of buttons and inputs.
4. Use `xdotool key Tab` and `Return` to navigate, or `click` if coordinates are known.
5. If text-based inspection fails, fallback to taking a screenshot and using vision.

### Forcing Accessibility in Electron Apps
Many modern apps (VS Code, Discord, Cider, Chrome) need a flag to expose their UI tree:
```bash
pkill <app>
nohup <app> --force-renderer-accessibility > /dev/null 2>&1 &
```

## Tool Reference

- **wmctrl**: Window management (list, activate, move, resize).
- **xdotool**: Input simulation (click, type, key, mousemove).
- **dogtail**: UI tree extraction via AT-SPI (Accessibility bus).
- **scrot**: Lightweight screenshot tool.

_meta.json

{
  "ownerId": "kn73q6e6sgjtn6nd5pca7wgh9n80ax0y",
  "slug": "guicountrol",
  "version": "1.0.0",
  "publishedAt": 1769949980822
}

skill-card.md

## Description:

Control the Linux desktop GUI using xdotool, wmctrl, and dogtail for non-browser application interaction, mouse and keyboard input simulation, window management, UI hierarchy inspection, and screenshot capture on X11/GNOME.

This skill is ready for commercial/non-commercial use.

## Publisher:

[dreamtraveler13](https://clawhub.ai/user/dreamtraveler13)

### License/Terms of Use:


## Use Case:

Developers and automation agents use this skill to operate Linux desktop applications by focusing windows, sending keyboard and mouse input, inspecting accessible UI trees, and capturing screenshots when text-based inspection is insufficient.

### Deployment Geography for Use:

Global

## Known Risks and Mitigations:

Risk: GUI controls can click, type, focus, move windows, and otherwise act broadly on the live desktop session.

Mitigation: Use only in desktop sessions where broad GUI control is acceptable, verify target windows before acting, and keep important work saved before running automation.

Risk: The screenshot helper can be abused through unsafe filename handling.

Mitigation: Patch or avoid the screenshot helper until it validates filenames and passes an end-of-options marker to scrot.

Risk: Untrusted action arguments could drive unintended desktop input or window changes.

Mitigation: Avoid using untrusted text as action arguments and review commands before execution.

Risk: The accessibility workflow suggests pkill instructions that can terminate running applications.

Mitigation: Save work and confirm the target process before following pkill-based accessibility instructions.

## Reference(s):

- [ClawHub skill page](https://clawhub.ai/dreamtraveler13/skills/guicountrol)

## Skill Output:

**Output Type(s):** [Shell commands, Code, Guidance]

**Output Format:** [Markdown with inline bash commands and script references]

**Output Parameters:** [1D]

**Other Properties Related to Output:** [May include desktop automation commands that act on the live GUI session.]

## Skill Version(s):

1.0.0 (source: server release metadata)

## Ethical Considerations:

Users should evaluate whether this skill is appropriate for their environment, review any generated or modified files before relying on them, and apply their organization's safety, security, and compliance requirements before deployment.

Sponsored

Ads related to Linux GUI Control and adjacent AI workflows.