linux-desktop

# Linux Desktop GUI Automation

Automate any X11 Linux desktop: capture screens, find and click elements, type, use hotkeys,
manage windows.

Preferred screenshot interpretation path: capture with `capture.sh` and interpret the image directly in your OpenClaw chat (existing image-capable model connection).

## Prerequisites

- X11 session running (XFCE, GNOME on X11, KDE on X11, i3, openbox, etc.)
- `DISPLAY` environment variable set (usually `:0`)
- Run `bash install.sh` once to install dependencies
- No extra key needed for screenshot interpretation when using OpenClaw's image-capable chat path

## Quick Reference

| Task | Command |
|------|---------|
| Take screenshot | `bash capture.sh` |
| Screenshot of window | `bash capture.sh --window "Firefox"` |
| List windows | `bash inspect.sh` |
| Active window info | `bash inspect.sh --active` |
| Find window by name | `bash inspect.sh --window "Firefox"` |
| Click at coordinates | `bash click.sh --x 500 --y 300` |
| Right-click | `bash click.sh --x 500 --y 300 --button right` |
| Double-click | `bash click.sh --x 500 --y 300 --double` |
| Click relative to window | `bash click.sh --window "Firefox" --x 200 --y 150` |
| Type text | `bash type.sh "hello world"` |
| Type into window | `bash type.sh --window "Terminal" "ls -la"` |
| Send hotkey | `bash hotkey.sh "ctrl+c"` |
| Send Enter | `bash hotkey.sh "Return"` |
| Scroll down | `bash scroll.sh --direction down --amount 3` |
| Scroll up at position | `bash scroll.sh --x 500 --y 300 --direction up --amount 3` |
| Focus window | `bash window.sh --action focus --window "Firefox"` |
| Minimize window | `bash window.sh --action minimize --window "Firefox"` |
| Maximize window | `bash window.sh --action maximize --window "Firefox"` |
| Close window | `bash window.sh --action close --window "Firefox"` |
| Move window | `bash window.sh --action move --window "Firefox" --x 100 --y 50` |
| Resize window | `bash window.sh --action resize --window "Firefox" --width 1280 --height...

# linux-desktop — OpenClaw Linux GUI Automation Skill

Full GUI automation for Linux X11 desktops. This skill gives OpenClaw the ability to capture screenshots, inspect windows, click, type, send hotkeys, scroll, and manage windows.

## Requirements

- Linux with X11 session (XFCE, GNOME on X11, KDE on X11, i3, openbox, etc.)
- `DISPLAY` environment variable set (usually `:0`)

## Installation

```bash
bash install.sh
```

This installs system dependencies (`xdotool`, `wmctrl`, `scrot`, `x11-utils`, `imagemagick`, Python).

Supported package managers: apt (Debian/Ubuntu), dnf (Fedora/RHEL), pacman (Arch).

## Tools

| Script | Purpose |
|--------|---------|
| `capture.sh` | Take screenshots (full screen or specific window) |
| `inspect.sh` | List windows, get window details (JSON output) |
| `click.sh` | Mouse click at coordinates (left/right/middle, single/double) |
| `type.sh` | Type text into the focused window |
| `hotkey.sh` | Send keyboard shortcuts (ctrl+c, alt+F4, etc.) |
| `scroll.sh` | Scroll up/down at current or specified position |
| `window.sh` | Window management (focus, minimize, maximize, close, move, resize) |

## Quick Start

```bash
# Take a screenshot
bash capture.sh

# List all open windows
bash inspect.sh

# Click at coordinates
bash click.sh --x 500 --y 300

# Type text
bash type.sh "Hello world"

# Send Ctrl+C
bash hotkey.sh "ctrl+c"
```

Preferred workflow: capture screenshot, interpret it directly in your OpenClaw chat, then act with coordinates.

## Testing

```bash
bash test.sh
```

## OpenClaw Integration

Because this repo is dedicated to this single skill, use the repo root as the skill folder:

```bash
cp -r <this-repo> ~/.openclaw/workspace/skills/linux-desktop/
```

Restart the OpenClaw gateway; the agent will read `SKILL.md`.

## Limitations

- **X11 only** — does not work on Wayland sessions
- Some applications with custom rendering may resist automation