computeruse

Computer Use Skill (Gemini Browser Automation)

Enable Rhea to see and control a browser using Gemini 2.5 Computer Use model. The AI analyzes screenshots and generates mouse/keyboard actions.

Features

• Visual Understanding: AI sees the screen via screenshots
• Browser Control: Click, type, scroll, navigate
• Multi-step Tasks: Chains actions to complete complex goals
• Safety Checks: Built-in confirmation for risky actions

Capabilities

Requirements

pip install google-genai playwright
playwright install chromium

Usage Examples

Run a Web Research Task

from rhea_noir.skills.computeruse.actions import skill as cu

result = cu.run_task(
    goal="Search for 'best AI frameworks 2026' on Google and list the top 3 results",
    start_url="https://www.google.com",
    max_steps=10
)
print(result["final_answer"])

Automate Form Filling

result = cu.run_task(
    goal="Fill out the contact form with name 'Dave', email 'dave@example.com', and submit",
    start_url="https://example.com/contact"
)

Web Scraping with Context

result = cu.run_task(
    goal="Go to Amazon and find the price of Sony WH-1000XM5 headphones",
    start_url="https://www.amazon.com"
)

Safety Features

The model includes built-in safety checks:

• require_confirmation: User must approve risky actions
• Excluded actions: Block specific UI actions if needed

result = cu.run_task(
    goal="...",
    excluded_actions=["drag_and_drop"],
    require_human_confirmation=True
)

Supported UI Actions

Model

Uses gemini-2.5-computer-use-preview-10-2025 - specialized for browser control.

[!CAUTION] Computer Use is a Preview feature. Supervise closely for important tasks.