🦀 Gajae Code

Tool guide

computer desktop-control tool

Tell your GJC session to use the computer tool when it needs to see the local screen, click, type, scroll, or drive a desktop app.

How to use it

There is no separate gjc computer subcommand. Start a normal GJC session, enable the tool on a supported host, then ask the agent to use the computer tool.

# 1. Launch GJC from the repo/workspace where you want the agent to work
gjc

# 2. In your prompt, be explicit
"Use the computer tool to inspect the app window, click the login button, and type the test account."

# 3. Keep the kill-switch available
# Default: Control+Option+Command+Escape

What it can do

The tool mirrors the OpenAI computer-use action set:

  • screenshot — capture the current primary display.
  • click, double_click, move, drag, scroll — pointer control.
  • type, keypress — keyboard input.
  • wait and batch — bounded timing and grouped actions.

The screenshot's pixel dimensions are the coordinate space. Rust owns the Retina/HiDPI transform to macOS logical points, so the agent should use coordinates from the most recent screenshot.

Platform and permissions

  • Platform: Apple Silicon macOS only in v0.7.2. Linux/Windows are deferred behind the same schema.
  • Permissions: Screen Recording and Accessibility are required. Missing permissions are surfaced as explicit errors rather than silent black screenshots.
  • Safety: the global kill-switch is outside model control and can stop queued input, release held buttons/keys, and suspend further computer actions.

This is an experimental, opt-in surface. Use it for local desktop coordination where visual feedback matters, and verify important actions before relying on them.

Prompt examples

Use the computer tool to take a screenshot and tell me what app is frontmost.

Use the computer tool to click the first empty row in the spreadsheet and type this CSV row.

Use the computer tool to open the browser window, inspect the error banner, and stop after the screenshot.

Keep prompts concrete: name the visible app, the action to perform, and the stop condition.