Does it take control of my mouse?

No. Actions go through the macOS accessibility layer, so the real cursor never moves. The pointer you see on screen is drawn by Spooky Action. There is a separate fallback command group, spooky real, for apps that expose nothing to press — it does borrow the cursor, it says so out loud, and every result it returns is marked cursor_touched: true.

How do I take my screen back?

Move your mouse. The agent loses the screen immediately and is told immediately, rather than finding out from its next screenshot. The glow turns blue while you hold it. It resumes once you have been still for a couple of seconds.

What does it need permission to do?

Accessibility, and nothing else. spooky setup opens the exact System Settings pane, names the one row to switch on, waits, and then proves the permission works by using it for real.

How do I uninstall it?

Run ./uninstall.sh in the checkout. It removes the state directory, the virtualenv, the symlink and any running overlay, names each thing as it goes, and opens the Accessibility pane so you can untick the permission — the one part a script cannot do for you.

Which agents work with it?

Any of them. It is a command line tool, so Claude Code, an SDK loop, a shell script or a cron job all drive it the same way. Every command takes --json, and every command exits 75 with a concrete suggestion when it genuinely needs a human.

Spooky Action

The agent has the screen

This is what it looks like when a machine is driving.

Invisible automation is worse than no automation. So while the agent is working, the edges of every display glow, a drawn pointer glides to whatever is about to be pressed, and a pill at the top says what the job is.

A Mac desktop with a yellow glow around all four edges. A drawn pointer with a small chip reading "bold" rests on the Bold button in the TextEdit toolbar, and a dark pill at the top of the screen reads "Tidying up a TextEdit draft". — A real screenshot, not a mockup. The pointer on the Bold button is drawn by Spooky Action — the actual cursor is somewhere else entirely, wherever its owner left it.

The glow follows the job, not the click

An agent that lights the screen for every individual press and drops it a second later reads as a fault, and teaches you to ignore the one signal that matters. So the agent opens a run, and the run is what the screen reflects.

zsh — spooky

$ spooky begin "Tidying up a TextEdit draft"
  ✓ the screen is yours — Tidying up a TextEdit draft
    it closes itself in 15 min if you forget `spooky end`

$ spooky press TextEdit 8 --expect "bold"
  ✓ pressed bold — your cursor did not move

$ spooky say "Step 2 of 3 — setting the revised date"
  Pill now reads: Step 2 of 3 — setting the revised date

$ spooky end
  ✓ done — screen released

The same desktop, still glowing yellow. The pointer has moved into the body of the document with a chip reading "the date line", and the pill now reads "Step 2 of 3 — setting the revised date". — Same run, one step later. The glow never blinked; only the label changed.

You have the screen

Move your mouse. That is the whole gesture.

The instant you touch the mouse the agent loses the screen and is told immediately — not on its next screenshot. That word is the entire safety argument. An agent that finds out it lost the screen by looking at a picture afterwards has already typed into your document.

Everything turns your colour. The run does not end, it waits; you get it back the moment you have been still for a couple of seconds, and it is told that too.

Agent

Close-up of the control pill: a yellow dot and the text "Tidying up a TextEdit draft", with pause and move buttons.

You

The same pill in blue, reading "You have the screen — the agent is waiting".

The pill is draggable and remembers where you put it. The ▮▮ button pauses everything from the screen itself; spooky pause from a terminal means exactly the same thing, and neither can leave the other stuck.

How it presses things

It presses the element. Not the coordinate the element used to be at.

Every other desktop-automation tool warps your cursor to a point and synthesizes a click. A screenshot goes to a model, the model says the Save button is at 840, 612, and by the time the click lands the sheet has moved fourteen pixels. Something else has been pressed and nobody knows what.

Spooky Action goes through the macOS accessibility layer — the same tree a screen reader uses. Elements have identities there, so it can name what it is about to touch and check it is still what it was.

zsh — spooky tree

$ spooky tree Safari

    12  Button           Reload                                    Press
    13  TextField        Address and Search                        Confirm
    27  Button           Save                                      Press

$ spooky press Safari 27 --expect "Save"
  ✓ pressed Save — your cursor did not move

# and when the sheet moved between reading and pressing:
$ spooky press Safari 27 --expect "Save"

  index 27 is now "Don't Save", not "Save"
  → the interface changed after `tree` ran — take a fresh
    tree and use the new index
    exit 75 · nothing was pressed

That guard is the difference between automation you can leave running and automation you have to babysit. It costs one extra accessibility read and it turns the worst failure mode in computer use — pressing the wrong thing confidently — into a refusal with an exit code.

This is how Codex does computer use on a Mac. Not a guess — the shipped binaries were checked. SkyComputerUseService links the whole AXUIElement family and contains zero mouse or keyboard event symbols. Its on-screen cursor is drawn, not moved.

Identity, not pixels

Indices come from a deterministic walk of the accessibility tree. The same index means the same element until the UI actually changes.

Every result is auditable

A click is a click and leaves no record. A press names the element it hit, in text, in the log, and in --json.

Locks expire

An agent that crashes mid-action cannot keep your screen. Held locks and open runs both time out on their own.

When there is nothing to press

The honest fallback, kept deliberately separate.

A canvas, a game, an old Java app — some things offer the accessibility layer nothing at all. There is a way through, and it is its own command group because it does borrow your cursor:

zsh — the fallback

$ spooky real click 840,612
  ! borrowing your cursor — nothing here exposes an element
  ✓ clicked 840,612 · cursor put back at 1204,388
    cursor_touched: true

It announces itself, puts your pointer back where it found it, and marks every result cursor_touched: true. The moment a tool stops distinguishing between I pressed the button and I took your mouse, you stop being able to trust either.

The command line

Everything it does, from a terminal.

There is no daemon to configure and no protocol to speak. Every command takes --json, and every command exits 75 with a concrete suggestion when it genuinely needs a person — a handoff, not a failure.

Command	What it does
spooky apps	what it can act on right now
spooky tree APP	what is on screen, with indices
spooky find APP TEXT	search an app's screen by label
spooky press APP N	press by index, with --expect
spooky type APP N TEXT	put text into a field
spooky click APP LABEL	press by label, if exactly one matches
spooky focus APP	bring an app forward
spooky at X,Y	what is at this point — identified, not guessed
spooky begin / say / end	open a run, retitle it, close it
spooky control / pause / resume / wait	who has the screen
spooky watch	start or stop the on-screen presence
spooky config	every setting, in plain English
spooky setup / doctor	permissions and health
spooky real …	the fallback that does take your cursor

Change how it looks and behaves, live

zsh — spooky config

$ spooky config accent=yellow wash=0   # glow only, no screen tint
$ spooky config pointer=off hud=left
$ spooky config glide_seconds=1.2      # slow the pointer down to follow it
$ spooky config --reset

The running overlay picks it up live. No restart.

Arriving, and leaving

One command in. One command out.

zsh — install

$ git clone https://github.com/Jumpsy/spooky-action.git
$ cd spooky-action && ./install.sh

  Spooky Action
  ✓ Python 3.12.7
  ✓ spooky installed
  ✓ built the accessibility helper and the overlay, from source
  ✓ skill linked into ~/.claude/skills — Claude Code picks it up next start
  ✓ spooky linked into ~/.local/bin

  Next:

      spooky setup     grant Accessibility — it opens the pane and checks the fix took
      spooky apps      what it can act on
      spooky doctor    what works right now

  Changed your mind? ./uninstall.sh removes all of it.

setup does not tell you to go find System Settings. It opens the exact pane, names the one row to tick, waits, and then proves the permission took by using it for real. If it did not take, it says so and stays on that pane.

It builds into a virtualenv inside the folder, and it compiles the two Swift helpers on your machine from the source in this repo. There is no prebuilt binary, because a prebuilt binary that reads your screen and drives your apps is exactly the thing nobody should run on trust.

And leaving should be as easy as arriving

zsh — uninstall

$ ./uninstall.sh

  Removing Spooky Action.

  This will delete:
    ~/.spooky  (settings, logs, built binaries)
    ~/spooky-action/.venv
    ~/.local/bin/spooky
    ~/.claude/skills/spooky-action  (the Claude Code skill)
    the overlay, if it is running

  It will NOT touch this folder itself, or anything you made with it.

  Go ahead? [y/N] y

  ✓ removed the running overlay
  ✓ removed ~/.spooky
  ✓ removed ~/.local/bin/spooky
  ✓ removed ~/.claude/skills/spooky-action
  ✓ removed ~/spooky-action/.venv

  One thing a script cannot undo for you: the macOS permission.
  Opening the pane now — switch OFF the row for your terminal app,
  if nothing else you use needs it.

If you use this once and decide it is not for you, that should take ten seconds and leave nothing behind.

The other half

Spooky Action has hands. It does not have eyes.

No screenshots, no vision, no web. That is deliberate — plenty of people want an agent that can act on their Mac without one that watches their screen, and bundling the two means granting a permission for a capability you did not ask for. Spooky Action never requests Screen Recording, because it has no use for it.

Agent Eyes is the other half: screenshots an agent can actually look at, design systems pulled out of the live CSSOM, local vision models so looking costs you nothing. Separate repos, separate installs, separate permissions.

zsh — both halves

$ agent-eyes screen --grid        # see it
$ spooky press Safari 27          # act on it

Before you start

What it needs, and what it is.

macOS

The accessibility layer this is built on is a macOS API. There is no Windows or Linux port and there cannot be a straight one.

Python 3.10+ and Xcode CLT

xcode-select --install — the two Swift helpers are compiled on your machine from the source here.

Accessibility permission

One row in System Settings, and nothing else. It never asks for Screen Recording.

Early, and building in the open. MIT licensed. The screenshots on this page are real captures of the tool running on a real desktop — if something here looks better than it works, that is a bug worth filing.

Your agent acts at a distance.Your cursor never moves.