# Paintapp

Paintapp (https://paintapp.org) is a browser-based image editor: Photoshop-class raster and
vector editing, layers and layer styles, a node-based compositing graph, natural-media painting,
camera raw, 3D and video — delivered as an installable PWA that runs entirely in the browser.
It reads and writes a large set of image, document and video formats.

**The entire user interface is rendered into one `<canvas>` element.** There is no DOM behind
the pixels and no accessibility tree: menus, panels, buttons, layers and text are drawn, not
elements. DOM queries, accessibility-tree automation and text-based computer-use heuristics all
fail here — an empty DOM does not mean the page failed to load. Use the entry points below.

## Driving this app

In order of preference:

1. **MCP bridge (recommended, needs a checkout).** The repository's `tools/paintapp-mcp` is an
   MCP server that exposes this app. It is not on npm, so run it from a checkout: `npm install`
   in that directory, then `node index.mjs --url https://paintapp.org` — pass the URL, it
   defaults to a local dev server. It launches a browser session, waits for the app to boot, and
   mirrors every in-app tool 1:1 as an MCP tool, so an MCP client drives the app with no custom
   code. The tool list is read from the running app, not hardcoded — new tools appear
   automatically.
2. **In-page bootstrap contract: `window.app.ai`.** The one documented, stable call point for a
   console, CDP or Playwright agent:
   - `await window.app.ready` — Promise; resolves when the app shell has booted.
   - `await window.app.ai.ready` — Promise; resolves when the tool handlers are registered.
   - `await window.app.ai.listTools()` — the tool schemas, in OpenAI function-calling format.
   - `await window.app.ai.dispatch(name, args)` — invoke one of those tools; returns the raw
     handler result. Image results come back as data URLs.
   - `await window.app.ai.ask(text, opts)` — ask the app's OWN assistant and run the whole turn
     (model call, tool loop, edits) to completion; resolves with `{reply, toolLog, messages}`.
     `dispatch` makes YOU the agent; `ask` drives the assistant the user talks to, on the same
     engine the panel uses. With the Assistant panel open the turn renders there as it runs.
   - `await window.app.ai.newSession(opts)` — an isolated conversation object with the same API.

   Representative tools (call `listTools()` for the authoritative list and schemas):
   `getUIState` (what is on screen right now), `getObjectModel` (the scripting type library),
   `getCommandInfo` (every command with its parameter signature),
   `execute_command` (`{command, params, mode}`), `getProperties` (settings of the current
   brush/tool/layer/document), `listLibraryItems` (brushes, swatches, shapes, textures),
   `listFilters` / `inspectFilter`, `listFileFormats`, `browseFiles`, `searchDocs`, `runScript`,
   `generate_image` (a prompt in, an image out — spends the signed-in user's AI credit; the
   image is saved to `app:user/ai_generated_images/` and that path is returned, so `fileOpen`
   or `newLayerFromFile` can then act on it).

   Example:

   ```js
   await window.app.ready;
   await window.app.ai.dispatch("execute_command",
       {command: "fileNew", params: {width: 1920, height: 1080}});
   const state = await window.app.ai.dispatch("getUIState", {});
   ```

3. **`window.app.ui` — raw canvas-UI automation.** Lower level; use it when a tool does not cover
   what you need. All coordinates are canvas-space: `query(className)`, `queryById(id)`,
   `queryText(text)`, `getBounds(className)`, `clickByText(text)`, `simulateClick(x, y)`.

`window.app` is the application root object; everything above hangs off it.

## Machine-readable model

- `/docs/model/object-model.json` — the scripting type library, as `{columns, classes}`: each
  class maps to its own rows of `[member, access, type, description]` (the same data
  `getObjectModel` returns).
- `/docs/model/commands.json` — every command with its id, description and parameter signature.
- `/docs/model/ai-tools.json` — the tool schemas, OpenAI function-calling format.
- `/docs/pages/pages-index.json` — the documentation index: `{pages: [{path, title}]}`.

These are generated from the running app by `npm run docs:model` and committed, so a very fresh
deploy can outrun them. `listTools()` and `getObjectModel` in the page are the authoritative live
model; these files are what to read before you have a page.

## Documentation

- `/docs/` — the documentation site.
- Deep links use a hash fragment naming the page: `/docs/#brushes`, `/docs/#automation`,
  `/docs/#assistant`, `/docs/#node-based-vs-layer-based`, `/docs/#file-formats`. The fragment is
  the basename of a `/docs/pages/pages-index.json` entry, without the `pages/` prefix and the
  `.html` suffix: `pages/brushes.html` → `/docs/#brushes`.
- `searchDocs` (via `window.app.ai.dispatch`) searches the same pages and returns snippets plus
  the `/docs/#page` URL to cite.

## Notes

- Documents live in the user's own browser (IndexedDB and the local filesystem). Nothing here
  grants access to anyone else's data.
- Everything reachable through `window.app.ai` is what the user's own in-app assistant can
  already do in that page; the blast radius is the browser session it runs in.