Panel actions

A panel can offer actions to a coding agent: read the notes on a board, add an image, change a card. The person decides which task may use them, by connecting your panel to that task, and the agent calls them while it works.

• Your panel declares its actions in panel.json, each with a name, a description, a kind (read or write) and a schema for what goes in and, for a read, what comes out.
• The person connects the panel to a task, from Deska’s strip over the panel (Connect to a task) or from the task’s header (Connect a panel). The first time, Deska asks with its own sheet, which lists every action. Only local, supervised tasks connect: never Dex, never an agent in a terminal, never a task on another computer.
• The task’s agent sees two tools from Deska, one to list the panels connected to it with their actions and one to call an action. Deska routes a call only to a panel connected to that task, and checks it against your schema before your page sees it.
• Your page hears the call, does the work and answers. Deska checks the answer too.

Your panel never calls a tool and never sees the agent’s conversation. It hears only the calls to its own actions. The person can turn panel actions off for every task in Settings, Agents, Panel actions.

Actions need "sdk": 4. A version that declares them is always reviewed before it is published, and Deska reads the descriptions in review: they reach the agent, so write them for it, plainly, and never as instructions beyond what the action does.

What an agent hands your panel stays on the computer. So a version that declares agent may not also declare any of these, and the checks refuse each one:

• network with any host, so the page reaches no website.
• embeds or signedInEmbeds, so it shows no other site.
• The account permission, so it cannot sign people in.
• The open-url permission, since a link it builds could carry what the agent handed it.

It keeps everything else: deska.storage, deska.files, saving a file the person picks a place for, copying with clipboard-write, and notify.

The rule holds across versions, because storage and files outlive an update. Once any version of your panel’s id declared agent, no later version of it may declare any of those five: deska.dev refuses such a submission, and Deska refuses to install, update or reload it on a computer that ever had a version that worked with agents. A later version may drop agent and stay offline. To reach the network, submit the panel under a new id.

{
  "permissions": ["agent"],
  "sdk": 4,
  "agent": {
    "actions": [
      {
        "name": "add_image",
        "description": "Adds an image to the board, with an optional caption.",
        "kind": "write",
        "input": {
          "type": "object",
          "properties": {
            "image": { "type": "string", "maxLength": 1000, "format": "deska-image" },
            "caption": { "type": "string", "maxLength": 280 }
          },
          "required": ["image"]
        }
      },
      {
        "name": "read_board",
        "description": "Reads the notes and images on the board.",
        "kind": "read",
        "input": { "type": "object" },
        "output": {
          "type": "object",
          "properties": {
            "notes": { "type": "array", "maxItems": 100, "items": { "type": "string", "maxLength": 2000 } },
            "images": { "type": "array", "maxItems": 10, "items": { "type": "string", "maxLength": 36, "format": "deska-file" } }
          }
        }
      }
    ]
  }
}

At most 16 actions. The permission agent and the agent object need each other, and neither takes any other key.

A small part of JSON Schema, enough to describe a panel’s data and simple to check. Every node is one of these:

• The root of input and output is an object. Schemas nest at most 4 levels: the root is the first, and each property or array item adds one.
• Property names and enum values are short tokens, never sentences: the agent reads every one of them, and the person reads none, since Deska’s sheet shows each action’s name and description, not its schemas. One outside those characters refuses the package, with a problem that quotes it.
• Any node may carry a description of 1 to 200 characters, with no invisible characters. It is for whoever reads your panel.json, and it never reaches the agent (see What the agent sees): put what the agent must know in the action’s own description.
• Anything else ($ref, oneOf, minLength, pattern, a list of types) refuses the package, with a problem that names the keyword and where it is.

Deska checks every call against input before your page hears it: an unknown property, a missing required one, a longer text, more items, a value outside enum or a fraction for an integer never reaches you, and the agent is told what was wrong. Invisible characters are removed from the text it sends.

When the agent asks which panels are connected to its task, Deska lists each panel by its id and the name the person sees, and each action with only what the person saw on the connect sheet and what a call needs:

• Its name, kind and description, with invisible characters folded into spaces, at most 300 characters.
• Its input and output schemas, with every schema description taken out. No person read those, so they never reach the agent.

An action whose property names or enum values fall outside the rules above (a version installed before they held) is left out, and a call to it answers unknown_action. The whole list is held to 8 KB: actions are kept in panel.json’s order while they fit, and the rest are counted as left out. Put your most useful actions first.

const EMPTY = { notes: [], images: [], captions: {}, revision: 0 }
const readBoard = async () => (await deska.storage.get('board')) ?? EMPTY

async function run(action, input) {
  const board = await readBoard()
  if (action === 'read_board') {
    return { notes: board.notes.slice(-100), images: board.images.slice(-10) }
  }
  // add_image: input.image is a Blob Deska read from the task's folder
  const kept = await deska.files.put(input.image)
  const next = {
    ...board,
    images: [...board.images, kept.id],
    captions: { ...board.captions, [kept.id]: input.caption ?? '' },
    revision: board.revision + 1,
  }
  await deska.storage.set('board', next)
  render(next)
  return { ok: true, revision: String(next.revision) }
}

// Listen before ready: a task already connected may call as soon as the page is up
deska.on('agent.action', async ({ callId, action, input }) => {
  const result = await run(action, input).catch(() => ({ ok: false, error: 'failed' }))
  await deska.agent.respond(callId, result).catch((err) => console.warn(action, err.code))
})

deska.on('agent.connected', ({ tasks }) => {
  connectedNote.hidden = tasks === 0
})

render(await deska.ready().then(readBoard))

Your page hears agent.action with { callId, action, input }: the action’s name and an input that already passed its schema. Listen before you call deska.ready(). A call that arrives before any listener waits for the first one (the SDK keeps the latest 16), but its 30 seconds are already running.

init.agent.tasks says how many tasks the panel is connected to when its page starts (0 on its Extras page), and agent.connected follows after init and on every change, so a page that reloads knows either way. A connection belongs to the panel on the desk, not to its Extras page.

A write answers only whether it worked: { ok, revision?, error? }. Nothing else your page says reaches the agent, and any other key refuses the answer. So a write can never carry the panel’s text back into the conversation.

• ok: true or false.
• revision: optional, 1 to 64 of letters, digits, ., _ and -. Use it to let the agent tell one state of the panel from the next.
• error: only with ok: false, one of invalid, conflict, not_found and failed.

The agent never sees your revision itself, since it is your text. It reads a counter Deska keeps for each task and panel instead: "revision":"r1", then r2, and the same counter again for the same revision (it keeps the last 256). When a later call’s input carries exactly a counter Deska gave that task for your panel, your page receives your own revision in its place. So use revisions as usual, for example to answer conflict when a change names a revision that is no longer current.

Make writes the person can undo, the way they undo their own edits: the agent works on their panel, not on a copy.

A read answers what its output schema allows. Deska checks it, removes every character drawn as nothing (line breaks and tabs stay), re-encodes its images, and hands it to the agent fenced as content from your panel that the agent must not take as instructions:

<panel_content package="nook" panel="Nook" trust="untrusted">
{"notes":["Make the header blue"],"images":["image:1"]}
</panel_content>

An answer that fails its schema is refused with invalid_args, and the call keeps waiting for a good one until its 30 seconds run out. Deska also holds the JSON to 65,536 characters, as sent and once escaped for the fence: past that, respond still answers true, and the agent is told the answer was too large. Because what a read answers can carry text anyone wrote, the turn pauses for approvals first. See The pause for approvals.

• In. A string with "format": "deska-image" in an input is a path the agent gives, to an image in the task’s working folder. Deska reads it itself (a PNG, JPEG or WebP up to 10 MB, never outside that folder or through a link that leads out of it), and your page receives a Blob where the path was. Keep it with deska.files.put.
• Out. A string with "format": "deska-file" in an output is the id of an image your panel keeps in deska.files. Deska reads it, writes it again as a PNG of at most 2,048 pixels a side with none of the original’s metadata, and the agent receives it as an image after the text, where the id says image:1, image:2 and so on. An id the panel does not keep becomes image:unavailable.
• At most 10 images in one call or one answer.

Some content should reach a task only when the person says so: feedback written on the board, the images they picked. Your page offers it, and Deska’s Send to task button lights up on its strip over the panel. Your page can never press it.

// Say what Send to task would send whenever the selection changes.
// No text and no images withdraws the offer.
async function offerSelection() {
  await deska.agent.offer({ text: commentBox.value.trim(), files: selectedImageIds.slice(0, 6) })
}

Text inside a panel can try to steer an agent. So when a read’s answer, or content the person sends with Send to task, enters a turn, Deska switches that task to Supervised for the rest of that turn, whatever mode it runs in, and puts the mode back when the turn ends. Supervised is each agent’s own asking mode: what it asks the person about is that agent’s choice, not Deska’s. A line in the task says why: <panel> sent content into this turn, so this task is in Supervised until the turn ends. A write’s answer carries no panel text and pauses nothing.

While the pause holds, Deska also:

• Refuses every call to its browser preview tools, and tells the agent to wait for the person.
• Ignores an Always allow this session answer given before the pause, where Deska keeps it, so the same request asks again. A new one counts as allow once.
• Launches no Claude Code subagent.

When the turn ends, Deska puts the mode back and adds a row: The turn is over, so this task is back to <mode>. The content <panel> sent is still in this conversation, and later turns can read it. Its Keep asking sets the task to Supervised until the person changes it. Only the person’s own change of mode ends a pause early. Dex’s never does: Dex never reads what a panel sent, and while a pause stands it needs the person’s confirmation to loosen the task’s mode or to approve something for it.

A panel’s message waits for its own turn, wherever it lands in the queue. Deska never adds it to a turn already running, Up next offers it neither Steer now nor Edit, and once sent it cannot be edited.

Some agents switch to Supervised in the middle of a turn, and some only from the next one. On those, a read is deferred: the call answers at once with a sentence from Deska and nothing from your panel, and when the turn ends Deska sends the panel’s answer to the task as its next message, marked as coming from the panel, in a turn that asks. One read waits per task and panel: a newer read of the same panel replaces it, and stopping or interrupting the task drops it. Listing the panels, writes and Send to task work on every agent.

The pause narrows what panel content can do in a turn. It does not hold everything:

• A rule an agent kept from an Always allow this session answer given before the pause still applies inside that agent. Deska can neither see nor lift it.
• A Claude Code subagent already running when the pause starts keeps its own mode.
• What each agent’s asking mode does not ask about is that agent’s choice. Codex and Cursor apply it from the next turn, which is why their reads are deferred.
• Antigravity’s ACP runtime refuses a read outside the task’s folder rather than asking about it.
• A host name your page writes into a resource hint can still reach the DNS resolver, carrying at most the name.
• Text drawn inside an image can still carry instructions.

• Load the folder in Build a panel, open a supervised task in the same project on this computer, and press Connect to a task on the panel’s strip. Deska asks the same question it will ask the people who install it.
• Ask the agent to list the panels connected to its task, then to call each action.
• The panel’s page in the Library counts, under What it did on this computer, every action a task called and every call Deska refused, with why.
• When Deska refuses a call, the agent receives a fixed sentence from Deska with a code (such as not_connected, invalid_input or panel_timeout), never your page’s text.

See also Panels connected to a task, what the person sees.