The agent action API
Tilo apps export typed actions. A program that cannot see the screen can ask the shell what is possible right now, and then perform one of those things by name. No screenshot, no hit-test, no synthetic click.
Two ops carry this, on either control socket: the device admin’s (below) or a
development session’s TILO_TEST_CONTROL socket. Both use the existing
newline-delimited JSON framing described in docs/headless-testing.md: one JSON
object per line, request and response.
Driving a device as its admin
Every compositor build serves the action model at
$XDG_RUNTIME_DIR/tilo-session/admin-control.sock in the device admin’s
session, once first login is complete. tilo-control (in every image) uses it
by default for actions list|invoke|status|cancel, windows and state-dump:
ssh admin@device
tilo-control actions list notes
tilo-control actions invoke applications so.tilo.notes # typed receipt
tilo-control windows # the window it opened
tilo-control actions invoke files /home/admin/Documents/report.md
tilo-control actions invoke session lock
Nothing about the session changes because the socket exists: first-use
onboarding, the lock screen and the capability gate behave exactly as with no
client attached. While the session is locked, ordinary app invocations are
refused with session_locked. The authenticated owner’s session-lock
provider remains available for the normal PAM unlock conversation.
Who may use it is decided by the system’s own identity and process model, not by a token:
- Only the device admin’s session binds it: an account in
wheelwhose active identity is the one that claimed the device at its first login and holds its primary device certificate (tilo_identity_host::provisioning::active_identity_owns_device). Another account on the machine, or the admin before first login, has no socket. - Only the admin’s own unconfined processes connect. Each connection is
checked against the kernel’s
SO_PEERCRED: same uid as the session, same user and mount namespaces (a sandboxed app has its own, and is never shown the path), and not a process this session launched, confined or not. An app acts through its own collaboration connection and grants, never with the admin’s. The socket is 0600 in a 0700 directory besides. - It serves the action model only: enumerate, invoke, status, cancel,
windows,state_dumpandforeign_frames. Synthetic input, screenshots, grants, fixtures and agent seats are refused withnot_permitted, naming the development socket that serves them. - Every invocation passes the same capability gate as the palette, as the person at this session.
Observing an installed native application’s exact window
Store’s native open action is a background action with an explicit JSON result.
After real adopted-window readiness and revalidation of the original caller, its
launch.adopted_window identifies the surface, package, room and an opaque host-issued adoption UUID.
The UUID identifies that resource lifetime, including across numeric ID reuse;
knowing it grants no authority. It contains no
kernel PID. Legacy hosts may return launch: null; that proves no particular
window and cannot qualify resident-package acceptance. Flatpak Open retains its
unit-job result contract.
The authenticated device owner’s existing state-dump adds
chrome.native_window_provenance: at most 128 current window associations from
confined, launcher-attributed Wayland connections. Each row carries the current
room, tab, surface, matching adoption UUID, package, local record instance and
kernel peer PID/start time.
The dump omits unattested, missing, closed, foreign or reused-process associations;
omitted reports overflow. Agent-control cannot request this owner diagnostic.
The official Store acceptance runner joins only the adopted surface to this current association, before and after reading retained kernel executable and SDK file descriptors. A closed/replaced window or changed process lifetime refuses acceptance. Existing unrelated app workers are reported separately, never required to use the newly installed package and never closed by the runner.
Unlocking the owner’s session remotely
Use the owner’s existing unconfined login and the owner socket. Discover the
current session-lock actions, then submit the password Secret parameter to
unlock@session-lock as a private JSON object on standard input:
env -u TILO_TEST_CONTROL tilo-control \
--socket "$XDG_RUNTIME_DIR/tilo-session/admin-control.sock" \
actions list session-lock --json
env -u TILO_TEST_CONTROL tilo-control \
--socket "$XDG_RUNTIME_DIR/tilo-session/admin-control.sock" \
actions invoke session-lock unlock@session-lock --args-stdin
Supply stdin through the existing private credential channel, never a password
in arguments, receipts or logs. The invocation starts the same PAM verifier
as the lock screen; its acknowledgement does not prove authentication succeeded.
Inspect chrome.session_locked and chrome.lock_auth in state-dump to observe
completion. An incorrect credential leaves the session locked.
The management transport tilo-vm vm ssh executes commands as root. When using
that transport, run the control client as the actual session owner with
runuser -u OWNER --, set that owner’s XDG_RUNTIME_DIR, and keep the secret on
stdin. Root itself does not pass the owner socket’s peer checks. This unlocks an
existing login session; the pre-session greeter has its separate sign-in action
below, and encrypted-disk unlock is a separate boundary.
Permission requests use that same authenticated session surface. When a portal
or Terminal is waiting for consent, query actions list permission-prompts and
use the returned exact UUID-scoped decision: allow-once@<id>,
allow-session@<id>, allow-always@<id> or deny@<id> from provider
permission-prompts, with {"confirm": true}. Only the current queue head is
offered; listing and invoking require the current workspace owner. The action
rechecks ownership and request identity before releasing the waiting request.
It cannot create a grant without a prompt, answer a replaced request or reopen a
Host session as a person on an agent’s behalf. The portal/ush supervisor still
owns private persistence and Host’s existing person/device-owner checks.
A successful decision receipt confirms the answer, not a running PTY: inspect
Terminal’s local session afterward to observe its actual startup outcome.
Reading is an action too. A framework resource that sets
Resource::READABLE and answers Resource::read_content is published with
the framework’s read view: no arguments answers the resource itself, item
answers one nested resource it lists. It needs read permission only, passes
the same gate as every other action, and is tagged as the document read, so
the runtime MCP finds it too. Notes answers its pages and tags, and one page
(notes.page:<id>) as its title, text and blocks with the ids its edit
actions take:
tilo-control read so.tilo.notes # {"pages": [{"item": "notes.page:…", "title": …}], …}
tilo-control read so.tilo.notes notes.page:<id> # {"title": …, "text": …, "blocks": [{"id", "kind", "text"}]}
read APP [ITEM] finds the open app’s read row among the app-actions the
desktop lists and invokes it, so the caller needs no opaque id.
The CLI teaches its own use: tilo-control --help leads with the action
model, a socket it cannot reach is explained (and how to get one), and
unknown_action / invalid_arguments refusals print the next command to try.
os/tests/installed-admin-actions.sh proves this on a booted image in
build-image.yml: after first login, over the admin’s ssh and with no
TILO_TEST_CONTROL, it invokes Notes by address, lists it, and reads the window
back through tilo-control windows. It then creates a note through Notes’ new-page action and reads it back through read.
actions_enumerate
{"op":"actions_enumerate","query":"terminal"}
{"answer":{"query":"terminal","count":2,"actions":[
{"provider":"actions","id":"window.new.terminal","kind":"Actions",
"title":"New terminal window","subtitle":null,"score":0}
],"total":2,"truncated":false,"next_cursor":null,
"revision":"3f9c0a17d2b4e861","complete":true}}
complete is false while a source that would change the answer is still on
its way: the app action index rebuilding after an app published, or an app’s
directory page not yet in. That is a different thing from truncated (see
paging below). truncated says this page is not all of the listing, and
following next_cursor gets the rest. complete: false says the listing
itself is not all there yet. revision names the listing: it is the identity
its cursors are cut from, and it changes when its rows do. Neither is a reason
to poll. wait_ms holds the
answer until the listing is complete (capped at 30 s); add
"after_revision":"…" and it is held until the listing also differs from the
one you have, which is how to wait for an app that was just opened to publish
its actions. A waited answer carries settled: false when the wait ran out
first, with the latest listing.
{"op":"actions_enumerate","query":"read","wait_ms":10000,"after_revision":"3f9c0a17d2b4e861"}
tilo-control actions list read --wait # 5 s; --wait=MS for another bound
A held listing is parked rather than polled. It runs again, through the same path and the same capability checks, only when the action surface moves: an invocation or publication in the broker, the index finishing, a directory page arriving. Each of these wakes the compositor.
Machine-oriented operations can be included explicitly through the same door:
tilo-control actions list so.tilo.browser --programmatic --json
{"op":"actions_enumerate","query":"so.tilo.browser","include_programmatic":true}
This adds programmatic descriptors to bounded enumeration. It keeps the same
workspace, membership and capability checks and does not alter the human palette.
Use the returned provider, ID, contract and parameter schema with actions invoke.
A programmatic page read may carry Write authority; the generic read convenience
command is limited to declared read-only operations.
Workspace Activity’s morning sources use the same actions provider. In
Activity → Explore → Details, select an existing workspace resource and
invoke room.activity.morning.toggle (the Choose for morning / Remove
morning source control). The choice is private to this device and room, with
up to eight resources. Selecting it does not grant access, refresh a provider,
connect an account or acknowledge activity.
Activity → Catch up shows these choices with their latest admitted resource
history and last recorded activity time. The native host separately reads bounded
provider previews through its certified, room-bound action catalogue. Provider
source, observation/fetch time and freshness remain provider-owned data, distinct
from history time; an unavailable preview makes no fresh-data claim. Weather
publishes its existing cached observation, never a fabricated summary.
One chosen source is shown at a time with a bounded visual preview; full
provider text stays available to accessibility. Use the keyboard source pager
or the room-bound room.activity.morning.next.<room>.<key> navigation action
to inspect the remaining choices, including unavailable sources.
Open/remove actions are enumerated for the current room as
room.activity.morning.open.<room>.<key> and
room.activity.morning.remove.<room>.<key>. Native opening invokes the existing
provider’s exact resource open action through the ordinary broker, including
its launch and current permission checks. Hosts without that adapter retain the
history launcher. Missing history evidence is unavailable and can still be removed.
room.activity.morning.refresh.<room>.<key> requests the selected resource’s
currently offered, nullary read-effect refresh. The broker rechecks the real
operation at invocation; status reports a request, not a fabricated refresh result.
room.activity.morning.draft.<room>.<key> imports an unsaved, authority-free
single-resource refresh program into Shortcuts through draft-shortcut on
shortcuts.workbench. Use returned IDs rather than constructing keys. Queued
commands remain room-bound. No catalogue read connects accounts or refreshes data.
In Shortcuts, Daily schedule chooses a daily device-local time on the draft
through its set-schedule action. The existing permission review shows the time
and the exact action grant; only the person’s explicit save uses the existing
owner-checked shortcut library and scheduler. The existing durable schedule ledger,
fire claims and catch-up policy avoid duplicate restart fires. Disable and save
to stop future runs. Importing a draft drops imported triggers, consents and saver
identity; opening the editor creates no user schedule. Real sources/account access
and real schedule creation require the user’s applicable authorization.
The providers on a stock build are actions (the shell’s own chrome),
applications, host-applications, notes, files, records and editor.
The answer is one page of the listing: up to 200 rows unless the request names
a limit (at most 2,000). Every answer says where it is:
{"answer":{"query":"","count":200,"total":431,"truncated":true,
"next_cursor":"200.5f0c3d2a9e81b7c4","actions":[...]}}
While truncated, ask again with cursor set to next_cursor for the next
page; the last page has "next_cursor":null. If the listing changed between
pages, the request is refused with stale_cursor; list again from the start.
"detail":"compact" leaves each row’s contract out, which is most of a row’s
size; params stay, so a compact row can still be invoked:
{"op":"actions_enumerate","query":"","limit":100,"cursor":"200.5f0c3d2a9e81b7c4","detail":"compact"}
A client that predates paging gets the first page, with truncated saying
whether there is more. tilo-control actions list takes --limit ROWS and
--cursor NEXT, asks for compact rows unless --json, and prints the next
page’s cursor on stderr.
An empty query lists every provider’s default set rather than guessing a
needle. The descriptors come from the same provider registry that backs the
Ctrl+K command palette — the palette renders these as rows, enumeration renders
nothing.
actions_invoke
{"op":"actions_invoke","provider":"actions","id":"window.new.terminal"}
{"answer":{"provider":"actions","id":"window.new.terminal",
"kind":"Actions","title":"New terminal window"}}
An address that is unknown, or hidden from the caller, is refused — and the two are deliberately indistinguishable, so probing cannot map what you may not see:
{"err":{"code":"unknown_action",
"message":"no visible action addressed by (actions, nope) in the current catalogue"}}
An application action answers once it is admitted: its receipt carries the
broker job (broker_invocation_id) and usually the state accepted, and
actions_status follows the job from there. wait_ms holds the answer instead
until the job settles — succeeds, fails or is cancelled — or the wait runs out
(capped at 30 s), and answers with the job’s latest receipt either way. The
desktop is woken by the job itself, not polled, so the answer leaves as soon as
it settles:
{"op":"actions_invoke","provider":"app-actions","id":"…","wait_ms":5000}
tilo-control actions invoke app-actions "$ID" --wait # 5 s; --wait=MS for another bound
--wait exits 1 when the action failed or was cancelled, and names the job to
follow with actions status when it is still running. Without wait_ms the
answer is the admission receipt, as before; with device it is ignored.
actions_status takes the same wait_ms (tilo-control actions status ID --wait), so following a job is one held request rather than a loop of reads.
tilo-control read uses both waits: one listing and one invocation, and no
sleeps.
A client making many calls can keep one connection: tilo-control actions session reads one wire request per stdin line and writes one answer per stdout
line, in order, over a single connection.
windows and state_dump
windows lists the open windows, one row per tab, read straight off the live
tree:
{"op":"windows"}
{"answer":{"windows":[
{"workspace":"Home","strip":"Main","group":100,"tab":1000,"kind":"Foreign",
"app_id":"so.tilo.notes","title":"Notes","active":true}
]}}
active marks the tab in front of the active workspace’s active group. A
workspace or strip without a name is named by its id. tilo-control windows
asks for this op, and reads the state dump instead when a compositor predates
it.
state_dump answers workspace_tree, chrome, peers and sharing. The
tree is the desktop as it is now. A workspace’s append-only history is left
out: effect_receipts, ownership_transfer_audit,
ownership_transfer_cancellations and each task’s audit. That history grows
with every action ever taken, so the dump stays the same size however long the
desktop has been in use. Ask for the history explicitly when you need it:
{"op":"state_dump","include_history":true}
tilo-control state-dump --history --pretty
Polling windows or the plain dump after each action is cheap. A dump with
history costs time on the compositor’s thread in proportion to that history.
Reading a terminal session
For Terminal, tilo-control read so.tilo.terminal --json returns the shared
session target, screen and cursor, with an explicit truncation flag when output
exceeds 64 KiB. That record can outlive the process or describe a collaborator’s
screen. Discover and invoke the read-only Inspect session action for this
participant’s actual lifecycle, attached process identity, input availability
and startup cause. Neither getter opens, restarts or grants a session. A command
is unavailable until the local process has opened; sending to a placeholder
writer cannot report success. The device’s own shell continues to require the
person, device ownership and an explicit grant; agents use managed environments.
Why enumeration and invocation are separate steps
They answer different questions and carry different risk. “What can I do here” must be free to ask; “do this” must not be.
Collapsing them into one call — a search that runs its best match — makes every query a potential side effect, so a caller exploring the surface cannot explore it safely. Keeping them apart means a caller can inspect what an address resolves to, reason about it, and only then commit. It also makes the failure modes distinguishable: “that action does not exist for you” and “I ran it and nothing happened” must not look alike to a caller with no way to check the screen.
Unlocked native desktop actions list --json includes completion alongside count and
actions. The top-level complete and completion.complete are true only when every
selected provider exhausted its bounded traversal and has no outstanding work or
failed reply. truncated and next_cursor still describe paging of the returned
catalogue, independently of whether the source catalogue is complete.
An empty actions array with complete: false is not proof that the application
publishes nothing. completion.incomplete names the provider and the observed
reason (pending, refused, unavailable, timed_out, stale_cursor,
budget_exhausted, continuation, or invalid_input); it exposes no document contents or
additional authority. Older transports without this field do not establish
completion. Keep any returned actions, but require completion before treating an
empty listing as authoritative or using a listing for readiness acceptance.
Invocation re-resolves the address against a freshly built catalogue before
anything runs, so an address that was valid at enumeration time and is no longer
valid now fails loudly instead of running something else. The request runs from
that one resolution: nothing else runs between resolving the address and
invoking it. An application’s directory action is admitted bound to the
directory revision it was resolved at, which the invocation journal records;
the host that runs it refuses (published_invocation_revision_changed) when
the app’s directory moved on before it ran, rather than acting on something
other than what was resolved. A remote caller that names no revision is bound
to the one the app resolved at, except for an action whose contract asks for
confirmation: that one needs the revision it was shown at.
Why enumeration carries no invoke handle
The descriptors are inert data: provider, id, kind, title, subtitle,
score. There is no callback, no token, no handle. Read-only is a property of
the reply’s shape, not a promise in a comment — there is nothing in an
enumeration result that could be fired, so surveying the surface cannot cause an
effect even by mistake.
An action is therefore addressed by a stable (provider, id) pair and never by
a screen position. That is what makes the surface drivable by something that
cannot see, and what makes an invocation reproducible across window layouts,
resolutions and themes.
The visibility gate
ActionCatalog (crates/session/shell/src/shell/search/action_api.rs) applies an
ActionVisibility gate when it is built, and keeps the result as an index list
of the entries that passed. Every public accessor — enumerate, select,
invoke — reads through that same list.
The consequence is the important part: a denied action is absent, not
disabled. It does not appear in enumeration, and select on its address
returns nothing, so actions_invoke refuses with unknown_action. A caller
cannot invoke what it was never shown, and cannot learn that a hidden action
exists by trying to run it. Contrast the usual arrangement, where a filtered
list and an unfiltered dispatcher disagree and the list is merely advisory.
The implementation the socket passes is CapabilityVisibility
(crates/session/shell/src/shell/search/visibility.rs), which resolves the active workspace,
builds the actor for the local user, and asks that workspace’s own capability
model. It is fail-closed twice over: decide_capability denies unless something
explicitly permits, and a session that cannot resolve a workspace at all errors
rather than falling back to showing everything. A gate that opens when it cannot
find its policy is not a gate.
What is gated, and what is not
The rule is the one crates/session/shell/src/shell/actions.rs already states, not a new one:
showing a resource is non-destructive and needs no permission; taking an action
on one requires permission. So notes, records, files and folders are never
hidden, and neither are the actions that merely open an overlay —
settings.open reveals a panel, it does not change a setting. What is gated is
acting: Write for spawning a terminal, duplicating a tab or launching an app,
Invite for peer operations. An action no mapping covers is treated as an act
requiring Write, so a provider that lands without one goes dark for a
restricted caller instead of being quietly reachable.
The effect is observable on a live session. Against a workspace the local user
owns, actions_enumerate returns the full surface — 191 entries on a stock
build. Against a workspace owned by someone else, the same call returns only
what anyone may see: the overlay-opening actions and the documents. No error, no
empty list, no hint that the rest exists.
Pointing Claude Code at it
scripts/tilo-mcp-server.py is an MCP stdio server exposing the typed Tilo
tools and a lazy resource tree. resources/list returns only application
surfaces at tilo://application/<app>; it does not put every action schema into
the model’s initial context. Reading one application returns the objects it
currently publishes with each object’s full action descriptors nested beneath
it.
This nesting is the resource/action model rather than an MCP-only taxonomy: an application owns resources (the object a person would recognise, such as a file or document), and each resource owns the operations valid for it now. Shell palette providers that do not publish a resource tree remain reachable as one provider application with one invokable operation per object; the shim does not infer a fictional hierarchy from dots in their display ids.
Every level re-enumerates through the visibility gate before answering. A closed object or revoked action therefore disappears from its parent and a stale URI fails as unknown; nesting changes when descriptors enter context, not what the caller is allowed to reach. Standard library only, no SDK.
Start a compositor, then register the shim:
export TILO_TEST_CONTROL=$XDG_RUNTIME_DIR/tilo-session/test-control.sock
cargo run --bin shell -- --backend headless &
# Verify the shim reaches the compositor before putting a model behind it:
scripts/tilo-mcp-server.py --selftest
claude mcp add tilo -- /absolute/path/to/scripts/tilo-mcp-server.py
The socket path comes from $TILO_TEST_CONTROL, or --socket. The shim never
fabricates an action list or a success: if the compositor is unreachable, or
answers {"err": ...}, the model receives that as a tool error with the
compositor’s own code and message intact.
Apps that run before the compositor
The installer and the greeter run under cage straight from greetd, so no
compositor socket reaches them. They serve the same three semantic ops
themselves — actions_enumerate, actions_invoke, state_dump — through
crates/session/console-control, so tilo-control --socket <path> actions list drives them unchanged:
- on the machine,
$XDG_RUNTIME_DIR/tilo-console/<app>.sock(0700 directory, 0600 socket) for the local console and assistive tooling of that user; - in a VM, the virtio-serial port
org.tilo.console.0, which exists only when the hypervisor attaches it (os/dev/vm.sh consoleprints its host end). Physical machines have no such bus; nothing listens on a network.
Action ids there are <action>@<resource> (next@installer,
[email protected]). A contract that asks for confirmation takes it as
an argument, the same projection the compositor publishes for app actions:
confirm: true for Confirm, confirmation_phrase for a typed Phrase.
An action tagged confirmation:by-argument (a framework app’s generic apply
edit) advertises an optional confirm: the app classifies each value, and a
destructive one — deleting a clip — is refused with
action_confirmation_required until it is resent with confirm: true.
Input injection is not served; the surface is semantic only.
tilo-vm scenario install drives the installer this way end to end
(tilo-vm guide scenarios).
The greeter publishes sign-in@greeter, account and session selection, and
the available fingerprint action. Sign-in uses the existing greetd/PAM
conversation and the selected installed session; its password parameter
is Secret. Use actions invoke greeter sign-in@greeter --args-stdin with
a private JSON object on standard input, rather than placing a credential
in the command line. An accepted invocation means the attempt started;
state-dump reports authentication progress and the mapped result.
For a bounded timing measurement, configure-session-diagnostics@greeter
configures this greeter’s next ordinary sign-in with four required booleans:
presentation, gpu_stages, gpu_timestamps, gpu_dispatch_timestamps.
Dispatch timestamps require GPU timestamps. The selection is transient and
inspectable as session_diagnostics; a fresh greeter starts without a selection,
and normal diagnostic flags are off unless explicitly configured by the host.
Set all four false to disable them for this sign-in. Authentication in progress
refuses changes. These are fixed content-free timing switches, never arbitrary
environment, session argv, output paths or input gates. Invoke normal sign-in
through private stdin afterward; PAM and the installed session remain authoritative.
No measurement qualifies without the matching actual input, body revision and
hardware presentation proof; GPU-feature support must be observed separately.
If PAM requires another answer, the current challenge has a unique ID and
a secret or visible response kind, and its bounded normal PAM question
with literal echoes of private answers removed. respond-challenge@greeter takes that
ID as challenge and a private Secret argument named response. Stale
IDs cannot answer a later prompt. cancel-authentication@greeter sends the
ordinary greetd cancellation request and keeps the attempt busy until its
worker finishes; no cancelled response can start a session. Credentials
and raw PAM diagnostics never enter resource values, encoded operations,
state dumps, action replies or logs.
Carrying one action result into the next
An action sequence is a data path, not just a list of commands. The broker’s
terminal Succeeded update contains the declared ActionOutput; the agent
retains that output for exactly the immediately following step. Content bytes
are represented by one canonical resource address, tilo.content:<hash>. An
action built with prompted_content consumes that hash from the preceding
application result and replaces any value the model tried to provide.
Local collaborative-document actions use the same rule through the owning application’s declared product function. The agent does not inspect a document and guess which bytes it represents. Missing content, the wrong resource kind, or more than one content result stops the sequence rather than selecting one.
This hand-off is deliberately not a workflow language: there are no named variables, JSONPath expressions, graphs or implicit reuse of an older result. The narrow one-step hand-off is enough to express a cross-app operation such as “rotate the photograph, then place the result in the document”, while keeping the origin of the content mechanically checkable.
Saved, multi-step pipelines are shortcuts (crates/collab/automation,
tilo-automation): named steps with typed bindings between them, checked at
save time against the same descriptors this API enumerates and run through
the same ActionHost door as the invoking person, inside the shortcut’s own
grant.
AI transforms
The workspace agent (so.tilo.agent, running with --task-runs) publishes
one programmatic resource, agent.transform:default, whose actions any caller
invokes like any other app’s: summarise, extract, classify and
freeform. A shortcut’s Ai step lowers to exactly these.
Each takes its text as input, or as a tilo.content:<hash> address in
resource, and declares a JSON result: text for summarise (bounded by
length), a label (or with multi a list of labels) for classify, a value
satisfying the caller’s own schema for extract, and text or a value of the
caller’s schema for freeform. The agent validates the model’s answer
against that shape before returning it; an answer that misses gets one repair
attempt, then fails with invalid_model_output rather than returning a guess.
Other failure codes are invalid_arguments, input_too_large,
content_unavailable, model_unavailable, timed_out and busy.
The caller never reaches a model. The agent asks the model gateway for one
bounded single-shot answer (Request::Transform, admitted only for
so.tilo.agent); the gateway builds the provider request, caps input, output
tokens and deadline, and returns only the answer text. A shortcut cannot feed
a secret to a transform: the checker refuses the binding and the interpreter
refuses it again at run time.
Source of truth
| What | Where |
|---|---|
| Command + response shapes | crates/session/shell/src/compositor/test_control.rs |
| Device admin’s control socket | crates/session/shell/src/compositor/admin_control.rs |
CLI (tilo-control) |
crates/session/control-cli |
| Launcher coverage proof | crates/session/shell/src/shell/overlay_host/launcher_action_coverage_tests.rs |
| Live enumerate / resolve | crates/session/shell/src/compositor/native.rs |
| Catalogue + visibility seam | crates/session/shell/src/shell/search/action_api.rs |
| Capability gate + policy | crates/session/shell/src/shell/search/visibility.rs |
| Live end-to-end proof | scripts/headless-examples/action-api-demo.py |
| Cross-app result hand-off proof | apps/agent/tests/discovered_operations.rs |
| Shortcut pipelines (check + run) | crates/collab/automation |
Keyboard shortcuts (keyboard-shortcuts: list, rebind, reset) |
crates/session/shell/src/shell/keyboard_shortcuts.rs, catalogue in crates/system/settings-store/src/keyboard_shortcuts.rs |
| AI transforms | apps/agent/src/transform.rs |
| MCP shim | scripts/tilo-mcp-server.py |
| Console apps (installer, greeter) | crates/session/console-control |
| Socket framing, other ops | docs/headless-testing.md |
Unlocking the device-owner session
The existing device-owner admin-control.sock publishes the session-lock
resource while locked. Invoke unlock@session-lock with a Secret password,
using tilo-control actions invoke session-lock unlock@session-lock --args-stdin.
Supply the JSON object through private stdin; never place a credential in argv.
The reply acknowledges asynchronous verification, rather than claiming unlock.
Wait for the ordinary state-dump chrome.session_locked transition and
chrome.lock_auth.verifying/non-leaky error fields. The normal PAM helper
verifies the existing session owner; no caller may replace the username.
Agent-scoped and enrolled-device actions do not receive this local capability.
Other app actions remain refused while locked.
This device’s shell
A live launcher-authenticated Terminal exposes owner operations through the
Shell’s terminal-device-shell provider. List device shell to obtain the
exact per-connection addresses. Invoke request@<connection> with
confirm: true to select the ordinary Host target. This does not grant it:
answer its existing permission request using permission-prompts, then use
inspect@<connection> to obtain the current local session UUID.
run@<connection> requires that exact session, a single reviewed command
(up to 8 KiB), and confirm: true. Its acknowledgement says submitted only
after the existing PTY accepted the bytes; it does not claim an exit status.
read@<connection> requires the same session UUID and returns at most 32 KiB
of visible output, including retained output after the process exits. Read
and inspect expose no clipboard, credential or arbitrary host-file API.
Only a person administrator/owner may use this provider. App, agent and token principals remain refused even with an Admin override. Generic app-broker Terminal operations cannot open Host or send input to an existing Host PTY; Default environments retain their normal app actions. Closed connections, changed sessions, missing acknowledgement and unavailable PTYs fail visibly. The matched Shell, SDK runtime and Terminal versions are required for this owner lifecycle operation; unsupported workers do not report success.
Workspace membership and invitations
The actions provider publishes workspace membership under the canonical ids
and parameter names of tilo_action_broker::collaboration_admin, acting on the
active workspace:
| Action | Parameters | Output |
|---|---|---|
member.role.set |
participant_id, role (viewer, commenter or editor) |
none |
member.remove |
participant_id |
none |
invitation.issue |
role, expires_in_seconds (1 to 30 days), optional invitee_name, optional recipient |
{"invite": token} |
invitation.list |
none | {"invitations": [{invite_id, name, role, expires_at}]} |
invitation.resend |
invite_id |
{"invite": token} |
invitation.cancel |
invite_id |
{"canceled": true} |
Each also takes confirm: true; the contracts are privileged, sensitive or
destructive and require Admin on the workspace. participant_id is the
member’s person id as the roster reports it. Setting a role only changes an
existing member’s role; admitting someone new is an invitation. recipient is
the invitee’s 64-hex person key (or a tilo-identity: card); with it, only
that person’s devices can redeem the token, which is otherwise a bearer
credential. The invitation actions need peer sync and complete on a later
frame, so callers poll the pending invocation. The Members tab in workspace
settings calls the same functions as member.role.set and member.remove, so
a click and an invocation pass the same authority check.
Reviewing a Tilo package in Store
Store accepts .tapp files through the normal file-open handoff, its shared
“Review package” file chooser, and the declared import-package action. Discover
Store’s current bound resource/action IDs before invoking them; resource IDs are
session-bound. The action accepts either a sandbox-readable path or a handed
content hash, never both. Omitting both opens the ordinary file chooser.
Opening or reviewing a file does not install it. The optional reported_size
is a preflight refusal hint for oversized handoffs; it never replaces independent
byte bounds, archive verification or the repository offer size.
Full and thin archives use the same review: every carried payload is hash-checked, then signed metadata is matched to an independently verified configured repository offer (app, version, root and publisher). The independently authenticated offer size is retained for the installer’s exact download-size check; full and thin local files can encode the same identity at different lengths. An archive’s own key never creates publisher trust. Unlisted packages require independently provisioned identity-scoped trust, as described below; component and runtime archives use the system updater.
After the review job succeeds, inspect-catalogue exposes imported_package
with its app, root and file version. The app details show that version, its signed
identity and whether the app was already installed. Invoke the separate
install-imported-package action with that exact app and root, or activate
“Install reviewed package” in those details. Store rechecks the current offer
before passing it to the existing authenticated InstallOffer installer. The
installer obtains the canonical offer and validates its runtime and device target;
reviewing a full file does not bypass fetching or authorize an unavailable runtime.
Poll each numeric broker job to a terminal outcome, then separately inspect the
selected app’s installed state before invoking its ordinary Open action. A
pre-existing installed app is an update/reuse test, not a first installation.
Trusted local .tapp files in Store
Store’s existing Review package and Install reviewed package actions also
accept unlisted local app packages when the user’s host-side
~/.config/tilo/trusted-attestors.toml already authorizes a valid scoped
attestation. The host checks revoked-keys.toml before honoring that attestation.
Opening the file never grants publisher trust or installs anything. Valid
self-signatures alone are refused on this Store path, even though the standalone
installer’s default policy can warn and continue.
The existing launcher-issued AppHost capability reviews handed bytes without receiving a publisher key or host filesystem path from the worker. Installation rechecks the exact reviewed archive digest and signed root against current host trust, then calls the canonical local installer with its verified attestor requirement. A publisher change is refused; normal pinned/held selection and runtime/interface policies remain in force. A thin local file needs its missing payload already in the canonical content store; Store does not invent an untrusted download source. The review reports actual installed state separately from the candidate, and successful installation requires the reviewed root to be the selected installed package.
Timing diagnostics in an active owner session
The authenticated local owner socket publishes session-diagnostics with
configure@session-diagnostics and inspect@session-diagnostics. Configure
requires exactly four booleans: presentation, gpu_stages, gpu_timestamps,
and gpu_dispatch_timestamps. Dispatch timestamps require GPU timestamps.
Use the ordinary tilo-control actions invoke argument JSON; no environment
names or output paths are accepted.
The setting is ephemeral to this desktop session. It sets or removes only the
four existing diagnostic variables for subsequently launched children.
Presentation also controls the compositor’s existing physical input delivery
observer immediately. Existing applications and the compositor’s initialized
GPU renderer retain their configuration; reopen the application to collect
its new timing stream. Inspect returns the current fixed options and their
scope, plus launch_history: at most 64 launch attempts with at most 12 events
each, original epoch timestamps and monotonic elapsed times. It joins the
immutable launch ID to its saved tab/record/workspace, launcher PID and kernel
start time, original package hash, mapped surface, exit status and executing
process-binding phases. App stderr is fully redacted; only its bounded retained
line count and a fixed failure category are reported. Runtime cohort and shell
source still require the matched component receipt and signed pins; missing
evidence is not guessed. Eviction counters disclose lost history.
For one app, use the optional app_id argument. These actions require the
existing authenticated owner socket, not the test-control socket:
tilo-control --socket "$XDG_RUNTIME_DIR/tilo-session/admin-control.sock" actions invoke session-diagnostics inspect@session-diagnostics --args-json '{"app_id":"so.tilo.browser"}'
The history is captured directly by the compositor, not reconstructed from
journal or flight-recorder replay. Inspect does not launch an app, replay an
action or change diagnostic configuration. An unsupported connection receives
not_permitted explaining the required owner socket. Agent-scoped, sandboxed, other-account and locked-session calls cannot
configure this owner-only surface. No application content or typed keys are
recorded by these timing options.
Cloud account mounts
Settings publishes sync-now and either pause-sync or resume-sync on each
settings.account:<id> row while the cloud account service lists that account
as mounted at ~/Remote/<label>; an account that is not mounted offers none of
them. They are device-only jobs: the effect answers once the service has
accepted the request, and the account’s health still arrives through the
account status store. sync-now cuts a failure back-off short and runs a pass
(an account whose mount could not start is tried again); a pause stops
background sync and uploads until resumed, survives restarts, and leaves
on-demand reads working while edits queue.
Settings reaches the mounts through the service’s account socket,
$XDG_RUNTIME_DIR/tilo-fs-cloud/control.sock (runtime_fs::cloud, mode
0600), whose directory its sandbox is granted. That socket lists, syncs,
pauses and resumes mounts and refuses everything else. File requests (a
file’s provider revisions, restoring one of them, and a share link, each
refused unless the provider’s capabilities include it) are served only on the
file socket, $XDG_RUNTIME_DIR/tilo-fs-cloud-files/files.sock, in a directory
of its own that no grant of the account socket reaches. No credential crosses
either socket. Files does not yet publish these file requests as entry
actions, and no sandbox is granted the file socket yet.
Operating a portal file chooser
When an application requests the normal portal chooser, the authenticated owner
can discover its file-picker actions on the same control socket. Each opaque
action ID includes the current request UUID. Inspect returns the requesting
application, mode, a bounded page of entries, and a state token. Selection and
navigation accept only entry IDs from that actual listing, together with its
latest state token; they do not accept arbitrary paths. Inspect again after an
operation before using the returned state for the next one.
Confirming Open/Save and approving a replacement require confirm: true.
Cancellation uses the same request-bound action. The original application’s
portal request resolves through the ordinary chooser/grant path. A successful
chooser receipt is not an application install receipt: track the original
application job and inspect its reviewed state separately.
The chooser refuses expired requests, stale selections, entries escaping its permitted roots, unsupported mode operations, and queued operations whose workspace owner or lock state changed. Agent, app and bearer-token principals cannot approve personal file access through an Admin override. Staging a file in Downloads does not grant an application access to it.
This page is docs/agent-action-api.md in the Tilo repository, at 29fbb80121. Changes to it are made there.