Operating agents

The procedures for an agent that is connected to Caldera: install, update, pin, roll back, pause and resume with and without Caldera, update the runtime, rotate a token, delete, move an existing agent onto the runtime, and a troubleshooting table from symptom to fix.

Conventions: <dir> is the agent's directory on its machine (for example ~/caldera-agents/my-agent), <name> its name. Commands on the machine are python3 caldera.pyz ...; the download sits wherever it was fetched, and the runtime keeps a copy at <dir>/.caldera/caldera.pyz.

Quick reference

Install an agent from a blueprint

Needs: an admin role in an activated workspace; a published version; a machine with Python 3 and (for the service) systemd; outbound HTTPS to app.calderaapp.io.

1. In Caldera: Blueprints, the blueprint (or a catalog one), "Install...". Pick the version, give the agent name (lowercase letters, digits, hyphens; not an existing agent's name), press "Create install code". If a binding's product is not activated, or the workspace would exceed ten bots, the dialog says so now.
2. Copy both commands now. The code is shown once and expires in 15 minutes.
3. On the machine, run the first command (fetch and verify). On macOS use the variant with shasum -a 256 -c -. If the check fails, stop; see the troubleshooting table.
4. Run the second command: python3 caldera.pyz init <code> --url https://app.calderaapp.io. Add --dir <dir> to choose the directory (default ~/caldera-agents/<name>). It asks for the declared secrets (hidden input); for unattended setups set CALDERA_SECRET_<NAME> instead.
5. It installs and starts caldera-<name>.service. For it to keep running when you are logged out: loginctl enable-linger <user>.
6. Check: python3 caldera.pyz status --dir <dir> shows process running and last report ok, and the agent appears in the overview within about a minute.
7. Do the independence check.

If init fails after the code was spent, the agent exists in Caldera without a machine. Delete it in Caldera and issue a new code.

Register an agent with an own reporter

1. Overview, "New agent": a name and the reporting interval if another is wanted.
2. Copy the token and the two environment lines now; they are shown once.
3. Put them in the reporter's private secrets file; start the reporter; check the overview. See Connecting an agent.

Update to a new version

1. Raise version in the manifest to a number that has never been published (published versions are immutable, and a draft may be replaced only until it is published), rebuild the bundle, upload it as a version of the blueprint and publish it.
2. On the agent's page, Blueprint panel: choose the version, "Move to this version". Caldera stores the wish and sends it in the answer to the agent's next report.
3. Follow the line under the panel. It goes: waiting for the agent's next report, then applied, rolled back by the agent, or failed, with the agent's own message.
4. After an applied update, the agent watches itself for 30 minutes. If its level gets worse in that time, it rolls back by itself, and the panel then shows the rolled-back outcome.

What the runtime does is in the runtime. In short: fetch, stage beside the running version, swap, restart, run health, keep or go back. Everything in state paths is untouched.

A version the agent rolled back from or refused is not applied again until a different version is named. To try a fixed version, publish a new version number.

Pin

Pinning asks the agent to stay on the version it runs: on the agent's page press "Pin". Technically it sets the wished version to the running one, which the agent sees as nothing to do. It also stops a stale wish for a newer version from being applied. "Pin" is disabled while the wish already equals the running version.

Roll back

• From Caldera: "Roll back" names the previous published version. Use this when the agent is healthy but the old behaviour is wanted.
• On the machine, without Caldera: python3 caldera.pyz rollback --dir <dir>. It swaps to the previous version and restarts, with no network. Only two versions are kept, so a second rollback goes forward again.
• After a local rollback the version it left is on hold: the next report will not undo it, even if Caldera still wishes that version. status shows held <version> is not applied until Caldera names another version. Then pin (name the version that runs) or name another version, so Caldera's page agrees.
• A catalog agent can only fetch the one shipped version, so for it the local rollback is the way back.

The hold clears when a different version is applied successfully.

Change a setting, run now, ask for a log

• Settings (admins): Settings tab, change, "Apply". The agent checks every change against its own bounds and answers in its own words; "A change is waiting for the next report of the agent" is shown until it does. A refusal reads "Nothing changed: : ." and changes nothing (all or nothing). Caldera repeats the bounds check only to tell a person at once.
• Run now: needs the agent to offer run_now in controls; refused while paused.
• Log: admins only. Caldera asks the agent for one of the log names it reported; the agent sends the last 512 KB. The text is kept 30 days with the command.

A command is delivered at most once. If a report's answer is lost, the command is lost and fails after ten minutes (when the agent next reports). Send it again.

Pause and resume, with and without Caldera

Everything Caldera does can be done without it. Know the offline way for every agent before it is needed.

• A pause set in Caldera is the same pause: the same file and the same stop.
• For a runtime agent, pause stops the agent's process; the runtime keeps reporting, and the state shows "Paused" (warn).
• Caldera disables the controls for a silent agent ("The agent is not reachable, so it cannot take a command now").
• A runtime on a remote machine is paused by logging in to it and running the command there. Write the exact command down next to the agent's documentation.

Update the runtime itself

The runtime updates itself on a press, signed (how it works is in the runtime):

1. On the agent's page, the Runtime panel shows the version the agent runs, the one this Caldera serves, a pin and the file kept before. "Update runtime" (admins only; members read) sets a wish. The next answer to a report carries it. The panel follows it: waiting for the agent's report, "has to prove itself first" while the new runtime runs its five minutes, then the runtime's own words for taken, gone back or not taken. A version that went back or failed is not offered again until Caldera serves another one, and the button is gone until then. A runtime that reports no version (older than this feature, or not caldera.pyz at all) gets a notice instead of a button: it is replaced by hand once, below.
2. The runtime downloads /cli/caldera.pyz and its signature, checks the checksum and the signature against the public key built into it, and lets the new file say its version. A bad or missing signature, or a machine without ssh-keygen: nothing changes and the agent page shows the outcome failed with the reason.
3. It keeps the old file as <dir>/.caldera/caldera.pyz.previous, swaps the file the unit names, and restarts in place: same process, the agent's own processes never stop.
4. The first successful report within five minutes makes it good. A new runtime that crashes, or does not report in time, goes back by itself; that version is then held until a newer one is offered.

On the machine, with no connection at all:

python3 <dir>/.caldera/caldera.pyz rollback-runtime --dir <dir>   # back to the previous file
python3 <dir>/.caldera/caldera.pyz pin-runtime 0.32.0 --dir <dir>  # never update to another version
python3 <dir>/.caldera/caldera.pyz pin-runtime off --dir <dir>     # release the pin

One-time catch: a runtime installed before this existed cannot update itself. Replace <dir>/.caldera/caldera.pyz once by hand (mode 700, checksum verified) and run systemctl --user restart caldera-<name>.service; the unit has KillMode=process, so the agent keeps running. After that, updates are presses. A runtime started from anywhere but <dir>/.caldera/caldera.pyz (the unit's path) cannot replace itself and says so.

Which Claude account an agent runs under

The Runtime panel on the agent's page says which Claude account the machine is signed in to. See The Claude account hint.

Rotate an agent's token

Do this when a token may have leaked, or was lost.

1. Caldera: agent menu, "Rotate token...". The old token stops working at once. The new one is shown once, with the two environment lines.
2. The agent is now silent in Caldera until the machine has the new token.
3. Runtime agent: edit <dir>/.caldera/agent.env, replace the value of CALDERA_TOKEN (keep the file mode 600), then systemctl --user restart caldera-<name>.service. The agent keeps running: it never held the token, and a restarted runtime adopts it.
4. Own reporter: update its secrets file and restart the reporter service.
5. Check that the agent reports again.

The runtime has no command that fetches the new token; Caldera never writes to the machine.

Binding tokens (access to other products) are not rotated here. They are the bot's tokens, handled in the product's own bot settings; see the table below ("binding is broken").

Delete an agent

Caldera: agent menu, "Delete...". The confirmation names what stays.

• Caldera deletes the agent, its runs and its commands. Its token stops working at once.
• Nothing on the machine changes. The runtime keeps running the agent and logs a refused report (HTTP 401) every interval.
• The bots are not revoked. Only the product that owns a bot can revoke it. The confirmation lists them by name ("These bots stay: ..."). Revoke them in that product's bot settings, or the tokens stay valid until they expire.
• To remove the agent from the machine, in this order:
  1. python3 caldera.pyz pause --dir <dir> (this stops the agent's process; disabling the unit first would leave it running),
  2. systemctl --user disable --now caldera-<name>.service,
  3. remove ~/.config/systemd/user/caldera-<name>.service and systemctl --user daemon-reload,
  4. remove <dir> (it holds the agent's state; keep it if it is wanted).

A blueprint can only be deleted when no agent is installed from it.

Move an existing agent onto the runtime

It works when the agent can run as one process the runtime may start and stop. (An agent with its own timers and a pause that other things set uses adapter mode, or stays on its own reporter.)

Before: the agent has an own reporter and its own timers. Nothing changes until the steps start.

1. Build the bundle and upload it as the first version of a blueprint (see Blueprints); publish.
2. Rename the old agent in Caldera (for example <name>-old): the install needs the name to be free.
3. Issue an install code for the blueprint, with the agent's name.
4. On the machine, stop the old mode first, so nothing runs twice: disable the old timers and the old reporter service.
5. Run the two install commands; init writes the directory and the unit.
6. Check: status, the unit's journal (journalctl --user -u caldera-<name>.service -n 20), the agent in Caldera, "Run now" once, and the independence check.
7. When it runs, delete the old agent in Caldera. Leave the old unit files in place: the way back needs them.

The way back: disable the new unit and start the old mode again.

systemctl --user disable --now caldera-<name>.service
systemctl --user enable --now <the old timers and the old reporter service>

In Caldera rename the old agent back and delete the new one. The old and the new mode must never run at the same time.

The quota gauge looks wrong

Symptom. An agent's quota gauge in Caldera does not match the provider's usage page. For example Caldera shows 79 % for the week while the provider shows 90 %.

Cause. This is a property of how the agent measures, not of Caldera. A gauge that an agent builds from the quota report of its own last model call is only as fresh as that call. Two things make it stale:

1. Other sessions use the same quota. The subscription is shared with interactive sessions on the same account, so the real figure can be higher than the last one the agent saw.
2. The agent holds runs above its threshold. Above its own limits the agent starts no run, and without a run there is no new observation. The gauge then stays where it was.

A window whose reset time has passed counts as empty (0), so a stale gauge can also fall to 0 at a reset the agent has not seen a call after.

What helps. An agent that builds a gauge from its own last observation should

• put when it observed into the gauge: the first words of the gauge's detail are the time of the last measurement (for example "As of 06.10. 14:05"), so that a stale value is recognisable as stale;
• refresh the observation by itself, at most once an hour, also while it is paused or held, with a request that asks for nothing but the quota (no context, no tools, the smallest model); and say in its log when that fails and that the last value stays.

There is no central quota gate in Caldera and none is planned: every agent holds its own limits.

Check. Read the time in the gauge's detail line. If it is older than about an hour, the measurement fails: look in the agent's own log, and check that the program that makes the measurement can be found. Compare the figure with the provider's usage page.

Fix. Caldera shows what the agent reported; there is nothing to change in Caldera. Do not treat a gauge without a measurement time as live.

Troubleshooting

Start with the agent's page (level, last report, "Process" and "Binding" health items) and, on the machine, python3 caldera.pyz status --dir <dir>.

Refusal reasons

Caldera's refusals carry a reason (and params for the sentence); the app writes the sentence in the reader's language. The server's own message is English prose for logs.