Lesson 14 · 24 in the route · 7 min read · Level 3
Presence and queue
enjambre watch knows which nodes are ready; jobs for a downed node wait in a queue.
Page 12 is the 3D and studio one; pages 13–14 are reserved for sessions B and C of
prompts/09-sesiones-2026-10-04.md.
Goal
python enjambre.py status tells you, without you asking by hand, which nodes can receive work right now.
In addition, an enjambre run against a powered-off node does not fail: the job waits in the queue and goes out
on its own when the node comes back.
Why
Up to here, enjambre run assumed the Mac was ready. If it was not, the job failed
and you had to remember to relaunch it. For the swarm to work without you, two pieces are needed:
-
Knowing who is alive. “Alive” has levels, because each one fails for different reasons:
Level What it means What fails if it does not arrive downNot even ping The machine is off or off the network pingAnswers ping, but SSH does not get in sshd or the key sshSSH gets in, but there is no tmux -L enjambresessionNobody logged in on the screen, so there is no keychain (step 11) readySSH and the session with keychain Nothing: it can be dispatched -
A queue. A job requested for a node that is not
readywaits its turn instead of getting lost. See the concept card job queue.
Polling or heartbeat. Windows asks every 30 s; the nodes do not report in. The full comparison and the reason are in the card polling vs heartbeat. In one line: polling measures exactly the path a job uses (ping → SSH → tmux), and does not require opening any port on Windows.
Steps
1. enjambre watch
PS> python enjambre.py watch --once # una ronda y sale (para probar)
PS> python enjambre.py status
nodes (watch, 0 s ago)
mac ready since 2026-10-04T14:09:44+00:00
queue 0 waiting
Each round makes a single SSH call per node: tmux -L enjambre has-session -t enjambre. That is
enough for two levels: code 0 is ready, 1 is ssh (it got in, but there is no session), and
255 means SSH did not get in, so ping is tried. The real round against the Mac took
0.46 s.
What it writes, in state/ (ignored by git):
| File | What |
|---|---|
nodes.json | Level of each node, since when and last check. It is written to a temporary file and renamed, so a reader never sees half a file |
watch.log | Startups, level changes and dispatches. It rotates at 512 KB and keeps 3 copies |
queue/, running/, done/ | One folder per job, which moves between the three. Nothing is ever deleted |
watch.pid, watch.lock | For --stop and so there are never two instances |
Debounce: dropping a level requires two observations in a row. A single lost reply does not trigger the “mac down” notification. Going up a level, on the other hand, is immediate.
Windows notification only when the level changes: “mac available” or “mac down”. The
first round after starting watch records the level, but does not notify.
2. The queue
(capture of the real setup: not published; see docs/CAPTURAS.md for the demo-mode ones)
Render of the real output with freeze. The “no tmux” state was simulated by writing nodes.json;
the poll that brought it back to ready was real.
What happens inside:
- The contract is validated before queueing. The restricted-repos rule, the
AGENTS.mdand the task sections are checked just as if it ran now. An invalid job never enters the queue. - A copy of the task is saved (
state/queue/<id>/TASK-<id>.md) and of theENJAMBRE_*variables. The job runs what you asked for, even if you edit the file afterwards. - There are never two jobs with the same id waiting or running at once.
- The ledger is written when the job runs, not when it is queued. A line in
jobs.jsonlstill means “this happened”. watchdispatches each job as a separate process (enjambre run --no-queue), respectingmax_jobsfromnodes.toml. A long job does not hold up the polling.- If
nodes.jsonis stale (more than 90 s, becausewatchis not running),rundoes not trust it and asks the node at that moment.
3. Automatic startup and a single instance
PS> powershell -File tools\install-watch-task.ps1
TaskPath TaskName State
\enjambre\ watch Running
PS> python enjambre.py watch
watch: another instance is running; not starting a second one
PS> python enjambre.py watch --stop
watch: stopped pid 38332
- The scheduled task
\enjambre\watchstarts when you log in, withpythonw.exeso that there is no window. It was registered without administrator permissions. - Against duplicates there are two layers: the task has MultipleInstances = IgnoreNew, and
watchtakes a file lock that the system releases if the process dies. A stale pid file never blocks the next startup. --stopkills by the pid thatwatchitself wrote, never by process name.
4. The Mac side: is the orchestrator there?
node-watch.sh runs every 60 s (LaunchAgent dev.enjambre.nodewatch, template in config/):
- It checks whether the Deskflow client has an
ESTABLISHEDconnection with Windows (netstat, without sending anything); if not, aping. (Until October 4 it opened a TCP connection to 24800 withnc -z, and Windows loggedfailed to accept secure socketevery minute: see page 01, “What actually went wrong”, point 0.) - It takes the address from Deskflow’s own configuration, so there is no IP in the repo.
- It notifies on the Mac only when the state changes.
Transition test, with a copy that does not notify:
up(1st) -> state: up 0
miss1 -> state: up 1 <- una falla sola no cambia nada
miss2 -> state: down 2 <- dos seguidas: "Windows no responde"
back -> state: up 0
What actually went wrong
- I overwrote a file I had not read. When saving this session’s prompt to
prompts/09-sesiones-2026-10-04.md, the tool answered “updated”: the file already existed, with sessions A, B and C, and uncommitted. I recovered it from the copy that Claude Code saves before each write (~/.claude/file-history/<sesión>/), verified that the original text was intact and inserted session 0 in front. Lesson: an “updated” where you expected “created” is an alarm; and an uncommitted file has no safety net in git. watch --oncedispatched but did not close. It waited for the job to finish and exited without moving it todone/, which stayed inrunning/. The real test caught it, not the unit tests: the tests covered the normal round, not the exit of--once. Fix: areapfunction that is called on both paths.- The Mac’s boot date came out as 1970.
sysctl kern.boottimereturns{ sec = …, usec = … }, and my regex.*sec =was greedy: it capturedusec.
How to verify
PS> python -m unittest discover -s tests # 28 tests: 16 con nodos simulados
PS> Get-ScheduledTask -TaskPath '\enjambre\' -TaskName watch | Select-Object State
PS> python enjambre.py status # "watch, N s ago" con N < 90
On the Mac: ssh mac 'cat ~/.enjambre/orq-state; tail -3 ~/enjambre-logs/node-watch.log' must
show up.
Still to be checked by eye: that the Windows notification and the Mac one are visible on screen. They were sent without error, but the agent cannot see your screen.
Self-check
- The Mac answers SSH but
statussayssshand notready. What happened and what do you do?Answer
Thetmux -L enjambresession does not exist: nobody logged in on the Mac's screen (for example, after a reboot with FileVault), so there is no keychain. Log in; the LaunchAgent creates the session and the next round sees itready. - Why does the ledger not get a line when the job is queued?
Answer
Because a ledger line means "this ran". Queueing is an intention; the queue's record is the folders instate/queue. - What prevents two
watchinstances from starting if the scheduled task fires twice?Answer
Two layers: MultipleInstances = IgnoreNew on the task and a file lock taken by the process itself. The system releases it if the process dies.