Skip to content
Projects

Signal Agent

In development prototype

A market monitor where the model has to write a falsifiable prediction before it can be scored — 3 open, 0 resolved, the first ones due January 2027.

A market monitor that collects on a schedule in GitHub Actions and then makes a model account for itself: every judgement is signed, every call ships a falsifiable prediction with its resolution criteria written down in advance, and the accuracy ledger is the actual product.

Technology

  • AI & agents
  • Automation

Sector

  • Economics & finance

Problem

Most agent demos are unfalsifiable. The model produces a confident paragraph, nobody writes down what would have counted as wrong, and next month there is a new confident paragraph. The interesting question is not whether a model can summarise a market — it is whether you can hold one to account afterwards, which means deciding in advance what would count as a miss.

What it will be

A case study on the part that is the point: the schema. A judgement records who made it (`judged_by`, one of claude-code, a human, or an import) and its verdict; confidence is forced to NULL when the verdict is no-change, so nothing can be confidently non-committal. A prediction carries a binary `claim`, the `resolution_criteria` written before the outcome is known, and a horizon that fixes the date it resolves. An outcome is correct, incorrect, or genuinely undetermined — and undetermined is excluded from the accuracy rate instead of being rounded into it. Re-judging never reads the previous verdict, so old confidence cannot anchor a new one. An indicator that fails to beat a coin flip over ten resolved predictions is flagged for removal. Collection and briefing run themselves; the judging step is run by hand inside Claude Code, and there is no LLM API key anywhere in the codebase.

Status: Phase 1 only: 95 observations, 6 signals and 3 open predictions collected. Nothing has resolved yet, so there is no accuracy number to show — the first predictions mature in January 2027, and the scoring commands are still stubs. Repo and demo follow.

Architecture

Market monitor: deterministic collection runs on GitHub Actions and the model's judgment lands in a SQLite ledger of falsifiable predictions.

  • GitHub
cronrun.py collect|briefpush · issuecarga domain.yamlHTTPS GET + ETagSQLescribe el .mdabre /analizarrun.py brief · judgelee ANALYZE.mdpersona · dueño del criterioOwnerlee el issue, lanza /analizarexterno · repo + cronGitHubrama main · cron · Issuesgithub actions · ubuntu-latestRunner de ActionsPython 3.12 · pip · gitsesión de claude code · /analizarSesión de juicioClaude Code ·.claude/commands/analizar.mdpython · núcleo del repoCLI y núcleorun.py · core/ · pydantic ·sqlite3externo · fuentes públicasFuentes de datosRSS · JSON · precios CSV/exchange ·HTMLsqlite · versionado en gitsignal.dbSQLite WAL · schema.sql ·views.sql · migracionesarchivos · la costurabriefings/YYYY-MM.mdmarkdown autosuficientearchivos · configuración y criterioDominios y criteriodomains/*/domain.yaml ·ANALYZE.mdC4 · nivel 2 (contenedores) · tráfico ambiente = llamadas reales · clic en una caja para el detallelínea continua = HTTP/SQL · punteada = adaptador elegido por configuración · caja punteada = existe fuera del proyecto
open in a new tab ↗

Flows you can replay

  1. Colecta semanal
  2. Briefing mensual con predicciones
  3. Juicio mensual y alta de predicciones