Gauntlet — methodology

Status: shadow (not auto-copy) · Source: audit · Current version: 1 · Legacy tag: audit_version=gauntlet, run_id containing gauntlet

The anti-Keel experiment. Every other audit strategy scans a universe and chooses what to bet — the rejects are data, but so is the selection bias. Gauntlet removes the choice: it is handed specific markets (e.g. every World Cup game before kickoff) and the forecaster must take a side on each one, at whatever edge exists — including negative. The result is an unfiltered calibration record that separates raw forecasting skill from selection skill.

How it works

1. Market set is given, not chosen: the kalshi-gauntlet Cloud Run job (nostream-bot repo, --mode kalshi_gauntlet) scans a Kalshi series (default KXWCGAME match-winner moneylines) for games dated within the window, or takes explicit tickers. Note: sports close_time sits weeks past the match, so scoping parses the game date from the event ticker.
2. Forecast: the palmtree Metaculus stack verbatim — multi-provider research (AskNews, native web search, prediction-market snapshots) into the 6-LLM ensemble (model=ensemble:palmtree) — pointed at a synthetic question built from the market title + resolution rules.
3. Forced side: whichever of YES/NO has the larger model-vs-ask edge, even when both are negative. model_prob_pct records the ensemble probability; edge_cents the claimed edge; cluster = event ticker (a game's 3 moneyline legs share one thesis).
4. Signals post to /api/signals with strategy=Gauntlet, audit_version=gauntlet (non-scored → off /performance, never auto-copy) and pick_kind=forced (out of the Telegram feed and funnel entirely — forced rows are lab data, not recommendations; most carry ~0 claimed edge by construction). Dedupe: a ticker is never signaled twice.

Market-blind since 2026-07-07: the first week's forced picks averaged <1¢ of claimed edge — the prediction-market snapshot provider plus odds-saturated sports search results let the market price anchor every forecaster, so the "forecast" was the market echoed back. The gauntlet job now runs with PREDICTION_MARKETS_ENABLED=false; residual leakage via web search remains, but divergence from the market is now mostly the model's own view.

What to measure

• Calibration of model_prob_pct vs resolution (the forced record has no selection bias — every question in scope is answered).
• P&L split by claimed-edge sign: positive-edge forced picks should behave like chosen picks; negative-edge ones estimate what selection is worth.
• Cross-strategy: Gauntlet's calibration vs Keel's (same model families, opposite selection regime).

Version history

• v1 (Jul 2026) — World Cup knockout pilot: daily pre-kickoff run over KXWCGAME moneylines.