← Ben Lai

Predictions

Claims from the essays, pulled into one dated ledger before resolution. Misses stay visible; ambiguous calls do not count toward the score.

Open10

Waiting on their resolve-by dates.

Graded0

0 ambiguous excluded.

Brier scorePending

Lower is better; starts once a claim resolves.

Open claims

Pre-registered checks

  1. Open68%

    Local or fine-tuned open models will handle most high-volume narrow LLM calls in cost-sensitive production stacks.

    From Local models won the long tail

    Stated
    Resolve by
    Resolution criterion
    By the resolve date, at least three credible production writeups show local or fine-tuned open models serving a majority of narrow classification, extraction, formatting, reranking, or summarisation calls after frontier comparison.
  2. Open72%

    Output-token volume will become a named margin-control metric for agent products, separate from input-token volume.

    From The output token tax

    Stated
    Resolve by
    Resolution criterion
    At least three agent-product teams or vendors publicly report output-token reduction, output-token budgets, or output-token caps as a distinct cost-control practice.
  3. Open62%

    AI-assisted coding will keep showing a higher serious-defect rate unless teams narrow it to well-tested or low-blast-radius work.

    From Vibe-coded code has more bugs. Price it in.

    Stated
    Resolve by
    Resolution criterion
    New controlled studies or large team retrospectives still show elevated serious-defect or security-defect rates for broad AI-assisted coding, while safer results cluster around test-heavy, low-blast-radius, or prototype code.
  4. Open70%

    Production agent teams will treat MCP servers as governed dependencies with pinning, scoped credentials, and health checks.

    From Your MCP server is a prod dependency

    Stated
    Resolve by
    Resolution criterion
    Public MCP deployment guides from at least three serious teams or vendors recommend version pinning, scoped credentials, and real-call health checks for MCP servers used in production agent paths.
  5. Open66%

    Postgres-backed queues will remain the better default than Kafka for most internal queues under roughly 1,000 events per second.

    From Postgres is a queue. Stop reaching for Kafka.

    Stated
    Resolve by
    Resolution criterion
    New small-team architecture writeups and incident reports still favor Postgres or similar database-backed queues for sub-1,000 event-per-second internal workloads unless replay fan-out or very high throughput is required.
  6. Open64%

    Teams will increasingly treat LLM model changes as product changes, not implementation details.

    From The model switch is a product change

    Stated
    Resolve by
    Resolution criterion
    At least three public release processes, vendor guides, or postmortems describe model changes as requiring product review for tone, latency, quality, cost, or trust changes.
  7. Open58%

    State-changing AI or ops CLIs will be expected to emit receipts that make later audit possible.

    From Your CLI needs a receipt

    Stated
    Resolve by
    Resolution criterion
    At least three developer tools or internal-platform writeups add explicit command receipts, run ledgers, or audit logs for CLI actions that change state.
  8. Open55%

    Teams shipping AI features will move model cards or model-behavior notes into the repository alongside code.

    From The model card belongs in the repo

    Stated
    Resolve by
    Resolution criterion
    At least three public engineering examples keep model behavior, limits, eval notes, or prompt contracts versioned in the same repo as the feature using them.
  9. Open57%

    AI-product teams will treat cache behavior as a product contract because cache misses change latency, cost, and quality.

    From The cache is a contract

    Stated
    Resolve by
    Resolution criterion
    At least three public AI-product or infrastructure writeups define cache hit behavior, invalidation, or fallback behavior as user-facing contract surface rather than a pure optimization.
  10. Open60%

    Early-stage teams will keep underinvesting in observability until incidents force the first boring dashboard.

    From Observability is a tax you pay before you owe it

    Stated
    Resolve by
    Resolution criterion
    Multiple public startup postmortems or retrospectives still cite missing baseline dashboards, logs, or traces as a contributor to avoidable debugging time or incident duration.

Graded claims

No grades yet

The first extracted claims resolve in 2027. This section is intentionally empty until a claim reaches its date and gets scored.