Data • Analytics • Evaluated AI

Chess Analytics Lab

A reproducible chess data pipeline, explicit analytical metrics, and an AI analyst evaluated against checked evidence.

Data pipeline 1,042,346

Opening-corpus training games. Completed games with at least ten moves, after the archives land in Parquet and DuckDB.

  1. Lichess archives
  2. Parquet / DuckDB
  3. dbt marts
  4. Metrics
  5. Dashboard
reports/opening-corpus.json · counts.training_games
Transpositions 113,583

Elite positions reached by two or more move orders, inside 240,086 Elite games.

Elite endpoint compression by ply
Ply 6Axis 0–0.50Ply 20
reports/opening-positions.json · cohorts.elite_reference.views.all.transposing_positions
AI analyst 39/40

Decisions correct routes on the same 40 historical questions (97.5%). This is route choice, not a correct chess answer.

reports/decisions-checkpoint.json · decisions-historical-test.json scoring
Analyze games
Drop in a PGN Choose a player, then compare White games with the public study set. Nothing is uploaded.

Python · SQL · DuckDB · Parquet · dbt · Streamlit · AI evaluation

What the lab demonstrates

1,042,346

Selected public games

A checked opening-training game index, with Elite and broader cohorts kept separate. Opening-position rankings are measured; drills remain proposed.

Inspect corpus evidence
31/32 · 31/32

Final-answer comparison

The direct analyst and Jev gate tied in one small paired Luna experiment under a versioned offline rescore. The gate cost more; it remains optional.

See results and limitations
4 games · offline

Reproducible demo

Authored synthetic games exercise ingestion, metrics and checked answer replay. Fixture checks are separate from live model results.

Run the walkthrough
Read the measured results

Rules, Jev and Decisions on shared questions; explore routing errors and thresholds.

Actual classification runs + explicitly labeled replay
Follow the pipeline

Inspect how source bytes become checked metrics and evidence-bound answers.

Six stages with code links
Understand the definitions

Search versioned contracts and schemas; explore eligible versus evaluable denominators.

Explicit missingness and coverage
Explore positions across move orders

Compare recurring boards, route balance and coverage across public cohorts.

Measured rankings, charts and fianchetto lenses
Design lessons around positions

Replay an illustrative transposition and inspect the learning proposal.

Lesson design and evaluation plan
Run it yourself

Build tiny synthetic data, explore the dashboard and replay a checked answer.

No API key; no real archive download

Reliable data. Inspectable analytics. Measured AI.

The project connects three pieces: a bounded chess-game ingestion pipeline, opening and clock analytics with explicit denominators, and a read-only AI analyst whose answers carry snapshot-bound evidence. Local DuckDB and Parquet keep the system reproducible; dbt models and recovery checks demonstrate how the data is validated before publication.

The dashboard runs locally. The website lets you inspect recorded results, explore measured opening positions and move orders, try illustrative controls, and follow the offline demo.

What the comparison teaches

Jev is TypeSafe AI's typed decision model. OpenAI Decisions is the optional classifier studied alongside it. Both select task routes; neither replaces the checked calculations that produce a chess metric.

Better routing did not establish better final answers. The retained paired experiment found equal correctness and a higher cost with the Jev gate. The later Decisions final-answer comparison uses saved-answer replay rather than a new live pairing. These are small, dated experiments, with failures and limitations retained alongside their results.

Read further

Question Evidence and guide
How does the pipeline work? Architecture
How are metrics defined? Definitions and denominators
What do the models demonstrate? Measured comparison
Which study boards occur in my games? Local PGN analysis, with explicit player selection and a bounded public reference
Which opening positions recur across move orders? Transposition analysis and learning design
Can I reproduce a small example? Offline walkthrough
What is complete and what comes next? Current project status

The documentation guide indexes operational runbooks, analytical memos and historical evidence. The repository contains the maintained code and contracts. Project author: jfk-ssa on GitHub.