Leaderboard

Live rankings of compaction methods, segmented by benchmark version and target model. Submitted via PR; evaluated on GitHub Actions; ranked by elite_score. See methodology for how scores are computed.

Reading the board: baselines and controls

Rows tagged baseline are maintainer runs of the built-in methods. Rows tagged control are reference points, not compaction methods. Neither is ranked — they are published so that submitted scores mean something.

The two controls that matter most:

• oracle hands the model the full, uncompacted transcript. It is the ceiling: what you score when compaction loses nothing. It is not 1.0 — on a small local model it scores around 0.65, because some evaluation items are simply hard for that model. A method at 0.5 has therefore not lost half the information; it is at roughly 77% of what was achievable.
• null returns an empty artifact. Every point it scores is a point available with no information at all, which tells you how much of a score is a free pass rather than retained state.

Read a method's score as its position between those two numbers, not as a percentage.

How to get on the board

1. Write a compactor subclassing compactbench.compactors.Compactor — see writing a compactor.
2. Run it locally against the elite_practice suite until you're happy.
3. Open a PR to submissions/ — full protocol in submitting.
4. A maintainer applies the evaluate label after code review; GitHub Actions runs your method against the hidden ranked set and posts the score.
5. If you qualify, your entry lands here on merge.

Tracking

• Submission workflow + leaderboard site: #5
• Launch Elite templates: #6
• PyPI release + docs polish: #7