Golden fixture library
Adapter expectations are tracked.
Each adapter should keep at least one known-good raw snapshot and one expected parsed output. Current generated snapshots are listed until explicit golden fixtures are added.
| Adapter | Parser | Snapshot hash | Raw snapshot | Expected output | Status |
|---|---|---|---|---|---|
| arena | 0.6.0 | 60b99dc9f6dac066eb0d2adb4ded78c9db0c5c6272f3d0c7920b7b7a27d28138 | workers/data/latest/arena.json | workers/data/latest/arena.json | manifested golden candidate |
| livebench | 0.5.0 | cf57b8354c09ecae3a05e6dd4bab0c95bb432274689c3c753ea7983418ec951e | workers/data/latest/livebench.json | workers/data/latest/livebench.json | manifested golden candidate |
| artificial-analysis | 0.5.0 | 7649ce02398ef24f2ec8593949ad0f7bbb55e82c13aed81751348b4be370f946 | workers/data/latest/artificial-analysis.json | workers/data/latest/artificial-analysis.json | manifested golden candidate |
| bridgebench | 0.3.0 | e88bdde9953c1db7dc3703cb374374f9e117a8ce1421f1f70a195f8cea3766a6 | workers/data/latest/bridgebench.json | workers/data/latest/bridgebench.json | manifested golden candidate |
| vals-ai | 0.2.0 | 7832d50554658edba8e81fdbcde0aec9caee597a004047e3b66f398373a47ecb | workers/data/latest/vals-ai.json | workers/data/latest/vals-ai.json | manifested golden candidate |
| terminal-bench | 0.3.0 | 1eeffc806116555851d93bbc2894350c1f60ede63a4d0126c09e37eb6c998c4f | workers/data/latest/terminal-bench.json | workers/data/latest/terminal-bench.json | manifested golden candidate |
| llmbase | 0.2.0 | b4e40639ead08ad1025cf15d6014dd72363dcb5532421df0409a050f06ebd187 | workers/data/latest/llmbase.json | workers/data/latest/llmbase.json | manifested golden candidate |
| scale-labs | 0.2.0 | 9210b181118748f99290cf1f03907715e1b00f8989faf84ea3fcd37ccb4164b5 | workers/data/latest/scale-labs.json | workers/data/latest/scale-labs.json | manifested golden candidate |
| opencompass | 0.2.0 | 801864a2daa42afe8a6c87c26fb901fc37dae70c272836655b3f7f118a9af93b | workers/data/latest/opencompass.json | workers/data/latest/opencompass.json | manifested golden candidate |
| mteb | 0.2.0 | 23f77e243092b2dc66631c59ae87b05da193d0ed6b92036a4c2401dfd51e1b12 | workers/data/latest/mteb.json | workers/data/latest/mteb.json | manifested golden candidate |
| provider-official | unrecorded | 9604bb00156d076e50019e92b4839563dd1984dca66c28572795fea407f0d812 | workers/data/latest/manual-verified.json | workers/data/latest/manual-verified.json | needs review before promoting latest snapshot to golden |