Golden fixture library
Adapter expectations are tracked.
Each adapter should keep at least one known-good raw snapshot and one expected parsed output. Current generated snapshots are listed until explicit golden fixtures are added.
| Adapter | Parser | Snapshot hash | Raw snapshot | Expected output | Status |
|---|---|---|---|---|---|
| arena | 0.6.0 | 2d1617e1d03105e5ccbc03880476246e13fc98dd07f31af80fe11dba0fed73ab | workers/data/latest/arena.json | workers/data/latest/arena.json | manifested golden candidate |
| livebench | 0.5.0 | b8dd06726275e41433d31647c3cc3aa461f053072c112e84faecc4b7627fcdbb | workers/data/latest/livebench.json | workers/data/latest/livebench.json | manifested golden candidate |
| artificial-analysis | 0.5.0 | 647c39936b801edf4575dc126dcb9296907d6e9841c82ab2d760a07eb8187616 | workers/data/latest/artificial-analysis.json | workers/data/latest/artificial-analysis.json | manifested golden candidate |
| bridgebench | 0.3.0 | e88bdde9953c1db7dc3703cb374374f9e117a8ce1421f1f70a195f8cea3766a6 | workers/data/latest/bridgebench.json | workers/data/latest/bridgebench.json | manifested golden candidate |
| vals-ai | 0.2.0 | 7832d50554658edba8e81fdbcde0aec9caee597a004047e3b66f398373a47ecb | workers/data/latest/vals-ai.json | workers/data/latest/vals-ai.json | manifested golden candidate |
| terminal-bench | 0.3.0 | 440708058ffa16a128c7b6483e2dfc6c8762c48ee2136b4a72a66fa54426a649 | workers/data/latest/terminal-bench.json | workers/data/latest/terminal-bench.json | manifested golden candidate |
| llmbase | 0.2.0 | e3ab53b2f1030676f3dcb3876293f14a6e0300640d5c01a581366be538c8de9f | workers/data/latest/llmbase.json | workers/data/latest/llmbase.json | manifested golden candidate |
| scale-labs | 0.2.0 | c605e092f74a1a3f9bf8ff6047e7915821e9128a882efa1ff9ccb40983a8e70e | workers/data/latest/scale-labs.json | workers/data/latest/scale-labs.json | manifested golden candidate |
| opencompass | 0.2.0 | 801864a2daa42afe8a6c87c26fb901fc37dae70c272836655b3f7f118a9af93b | workers/data/latest/opencompass.json | workers/data/latest/opencompass.json | manifested golden candidate |
| mteb | 0.2.0 | 23f77e243092b2dc66631c59ae87b05da193d0ed6b92036a4c2401dfd51e1b12 | workers/data/latest/mteb.json | workers/data/latest/mteb.json | manifested golden candidate |
| provider-official | unrecorded | 9604bb00156d076e50019e92b4839563dd1984dca66c28572795fea407f0d812 | workers/data/latest/manual-verified.json | workers/data/latest/manual-verified.json | needs review before promoting latest snapshot to golden |