Golden fixture library
Adapter expectations are tracked.
Each adapter should keep at least one known-good raw snapshot and one expected parsed output. Current generated snapshots are listed until explicit golden fixtures are added.
| Adapter | Parser | Snapshot hash | Raw snapshot | Expected output | Status |
|---|---|---|---|---|---|
| arena | 0.5.0 | 5361466e58e726de5efcc035b8403ee2a08e7dbfe91c672e8fbff95a61ed2921 | workers/data/latest/arena.json | workers/data/latest/arena.json | manifested golden candidate |
| livebench | 0.5.0 | b5adef436935a96890343d426c265c11128a1bb472de774f1eb00c22747b989a | workers/data/latest/livebench.json | workers/data/latest/livebench.json | manifested golden candidate |
| artificial-analysis | 0.5.0 | 7fe9c753fce694973a577f727f0749b7da8bcd1e9182c35bd63b64031a2f34a9 | workers/data/latest/artificial-analysis.json | workers/data/latest/artificial-analysis.json | manifested golden candidate |
| bridgebench | 0.3.0 | e88bdde9953c1db7dc3703cb374374f9e117a8ce1421f1f70a195f8cea3766a6 | workers/data/latest/bridgebench.json | workers/data/latest/bridgebench.json | manifested golden candidate |
| vals-ai | 0.1.0 | 0e6691b3c40d18e2d6ee9d2f830cca31efb75468d15e517c239dc25f6c789348 | workers/data/latest/vals-ai.json | workers/data/latest/vals-ai.json | manifested golden candidate |
| terminal-bench | 0.2.0 | 05252fe4359b5b454b1d0784e7e6c588b9630f1479fcec998eb240b6c0ccfd02 | workers/data/latest/terminal-bench.json | workers/data/latest/terminal-bench.json | manifested golden candidate |
| llmbase | 0.2.0 | a8050b6b98e22e853ec61f955c41ff08c8e9e5bcf940479579c3281dc85e9b43 | workers/data/latest/llmbase.json | workers/data/latest/llmbase.json | manifested golden candidate |
| scale-labs | 0.1.0 | bb10720e348bc45836c7ed0a62af983108e828947253501d71d57e168b3d20c3 | workers/data/latest/scale-labs.json | workers/data/latest/scale-labs.json | manifested golden candidate |
| opencompass | 0.1.0 | 801864a2daa42afe8a6c87c26fb901fc37dae70c272836655b3f7f118a9af93b | workers/data/latest/opencompass.json | workers/data/latest/opencompass.json | manifested golden candidate |
| mteb | 0.1.0 | 23f77e243092b2dc66631c59ae87b05da193d0ed6b92036a4c2401dfd51e1b12 | workers/data/latest/mteb.json | workers/data/latest/mteb.json | manifested golden candidate |
| provider-official | unrecorded | 9604bb00156d076e50019e92b4839563dd1984dca66c28572795fea407f0d812 | workers/data/latest/manual-verified.json | workers/data/latest/manual-verified.json | needs review before promoting latest snapshot to golden |