Golden fixture library
Adapter expectations are tracked.
Each adapter should keep at least one known-good raw snapshot and one expected parsed output. Current generated snapshots are listed until explicit golden fixtures are added.
| Adapter | Parser | Snapshot hash | Raw snapshot | Expected output | Status |
|---|---|---|---|---|---|
| arena | 0.5.0 | 5361466e58e726de5efcc035b8403ee2a08e7dbfe91c672e8fbff95a61ed2921 | workers/data/latest/arena.json | workers/data/latest/arena.json | manifested golden candidate |
| livebench | 0.5.0 | 0ee651aeff4abb8a54737851a0ee060d0dea502edbb744ffaf9b5ed16efa4c77 | workers/data/latest/livebench.json | workers/data/latest/livebench.json | manifested golden candidate |
| artificial-analysis | 0.5.0 | 067c2ec0dd039dca70737edb309f26c98cb1519080688de8305ec0cb9ab10333 | workers/data/latest/artificial-analysis.json | workers/data/latest/artificial-analysis.json | manifested golden candidate |
| vals-ai | 0.1.0 | 0e6691b3c40d18e2d6ee9d2f830cca31efb75468d15e517c239dc25f6c789348 | workers/data/latest/vals-ai.json | workers/data/latest/vals-ai.json | manifested golden candidate |
| terminal-bench | 0.2.0 | 42f23b577fc8dac27f9d592fde3c2cffa73c1a42adb9c03bbc4e4dac53a7bec2 | workers/data/latest/terminal-bench.json | workers/data/latest/terminal-bench.json | manifested golden candidate |
| llmbase | 0.2.0 | 9a18e61532a1311e3d289b20d067ce048210ee4c4400bdc53afacb69613a4835 | workers/data/latest/llmbase.json | workers/data/latest/llmbase.json | manifested golden candidate |
| scale-labs | 0.1.0 | bb10720e348bc45836c7ed0a62af983108e828947253501d71d57e168b3d20c3 | workers/data/latest/scale-labs.json | workers/data/latest/scale-labs.json | manifested golden candidate |
| opencompass | 0.1.0 | 801864a2daa42afe8a6c87c26fb901fc37dae70c272836655b3f7f118a9af93b | workers/data/latest/opencompass.json | workers/data/latest/opencompass.json | manifested golden candidate |
| mteb | 0.1.0 | 23f77e243092b2dc66631c59ae87b05da193d0ed6b92036a4c2401dfd51e1b12 | workers/data/latest/mteb.json | workers/data/latest/mteb.json | manifested golden candidate |
| provider-official | unrecorded | 9604bb00156d076e50019e92b4839563dd1984dca66c28572795fea407f0d812 | workers/data/latest/manual-verified.json | workers/data/latest/manual-verified.json | needs review before promoting latest snapshot to golden |