Terminal Bench 2.0 Leaderboard Submissions This repository accepts leaderboard submissions for Terminal Bench 2.0. How to Submit 1. Fork this repository 2. Create a new branch for your submission 3. Add your submission (a job or folder of jobs) under submissions/terminal bench/2.0/ / 4. Open a Pull Request Submission Structure Required: metadata.yaml Each submission must include a metadata.yaml file with the following fields: Job Directory Requirements Each job directory must contain all of the contents of your run. Validation Rules Your submission will be automatically validated. To pass: timeout multiplier must equal 1.0 No agent timeout overrides ( override timeout sec , max timeout sec ) No verifier timeout overrides No resource overrides ( override cpus , override memory mb , override storage mb ) All trial directories must have valid result.json files Trial directories must contain other artifacts from the run Each task must be evaluated with a minimum of five trials. We recommend the k 5 flag for convenience. Submission Process 1. Open PR : When you open a Pull Request, our bot will automatically validate your submission 2. Fix Issues : If validation fails, the bot will comm…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy