Run it yourself

Stand up the range in your own environment


        $ make tooling-check
        
      
        $ make venv
        
      
        $ make host-bootstrap
          # one-shot: libguestfs perms, swtpm, host iptables
      
        $ make terraform-init
        
      
        $ make networks
          # 5 nets: 4 victim mode=none + 1 attacker NAT
      
        $ make packer-kali
          # needs a prebuilt Kali qcow2
      
        $ make packer-winserver2022
          # ~7 min; needs a Windows Server 2022 eval ISO
      
        $ make packer-win11
          # ~18 min; needs a Windows 11 eval ISO + swtpm
      
        $ make apply
          # terraform: networks + base/COW volumes
      
        $ make vms
          # define + start all 16 domains via virsh
      
        $ make m1 m2 m3 m4 m5 m6 m7 m8-m9
          # per-milestone Ansible provisioning
      
        $ make snapshot-all
          # baseline snapshots for a clean eval state
      
        $ make eval MODEL=anthropic/claude-opus-4-7

Full details, prerequisites, and per-milestone role documentation live in the repository README.

Contribute data

Update the leaderboard with a pull request

Edit docs/leaderboard/results.json

This is the file contributors edit directly -- add a model entry, correct a milestone's numbers, or flip data_quality.status to "measured" / verified: true once real eval logs back a result.

Open a PR

validate-data.yml runs automatically, checking the edit against schema.json plus semantic rules (flag counts vs. flags/manifest.yaml, pass@3 fractions, monotonicity across token budgets, required caveats).

Merge to main

deploy-pages.yml triggers automatically, rebuilds the site with the new numbers baked in, and redeploys to GitHub Pages -- no manual deploy step.

Found an issue, or have real eval data?

Contributions to the range, the scorer, or the leaderboard data are all welcome.

Open a pull request