Runner benchmark
What a CI or agent job costs on each option, at 2, 4 and 8 vCPU. Updated 2026-11-07.
Get the data: benchmark.json · benchmark.csv · ranking rule · method. Jump to 2 · 4 · 8 vCPU.
We run Sengi. Rows marked “we run this” are ours. Rows are listed cheapest first by cost per job; Sengi gets no special placement.
2 vCPU
| Option | $/min | Billing rule | Median wall time | Median queued to running | Cost per job | With GitHub self-hosted fee |
|---|---|---|---|---|---|---|
| AWS EC2 spotavailable, can be interruptedm7i.large / xlarge / 2xlarge, us-east-1, Linux; price snapshot, moves hourly | $0.00075$0.0449/h | Per second, 60 s minimum, at the spot price; boot is billed; disk and public IPv4 extra | not yet measured | not yet measured | $0.00748 | $0.0275 |
| Sengi Basiclaunch pendingwe run thissengi-basic-N-ubuntu-2404 | $0.001 | Per minute, each job rounded up once; boot and teardown unbilled | not yet measured | not yet measured | $0.01 | $0.03 |
| Ubicloudubicloud-standard-N | $0.00125 | By the minute, per job | not yet measured | not yet measured | $0.0125 | $0.0325 |
| AWS EC2 on-demandm7i.large / xlarge / 2xlarge, us-east-1, Linux | $0.00168$0.1008/h | Per second, 60 s minimum; boot is billed; disk and public IPv4 extra | not yet measured | not yet measured | $0.0168 | $0.0368 |
| Sengi Premiumbetawe run thissengi-premium-N-ubuntu-2404, AMD Ryzen 9 7950X3D | $0.003 | Per minute, each job rounded up once; boot and teardown unbilled | not yet measured | not yet measured | $0.03 | $0.05 |
| GitHub-hostedLinux x64 (ubuntu-latest and larger runners) | $0.006 | Per minute, each job rounded up to a whole minute | not yet measured | not yet measured | $0.06 | $0.06no change |
| Hetzner CloudCCX13 / CCX23 / CCX33 dedicated vCPU, Germany/Finland | $0.0013€0.0697/h | Per started hour, capped at the monthly price; one server per job pays a full hour | not yet measured | not yet measured | $0.0778 | $0.1978 |
| Sengi partner rateopenrunners.io partner ratePriced 20% below Ubicloud per job, after measured speed. | awaiting measurement | Same as Sengi: per minute, rounded up once | awaiting measurement | awaiting measurement | awaiting measurement | awaiting measurement |
4 vCPU
| Option | $/min | Billing rule | Median wall time | Median queued to running | Cost per job | With GitHub self-hosted fee |
|---|---|---|---|---|---|---|
| AWS EC2 spotavailable, can be interruptedm7i.large / xlarge / 2xlarge, us-east-1, Linux; price snapshot, moves hourly | $0.0015$0.0897/h | Per second, 60 s minimum, at the spot price; boot is billed; disk and public IPv4 extra | not yet measured | not yet measured | $0.0149 | $0.035 |
| Sengi Basiclaunch pendingwe run thissengi-basic-N-ubuntu-2404 | $0.002 | Per minute, each job rounded up once; boot and teardown unbilled | not yet measured | not yet measured | $0.02 | $0.04 |
| Ubicloudubicloud-standard-N | $0.0025 | By the minute, per job | not yet measured | not yet measured | $0.025 | $0.045 |
| AWS EC2 on-demandm7i.large / xlarge / 2xlarge, us-east-1, Linux | $0.00336$0.2016/h | Per second, 60 s minimum; boot is billed; disk and public IPv4 extra | not yet measured | not yet measured | $0.0336 | $0.0536 |
| Sengi Premiumbetawe run thissengi-premium-N-ubuntu-2404, AMD Ryzen 9 7950X3D | $0.006 | Per minute, each job rounded up once; boot and teardown unbilled | not yet measured | not yet measured | $0.06 | $0.08 |
| GitHub-hostedLinux x64 (ubuntu-latest and larger runners) | $0.012 | Per minute, each job rounded up to a whole minute | not yet measured | not yet measured | $0.12 | $0.12no change |
| Hetzner CloudCCX13 / CCX23 / CCX33 dedicated vCPU, Germany/Finland | $0.00258€0.1386/h | Per started hour, capped at the monthly price; one server per job pays a full hour | not yet measured | not yet measured | $0.1547 | $0.2747 |
| Sengi partner rateopenrunners.io partner ratePriced 20% below Ubicloud per job, after measured speed. | awaiting measurement | Same as Sengi: per minute, rounded up once | awaiting measurement | awaiting measurement | awaiting measurement | awaiting measurement |
8 vCPU
| Option | $/min | Billing rule | Median wall time | Median queued to running | Cost per job | With GitHub self-hosted fee |
|---|---|---|---|---|---|---|
| AWS EC2 spotavailable, can be interruptedm7i.large / xlarge / 2xlarge, us-east-1, Linux; price snapshot, moves hourly | $0.00296$0.1774/h | Per second, 60 s minimum, at the spot price; boot is billed; disk and public IPv4 extra | not yet measured | not yet measured | $0.0296 | $0.0496 |
| Sengi Basiclaunch pendingwe run thissengi-basic-N-ubuntu-2404 | $0.004 | Per minute, each job rounded up once; boot and teardown unbilled | not yet measured | not yet measured | $0.04 | $0.06 |
| Ubicloudubicloud-standard-N | $0.005 | By the minute, per job | not yet measured | not yet measured | $0.05 | $0.07 |
| AWS EC2 on-demandm7i.large / xlarge / 2xlarge, us-east-1, Linux | $0.00672$0.4032/h | Per second, 60 s minimum; boot is billed; disk and public IPv4 extra | not yet measured | not yet measured | $0.0672 | $0.0872 |
| Sengi Premiumbetawe run thissengi-premium-N-ubuntu-2404, AMD Ryzen 9 7950X3D | $0.012 | Per minute, each job rounded up once; boot and teardown unbilled | not yet measured | not yet measured | $0.12 | $0.14 |
| GitHub-hostedLinux x64 (ubuntu-latest and larger runners) | $0.022 | Per minute, each job rounded up to a whole minute | not yet measured | not yet measured | $0.22 | $0.22no change |
| Hetzner CloudCCX13 / CCX23 / CCX33 dedicated vCPU, Germany/Finland | $0.00414€0.2227/h | Per started hour, capped at the monthly price; one server per job pays a full hour | not yet measured | not yet measured | $0.2486 | $0.3686 |
| Sengi partner rateopenrunners.io partner ratePriced 20% below Ubicloud per job, after measured speed. | awaiting measurement | Same as Sengi: per minute, rounded up once | awaiting measurement | awaiting measurement | awaiting measurement | awaiting measurement |
Ranking rule
- Cost per job, cheapest first. For measured runs this is the median, across all runs of the same job, of what each run was billed under that option's billing rule (minimum charge, rounding and billed boot time included).
- Ties go to the shorter median queued-to-running time.
- Failures count. A run that fails for infrastructure reasons (no runner, lost runner, out of capacity) is reported, never dropped. An option whose infrastructure failure rate is above 2% is flagged.
- Same job, same shape. Each shape compares machines with the same vCPU count. Memory differs slightly between providers; each row links to its source.
Until runs are measured, rows are ordered by list cost per 10-minute job. That is a price comparison, not a ranking.
Sengi partner rate
Sengi's openrunners.io partner rate is priced 20% below Ubicloud per job, after measured speed. It is computed from the benchmark, not set in advance:
partner_rate_per_min = 0.8 x (Ubicloud median cost per job) / (Sengi median billed minutes per job), per shapeIf Sengi is faster per job, its partner rate per minute can be above Ubicloud's; if it is slower, below. Until the harness has measured both, the row says “awaiting measurement”. It is our own discount, offered here, and is listed beside Sengi's normal prices, not instead of them.
The GitHub self-hosted fee scenario
In December 2025 GitHub announced a $0.002 per minute platform charge on self-hosted runner minutes, then postponed it. Every non-GitHub option here runs jobs as a self-hosted runner, so the last column shows its cost per job if that charge returned. GitHub-hosted prices already include it.
Method
- The benchmark runs the same real CI jobs on every option and records queued-to-running time, wall time, billed amount and failures for each run. Its raw per-run output will be published beside the data on this page.
- Prices are public list prices on the date checked, in USD. Euro prices use €1 = $1.1165 (ECB euro reference rate, 2026-11-06). AWS is us-east-1 Linux, m7i family; spot is a snapshot and moves hourly. Hetzner is the CCX dedicated-vCPU line in Germany and Finland.
- Not included unless measured: disk, network egress, public IPv4, caching services and support plans.
- Found a wrong number? Every row links to its source. Tell us and we'll correct it in the open.