Sulat.com / Writing on AIThursday, November 19, 2026
Agent Engineering/18 November 2026

OpenClaw says deleting tests and running only affected ones cut its CI compute value 95.5%

OpenClaw's Vincent Koc says weekly runner-hours fell 86.5% and the estimated value of its sponsored compute fell 95.5%. Blacksmith sponsored the capacity, and the workload changed between the two weeks.

Glossy 3D red OpenClaw lobster mascot as streams of coloured light converge into three green passing-check tiles beside dim server racks

At a glance

  • OpenClaw chief architect Vincent Koc says weekly CI runner-hours fell 86.5%, from 158,546 in the week of 17 September to 21,395 in the week to 13 November, while jobs fell 79.7%.
  • The 95.5% figure is the estimated value of Blacksmith-sponsored compute at published prices, US$1,268,958 against US$57,324 per 30 days, and Koc says OpenClaw didn't pay those amounts.
  • Koc credits deleting weak tests, moving checks off every push, one automatic bot review, running only tests a change can affect and balanced shards, and says no single fix explains the drop.
  • He says the write-up doesn't establish faster merges: median pull request CI time was 12 minutes 10 seconds in September and 9 minutes 18 seconds in a 4 November sample, before test selection.

OpenClaw chief architect Vincent Koc says the project cut its weekly continuous integration (CI) runtime by 86.5% between September and November, and the estimated value of that compute by 95.5%. CI is the automated testing that runs on every code change, and a runner is the machine that does the work. In a write-up published on 16 November and updated the next day, Koc puts the week of 17 September at 158,546 runner-hours and the week to 13 November at 21,395.

Blacksmith, which supplies the runners, sponsors the capacity, so the dollar figures show what the compute would cost at its published prices, not what OpenClaw paid. Koc prices the September pace at US$1,268,958 per 30 days and the latest week’s pace at US$57,324. He also thanks GitHub for donated runners and a GitHub Enterprise plan, and says OpenClaw has more recently started using runners from Boat.dev, with its sponsorship.

Bar chart: 158,546 CI runner-hours in the week of 17 September against 21,395 in the week to 13 November, 86.5% fewer
Weekly runner-hours fell 86.5%, though the workload changed between the two weeks.

What the 95.5% compares

Koc says the two weeks aren’t the same work replayed on two setups. Jobs fell 79.7%, from 1,482,257 to 301,413, and indexed commits in the main repository were about 44% lower, from 3,850 to 2,149. He says neither reduction belongs to one fix.

Why tests became the limit

Koc says OpenClaw’s coding agents ran into tests rather than tokens, memory or disk. The agents produced code faster, and with it more tests, more jobs and more retries, so work queued while checks ran. About 89% of OpenClaw’s code is TypeScript, he says, which is where he started.

What OpenClaw changed

The write-up lists eight changes, and the table below summarises the figures Koc reports for each. They are OpenClaw’s own measurements, and Koc says the savings aren’t additive.

ChangeWhat Koc reports
Deleted weak testsTests and test support fell from 68.9% of core source lines on 28 September to 63.2% on 13 November. Test files lost 417,906 lines net while implementation grew 444,121.
Capped test growthA guard added on 24 October requires added test lines to be offset by deletions or a justified higher ceiling. By 13 November a change can add 400 net test lines without a recorded allowance.
Moved checks off every pushPull requests get the checks needed to merge, pushes to main run small guards, and broader validation of main runs hourly. Release checks run on their own schedule.
Limited bot reviewsClawSweeper, the project’s review bot, gives a new pull request one automatic review. Follow-up reviews run on request.
Selected tests by diffA planner maps changed files to the tests that own or import them. Three pull request runs went from 4,018 to 15, 4,480 to 16 and 6,895 to 97 test entries.
Used faster toolsA local full typecheck went from 447 to 64 seconds after a switch from tsgo to tsrs. Koc says that is a local comparison, not an end-to-end CI speedup.
Cut repeated setupReusing workers in selected Vitest lanes cut case-execution time from 174.2 to 97.1 seconds. A 55.1 MB test archive became a 30.0 MB runtime package and a 0.48 MB tooling package.
Packed and balanced shardsOne change packed 95 planned Node shards into 63. Another raised planned jobs from 18 to 20 and cut the slowest compact job from 403 to 300 seconds.
Figures are OpenClaw’s own, as reported by Vincent Koc at vincent.build.

Koc says the selector is ordinary deterministic code, not a model: the same diff and rules give the same list, and each chosen test carries a reason such as explicit-owner or import-consumer. A missing dependency record widens the run instead of skipping tests, and setting the repository variable OPENCLAW_CI_NODE_SELECTION to full switches aggressive selection off. A learned model that predicts failing tests runs in shadow mode, scoring runs without changing which tests execute.

Flow diagram: a change to retry.ts runs retry.test.ts and send.test.ts, which import it, and skips profile.test.ts; one real PR ran 15 of 4,018 test entries
Tests that own or import a changed file run and the rest are skipped; the file names are an illustrative example.

Cheaper isn’t always faster

Koc reports one matched pair from a pull request. The same 240 test files cost an estimated US$1.152 on a 32-class runner and took 17 minutes 54 seconds, then US$0.672 on a 16-class runner and took 20 minutes 51 seconds. That’s 41.7% cheaper and 16.5% slower. Both runs share a known failing assertion, he says, so it’s a pricing comparison and not a pair of passing builds.

Side-by-side cards: 32-class runner US$1.152 and 17m 54s, 16-class runner US$0.672 and 20m 51s on the same 240 test files
The smaller runner cost 41.7% less and took 16.5% longer in one reported pair.

What the numbers don’t show

Koc says the write-up doesn’t establish that changes now merge sooner. Median CI time for a pull request was 6 minutes 41 seconds in June, 12 minutes 10 seconds in September and 9 minutes 18 seconds in a partial sample on 4 November, all before test selection rolled out. His own median time from opening a pull request to merging it over the 30 days to 4 November was 37.8 minutes, or 25.3 minutes with draft time removed.

The sharpest fall came as selection arrived around 6 and 7 November, when estimated daily compute value dropped from about US$7,738 on 5 November to US$1,507 on 7 November. Koc says earlier cleanup and runner changes were already building up by then, so the step can’t be credited to selection alone.

Koc says he wants to package the reusable parts into a skill and a standalone repository, covering test inventory, dependency collection, replay of selections and fallbacks. Neither is released. He also says OpenClaw is building its own Mac Studio cluster for slow background workloads.


Discover more from Sulat AI & Tech News

Subscribe to get the latest posts sent to your email.