Leeroo open-sources Kapso, an engine that runs experiments and keeps lessons from each run

Leeroo has open-sourced Kapso, an engine that runs experiments toward a goal you set, deploys the best result and keeps lessons from every run for the next one. The London startup announced it in a Show HN post on 12 November 2026, and the code is on GitHub under the MIT licence, installed with pip install leeroo-kapso.

Alireza and Majid, the Leeroo founders who wrote the post, say they previously built AI systems at LinkedIn and Meta, and Leeroo’s site says Y Combinator backs it. They say Kapso ranks first on all three RelBench leaderboards, a Stanford benchmark for predictions over relational data, and that it outscored all 471 human contestants at the International Olympiad in AI 2026. Those results are Leeroo’s own reports; the benchmark section below has the figures.

How a Kapso run works

You give Kapso an objective and success criteria. According to the Kapso docs, it then proposes a candidate, implements it on its own git branch, evaluates it, judges the result and repeats with what it learned. The README says ideas come from the earlier experiments and the bank of lessons, and that Kapso re-ranks them by expected return against the budget each would use.

Every result is checked by an evaluator that, in the README’s words, no experiment can touch. The HN post adds that the experiment modules can propose changes to the metrics and criteria, and the verifier approves or rejects them. The post says this exists because the success criteria and tests sometimes need to evolve while a project runs.

The best solution then goes through kapso.deploy(), a shorter round of experiments that adapts it to the target stack. The README lists Docker, Modal, BentoCloud and LangGraph Platform as targets that ship with it.

The bank of lessons

The part Leeroo stresses is what happens after a run. Each lesson Kapso files carries a credence score, which records how far the evidence backs it, plus a boundary on where it applies and the evidence behind it. After a run, learn() updates the scores from the outcome, and Leeroo says it A/B tests each change to the bank by rebuilding with and without it.

The bank is a git repository, so it can be shared: kapso experience share, list and update handle that from the command line. Leeroo says lessons earned in one environment suit it better than the generic ideas a bare language model or retrieval over documentation produces. A domain package called a harness is where a bank accumulates; Kapso ships harnesses for predictive modelling and competitive programming, and kapso harness new starts one of your own.

Leeroo also runs a public bank, Leeroopedia, which Kapso sessions query during ideation. The HN post says it holds around 100,000 pages and serves about 6 million requests a month, around 28% of them from AI agents. Leeroo’s homepage describes Leeroopedia as a 27,600-page wiki, and neither page explains the difference.

The benchmark claims

The figures below come from the Kapso README and the HN post. Leeroo says the RelBench numbers are entries on Stanford’s official leaderboard, and the IOAI score is the organisers’ grading. NMAE, normalised mean absolute error, is a regression error where lower is better; AUROC and MAP are scores where higher is better.

Benchmark	Kapso, per Leeroo	Compared with
RelBench classification	81.2 AUROC	79.3 for KumoRFM-v2, 79.0 for Stanford STAR’s RT-J
RelBench regression	0.2476 NMAE	0.2711 for RT-J, 0.2847 for KumoRFM-v2
RelBench recommendation	18.4 MAP	9.3 for ID-GNN, the best other entry
MLE-Bench	First among open-source systems	OpenAI’s benchmark of Kaggle-style ML competitions
ALE-Bench	1909 Elo	1879 for Sakana AI’s ALE Agent
IOAI 2026 AI track	536.07 points, above all 471 human contestants	Top three of 14 AI systems; Grand Master Trophy

Leeroo’s IOAI write-up says Kapso ran autonomously on the contest’s six tasks, scoring from 73.23 on model compression to 98.84 on text forensics.

One HN commenter reports a separate result. The user granthamctaylor, who says they work on Relflow, an open-source model factory, wrote that Relflow used Kapso to drive a hyperparameter search on RelArena, a RelBench-style benchmark that Prior Labs maintains. They say it beat TabPFN on most of the larger of 21 tasks with about 1% of TabPFN’s parameters and no in-context learning. Alireza replied with thanks and pointed to Leeroo’s own RelBench results.

What you need to run it

Kapso runs its inference through coding-agent command-line tools, and the README says there’s no direct-API fallback. You need Node.js 18 or later with both Claude Code and Codex installed and logged in, then Python 3.10 or later. The packaged config runs build sessions on claude-opus-5, the learning crews on claude-fable-5 and Codex roles on gpt-5.6-sol, and the README notes that access depends on your subscription. A Leeroopedia key is recommended, and a new account comes with $20 in free credit.

kapso doctor checks the setup and names the fix for anything missing. A first run is kapso.build() with a goal, an optional repository and a budget, such as Budget(minutes=120); the README also lists max_iterations, time_budget_minutes and a best-effort cost_budget in US dollars. The repository’s examples include fixing dbt models on Snowflake, raising a churn model’s ROC AUC and speeding up a CUDA attention kernel.

The name collides with an unrelated product. HN commenter fcortes pointed out that Kapso is already the name of a WhatsApp integration service, and the README warns that the PyPI package called kapso is an unrelated WhatsApp tool whose command shadows this one. If it’s installed, run pip uninstall kapso first. Alireza replied on HN that the name came from the initials of “Knowledge-grounded framework for Autonomous Program Synthesis and Optimization”, that Leeroo saw the WhatsApp product a few weeks later, and that it’s considering a new name.

What the announcement doesn’t show

The benchmark results are Leeroo’s own reports, and the post doesn’t say what the campaigns cost in model spend. The post says Leeroo is open-sourcing Kapso today, and the GitHub repository was created on 7 December 2025. For companies, the README points to an enterprise offer, and Leeroo says it used the same approach on a lakehouse migration covering more than 30 source-system families, thousands of tables and hundreds of BI reports. Leeroo’s site offers a free 48-hour read-only audit of a company’s systems.