Genie, Databricks’ data agent, passed 91% of runs on a new enterprise benchmark. Two frontier agents that Databricks doesn’t name passed 45% and 58%. The company’s blog post, dated 11 November, adds that Genie used less than half the tokens and finished three to four times faster. Databricks built the test with Turing and sells Genie, so every figure here is its own.
The post is by Jeff Fletcher and Rishabh Singh. On 16 November, US time, Databricks’ X account pushed it with one line of explanation: a valid SQL query can still return the wrong answer when it uses the wrong definitions or calculation rules.
We benchmarked Genie, powered by Genie Ontology, against two frontier agents on enterprise data tasks. Here’s what we found.
— Databricks (@databricks) November 16, 2026
What the benchmark covers
Most data agent benchmarks stick to tables and the pipeline code behind them, and Databricks says agents score about the same on them. This one also feeds in notebooks, dashboards, saved SQL queries and documents, where companies keep the definitions and calculation rules that a schema leaves out. Genie Ontology is Databricks’ name for the layer that extracts that knowledge and passes the relevant pieces to Genie; the company says it’s now generally available.
The test company is invented. Calderwick Energy is a UK electricity and gas supplier with hundreds of tables, and the biggest holds about 120 million rows of half-hourly meter data. Databricks seeded the workspace with the clutter of a real one: copied notebooks, abandoned analyses, stale documents, dev copies and reporting tables nobody had touched in months. Every agent got each task five times in the same live workspace. A run passed only if it cleared a grading rubric, and any tolerance for small numeric differences was fixed beforehand.

How the agents compare
Genie ran natively. Databricks wired the other two through a toolbelt of its own and left out the Genie MCP, since that would have handed them the Genie features being tested. It calls them Agent A and Agent B, describes them as agents from frontier AI labs, and doesn’t name them or their models.
| Agent | Pass rate | Avg wall time per run | Avg tokens per run |
|---|---|---|---|
| Genie | 91.0% | 100.3 | 431,829 |
| Agent A | 45.0% | 434.6 | 889,019 |
| Agent B | 58.0% | 306.5 | 1,051,070 |
The post’s title promises 1.6 times the accuracy. That figure compares Genie with Agent B, 91% to 58%; against Agent A it’s closer to double, 91% to 45%. The +46% and +33% in the text are percentage-point gaps.

The example Databricks walks through
A task asks for the planning rate Calderwick’s Commercial team set for FY24/25. In the bill cost stack table, the FY24/25 rows show 0.31 pence per kWh under a column called cfd_rate_p. The notebook that builds that table explains that the mart applies FY25/26 assumptions even to earlier billing periods, and its planning rates by year section puts FY24/25 at 0.52 p/kWh.
According to Databricks, Genie Ontology found a snippet with the historical rates and linked it to the source notebook, 45_mart_bill_cost_stack. Genie confirmed it there and answered 0.52 first time, in all five runs. One of the other agents said 0.31 every time. The second gave 0.31, doubted that a year-specific rate was documented, or did both.

What Databricks says limits the result
The benchmark is aimed at agents working in an established Databricks deployment, where Unity Catalog governs the data and workspace assets hold the business knowledge. Databricks writes that in new deployments, or where a table and its schema can answer the question, “the difference between agents may be smaller.”
The post doesn’t give the number of tasks. It says the tasks and assets were designed for sharing and points readers to their Databricks account team to run the benchmark on their own workspace. Databricks files it under its Week of Genie, with more news promised on Genie, Genie Agents and Genie MCP.
Sources
- Genie with Genie Ontology Delivers 1.6× the Accuracy with Less than Half the Tokens, Databricks blog
- Databricks on X, 16 November 2026
- Genie Ontology general availability announcement, Databricks blog
Discover more from Sulat AI & Tech News
Subscribe to get the latest posts sent to your email.




