Databricks, a Data and AI company, has launched Genie Code, an autonomous AI agent designed to assist with data engineering tasks. Genie Code performs functions such as building pipelines, debugging failures, managing dashboards, and maintaining production systems.
In testing on real-world data science tasks, Genie Code achieved higher success rates than leading coding agents, reflecting a shift from traditional autocomplete-style support to agent-driven workflows. It builds on the existing Genie platform, which allows users to interact with their data, by automating complex operations from initial development to production while keeping humans in control of key decisions.
Databricks also announced the acquisition of Quotient AI, a company specialising in evaluation and reinforcement learning for AI agents, to integrate continuous assessment into Genie and Genie Code.
Unlike traditional AI coding tools, which often lack access to important context such as data lineage, usage patterns, and business semantics, Genie Code provides support for end-to-end workflows with governance and accuracy suitable for production environments. Key features include:
- Machine Learning Workflow Management: Plans, writes, deploys, and monitors ML models while documenting experiments.
- Data Engineering Support: Designs workflows that account for differences between staging and production environments, including change data capture and data quality expectations.
- Proactive Maintenance: Monitors pipelines and models, identifying and addressing anomalies before human intervention.
- Enterprise Context Awareness: Integrated with Unity Catalog to apply governance policies and understand business semantics.
- Adaptive Learning: Improves performance over time using past interactions and coding preferences.
In practice, organisations report efficiency gains. Bernie Graham, VP of Data Engineering at SiriusXM, notes that Genie Code helps teams deliver quality work more efficiently. Emilio Martín Gallardo, Principal Data Scientist at Repsol, observes that the tool supports complex workflows while maintaining oversight and control.
