DowntownSan Francisco was buzzing the week of June 10th as 16,000 people gathered at the Moscone Center for Databricks' annual conference (DAIS). The event brought together attendees from a diverse group of companies — from Fortune 10 giants to startups — all united by a single vision: how to become a Data + AI company in 2024 and beyond.
Databricks has long championed a clear mission: democratize data and AI, helping every company transition into a Data + AI enterprise. During CEO Ali Ghodsi's keynote, he highlighted three primary challenges that companies face in this transformation journey:
- Universal Desire for AI: How can organizations get AI into production?
- Security and Privacy Pressures: How can data security and privacy standards be met?
- Fragmented Data Estates: How can companies unify their data estate?
The Data Intelligence Platform
Databricks' answer to these challenges is the Data Intelligence Platform—an end-to-end solution for data and AI. The platform enables organizations to effectively collect, integrate, manage, analyze, and derive insights from their data.
Key Product Announcements
Databricks made several significant product announcements to support their vision of democratizing data and AI for enterprises:
Mosaic AI and DBRX
GenerativeAI was a major focus this year. Databricks announced the complete integration of Mosaic AI into its platform, featuring:
- Mosaic AI Model Training: A zero-code solution for fine-tuning foundational models with private data.
- Mosaic AI Agent Framework: A resource for authoring, deploying, and testing Retrieval-Augmented Generation (RAG) applications using foundational models and enterprise data.
- Mosaic AI Agent Evaluation: An AI-assisted tool for measuring AI outputs, collecting feedback, and resolving model issues.
- Mosaic AI Gateway: A centralized management point for controlling the availability of Large Language Models (LLMs), simplifying the process of switching between LLMs without complex code changes.
- Mosaic AI Tool Catalog: A resource for governing, sharing, and registering AI tools, managed using Databricks Unity Catalog.
- DBRX: The LLM Databricks launched earlier this year that they built entirely on the Databricks platform was talked about a lot in the conference, and will be a very viable option for Enterprises to leverage as they look to train their own data on language models.
Unity Catalog Goes Open Source
Unity Catalog, Databricks' premier data governance solution, is now open source. This move is expected to foster greater collaboration and innovation within the data community, making data governance more accessible and flexible compared to the proprietary solutions offered by most vendors.
Databricks Goes Serverless
The entire Databricks platform is now serverless. Starting July 1, customers will only pay for what they use, eliminating the need for managing compute clusters. This shift aims to enhance cost efficiency and simplify setup.
AI/BI Enhancements
Databricks introduced an enhanced BI solution powered by AI, featuring:
- AI/BI Dashboards: An AI-powered, low-code dashboard solution with conventional BI capabilities for fixed business questions.
- Genie: A conversational interface that continuously learns from human feedback, providing answers to a broad set of business questions with certified responses for specific query patterns.
Tabular Acquisition
Databricks acquired Tabular, bringing together the creators of Apache Iceberg and Linux Foundation Delta Lake. This acquisition aims to enhance data compatibility, ensuring customers are no longer constrained by their lakehouse format.
Partnership with NVIDIA
NVIDIA was named Databricks' AI Partner of the Year. The two companies announced an expanded collaboration to optimize data and AI workloads by integrating NVIDIA CUDA accelerated computing into Databricks' core. This partnership aims to enhance the performance and efficiency of data processing and AI applications.
What’s Next
At Origin, we share Databricks' belief that every company needs to become a Data + AI enterprise. As companies embark on or continue their data and AI journeys, Databricks has established itself as a market leader, and Origin is here to help organizations leverage this powerful platform successfully. Contact us today if you’d like to learn more about how to become a Data + AI company, at hello@origindigital.com.
.svg)