A few weeks ago, I passed the Databricks Machine Learning Professional Exam. I took this exam after completing the Databricks Machine Learning Associate Exam, which you can read about my experience here. The professional exam proved more challenging than the associate level, as it tested more complex concepts. Despite the increased difficulty, I found the preparation more rewarding as I got more in the weeds of model experimentation, deployment, and monitoring. As mentioned in my blog post about passing the associate exam, while the material is well-suited for the Databricks platform, its applicability extends beyond it. With the growing fields of Machine Learning Engineering and MLOps, these topics are becoming essential knowledge—regardless of which platform you use them on.
Exam Information
The exam measures your ability to use Databricks Machine Learing perform advanced machine learning operations. The main pillars are experimentation, model management, deployment and monitoring. The exam guide provided by Databricks provides the breakdown of what the exam assesses.
The exam covers the following topics:
- Experimentation - 30%
- Model Lifecycle Management - 30%
- Model Deployment - 25%
- Solution and Data Monitoring - 15%
Study Process
As with my previous experiences with Databricks certifications, I heavily used the Databricks Learning Academy and Udemy as my main learning sources. For exam preparation, Databricks recommends completing their Machine Learning at Scale and Advanced ML Ops courses. I took these recommended courses along with several other machine learning courses available in the Academy. I also used Udemy's practice exams to test my knowledge. While these practice tests didn't closely mirror the actual exam questions, they proved useful for reviewing the core concepts that the exam assesses.
Topic Breakdown
Experimentation (30%)
This section covers Data Management, Basic and Advanced Experiment Tracking. You should familiarize yourself with how Delta tables work and how they manage data. For topics related to Delta tables, I recommend the book "Delta Lake: Up and Running" by Bennie Haelen and Dan Davis. The book effectively explains how Delta Tables function within a Delta Lake. Understanding Delta Tables is closely tied to Feature Stores, which is another important topic for the exam. You are expected to know what a Feature Store is and what challenges it addresses within ML pipelines. Experiment Tracking encompasses several topics, with MLflow being a key component. The exam included numerous questions about MLflow, particularly focusing on syntax. You'll need to understand MLflow model tracking and logging thoroughly. This includes advanced logging features such as model signatures, input examples, hyperparameters, and artifact logging. Since the syntax is quite unique, make sure to spend time practicing and experimenting with this section.
Model Lifecycle Management (30%)
Model Lifecycle Management covers model processing, management, and lifecycle automation. This section continues the exploration of MLflow with flavors—custom model classes you create with MLflow to apply preprocessing logic. You'll need to understand both their purpose and syntax. I can’t stress this enough, please understand the syntax. The exam asked several questions about MLflow syntax. Model management focuses on the Model Registry, a Databricks Machine Learning-specific component. Key knowledge includes registering models in the Model Registry, adding metadata, and managing different stages and versions. The exam emphasizes syntax knowledge in this area. For lifecycle automation, you should understand both Model Registry Webhooks and Databricks Jobs.
Model Deployment (25%)
This section is all about model deployment strategies. Databricks categorizes these into three types: Batch Deployment, Streaming Deployment, and Real-time Deployment. You need to understand the key differences between these approaches—including their advantages and disadvantages—and how to implement each one on the Databricks platform. The exam will present you with machine learning workflow scenarios, and you will be asked to choose which deployment strategies would work best for each problem.
Solution and Data Monitoring (15%)
The Solution and Data Monitoring section covers processes that occur after machine learning model deployment. This includes detecting various types of drift, implementing monitoring solutions for drift detection, and applying statistical tests to measure distribution changes. I found this section particularly challenging because it introduced unfamiliar concepts related to model deployment. You should know the different types of drift that can occur in your model development pipelines, such as feature and label shift. It's important to know what monitoring solutions you can use in different scenarios. For example, there are different ways to calculate drift in numerical features versus categorical ones. You should also be aware of the different methods for numerical drift detection, such as Jensen-Shannon or Kolmogorov-Smirnov. If you’re interested in further reading on this topics, Chip Huyen’s Designing Machine Learning Systems has some sections about model monitoring that relates to what is in this section as well.
Personal Study Guide
Following my tradition with previous exam summaries, I've created a personal study guide for this certification. You can access it here. The guide includes detailed notes aligned with the exam content and provides coding examples from multiple sources.
Summary
This certification was the most challenging one I studied for, as it introduced many new machine learning topics. The exam thoroughly tests one's ability to perform advanced machine learning tasks in production. A notable advantage of this exam is that its content extends beyond Databricks Machine Learning, despite being platform-specific. The knowledge gained prepares candidates for working across various platforms.