🎓 Free introductory course "LLM evaluations for AI product teams". Save your seat

Open-source AI observability course

Free Open-source AI observability course for data scientists and ML engineers. Join us to learn AI observability production ML monitoring.

Learn at your own pace or sign up to be notified when the next cohort starts.

Sing up for free
Explore course notes
Icon
Evidently AI Open-source ML observability course
about the course

What you will learn

The foundations of ML observability: from exploratory data analysis and model evaluation pre-deployment to continuous production monitoring and debugging.
Save my seat
Icon
Icon
How to evaluate ML models and design ML monitoring strategy.
Icon
How to monitor ML models for batch and real-time systems.
Icon
How to test data quality in production ML pipelines.
Icon
How to monitor quality of NLP and LLM systems.
Icon
How to detect and interpret data and prediction drift.
Icon
How to debug model performance decay.
perks

Why take the course?

Join the course to learn production ML monitoring.
Icon
Free
Icon
40+ on-demand videos
Icon
Practical code examples
Icon
Certificate upon completion

Sign up to the course updates

Save your seat and be notified when the next cohort starts.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
intro video

Welcome to the course

testimonials

What the students say

1200+ students have signed up to take the course.
Stamatis Karlos
Stamatis Karlos
Data Scientist, efood
"Model drift is not a phenomenon that will be eliminated from one day to another in production level. But after that course, someone can be ready to act before critical performance/business issues are raised."
Nissim Matatov
Nissim Matatov
Data Scientist and ML Engineer
"The course covered a wealth of practical MLOps material and featured many insightful discussions led by the course team. I highly recommend incorporating Evidently AI's open-source methodology and products into your MLOps toolbox."
Stamatis Karlos
Stamatis Karlos
Data Scientist, efood
"Model drift is not a phenomenon that will be eliminated from one day to another in production level. But after that course, someone can be ready to act before critical performance/business issues are raised."
Nissim Matatov
Nissim Matatov
Data Scientist and ML Engineer
"The course covered a wealth of practical MLOps material and featured many insightful discussions led by the course team. I highly recommend incorporating Evidently AI's open-source methodology and products into your MLOps toolbox."
Vishal Kumar S.
Vishal Kumar S.
Data Science and ML Enthusiast
"I enjoyed the Open-source ML Observability course, gaining valuable insights into the importance of monitoring throughout the ML pipeline. The practical integration of Evidently with tools like MLflow and Airflow enriched my learning experience, enhancing my skills in ML observability."
Victor Matekole
Victor Matekole
ML Engineer
"The course equipped me to navigate the intricate relationship between machine learning and business goals, giving me a holistic approach to MLOps. Emeli Dral is a great teacher, highly recommended."
Vishal Kumar S.
Vishal Kumar S.
Data Science and ML Enthusiast
"I enjoyed the Open-source ML Observability course, gaining valuable insights into the importance of monitoring throughout the ML pipeline. The practical integration of Evidently with tools like MLflow and Airflow enriched my learning experience, enhancing my skills in ML observability."
Victor Matekole
Victor Matekole
ML Engineer
"The course equipped me to navigate the intricate relationship between machine learning and business goals, giving me a holistic approach to MLOps. Emeli Dral is a great teacher, highly recommended."
modules

Course syllabus

The course is organized into six modules. You can follow the complete course syllabus or pick only the modules that are most relevant to you.
Introduction to ML monitoring and observability

Introduction to ML monitoring and observability

Basics of ML monitoring and observability.
ML monitoring metrics: model quality, data quality, data drift

ML monitoring metrics: model quality, data quality, data drift

Methods and evaluation metrics for structured data.
Monitoring unstructured data: LLM, NLP, and embeddings

Monitoring unstructured data: LLM, NLP, and embeddings

Methods and evaluation metrics for unstructured data.
Designing effective ML monitoring

Designing effective ML monitoring

Customizing ML monitoring to your model.
ML pipelines validation and testing

ML pipelines validation and testing

Deploying an end-to-end pipeline to check data and ML model quality.  
Deploying an ML monitoring dashboard

Deploying an ML monitoring dashboard

How to deploy an ML monitoring service and design a live dashboard.  
Icon
Icon
Sign up for free
Icon
LEARN now

Course materials

We published all 40 lessons with videos, course notes, and code examples so you can learn at your own pace.
Jump to the course website to start learning now!
Evidently AI Open-source ML observability courseExplore course notes
team

Meet course authors

Emeli Dral
LinkedIn logo

Emeli Dral

Co-founder and CTO
Evidently AI

Emeli Dral is a Co-founder and CTO at Evidently AI, a startup developing open-source tools to evaluate, test, and monitor the performance of machine learning models.
‍
Earlier, she co-founded an industrial AI startup and served as the Chief Data Scientist at Yandex Data Factory. She led over 50 applied ML projects for various industries: from banking to manufacturing. Emeli is a data science lecturer at GSOM SpBU and Harbour.Space University. She is a co-author of the Machine Learning and Data Analysis curriculum at Coursera with over 150,000 students.

Elena Samuylova is a CEO and Co-founder at Evidently AI, a Y Combinator-backed startup developing open-source tools to evaluate, test, and monitor the performance of machine learning models.

‍

She has been active in the applied machine learning space since 2014. Previously, she co-founded and served as a CPO of an industrial AI startup. She worked with global metal and chemical companies to implement machine learning for production optimization. Prior to that, she led business development at Yandex Data Factory, an enterprise AI division of Yandex. She focused on delivering ML-based solutions to retail, banking, telecom, and other industries. In 2018, Elena was named 50 Women in Product Europe by Product Management Festival.

Faq

Have a question or need help?

Is the course free?
Icon

Yes, the course is 100% free.

What is the course format?
Icon

The course is organized into six modules. Each module consists of on-demand videos and practical code examples.

To earn a certificate of completion, you must enroll in the course cohort. Sign up to save your seat and be notified when the next cohort is ready to launch.  

This is optional — you can also learn at your own pace and go through the course materials, which are publicly available.

How long does it take to complete the course?
Icon

The course lasts seven weeks. There are six modules with learning materials, and one extra week to complete the final assignment.

However, you can also learn at your own pace — whatever works best for you!

Will I get a course certificate?
Icon

You will receive a certificate if you enroll in the course cohort and successfully complete all the assignments.

Note that the option to receive the certificate is available only to those who participate in the course cohort. The next cohort will take place in 2024. Sign up to get updates when it starts.

Are there any course prerequisites?
Icon

There are both theoretical and code-focused modules that require knowledge of Python. But no worries: we will walk you through the code! You can also skip these parts and still learn a lot.

What if I need help?
Icon

Have a question or just want to say “Hi”? Jump to our Discord#-ml-observability-course channel to chat with fellow learners and get support from the course team.

Where can I find course materials?
Icon

We published all 40 lessons with videos, course notes, and code examples. Jump to the course website to start learning now!

Is there a newsletter?
Icon

Yes, sign up here to receive course updates and be notified when the next cohort is ready to launch.

Will course materials still be available after the course?
Icon

Yes! All course materials are public so you can get back to them at your convenience, during or after the course.

Evaluate, test and monitor your AI-powered products.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
LLM observabilityML observabilityOpen-source
BlogTutorialsGuidesML platformsML use casesML observability course
PricingDocsGitHubCommunity
© 2024, Evidently AI. All rights reserved.
Twitter logoDiscord logoYouTube logo
By clicking “Accept”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.
DenyAccept