🎓 Free introductory course "LLM evaluations for AI product teams". Save your seat

LEARN

AI quality and MLOps guides

Learn about AI observability, ML and LLM evaluation and MLOps with our in-depth guides.

course

LLM evaluations for AI product teams

Building an LLM-powered product? Sign up for our free course on LLM evaluations for AI product teams. A gentle introduction to evaluating LLM-powered apps, no coding knowledge required.

Save my seat

LLM GUIDE

A complete guide to LLM quality

How to evaluate the quality of generative outputs and LLM-based systems? In this guide, we break down different evaluation methods and metrics for AI-powered products.

LLM-as-a-judge

How to build an LLM evaluator and craft good prompts.

LLM benchmarks

20 commonly used LLM benchmarks, how they work, and why we need them.

Explore topics

course

Open-source ML observability course

Want to learn production ML monitoring? Sign up for our Open-source ML observability course for data scientists and ML engineers. It's free!

Save my seat

View course notes

MACHINE LEARNING IN PRODUCTION

A complete guide to ML in production

How to maintain ML models once after you deploy them, and what exactly to prepare for? In this guide, we look into the key concepts that relate to production ML model operations.

ML Monitoring metrics

ML monitoring metrics in production

"ML monitoring" can mean many things. Are you tracking service latency? Model accuracy? Data quality? This guide organizes everything one can look at in a single framework.

See the guide

Ranking and recommendations metrics in ml

A complete guide to ranking quality metrics

How do you evaluate the quality of ranking and recommender systems? In this guide, we look into the key ranking metrics and explain them step by step.

Metrics overview

10 metrics to evaluate recommender and ranking systems.

Precision and Recall at K

How to use Precision and Recall at K to assess the performance of ranking and recommender systems.

Mean Average Precision (MAP)

How to compute MAP to evaluate the quality of ranking and recommender systems.

Normalized Discounted Cumulative Gain (NDCG)

How to use NDCG at K to measure the ranking quality for both binary and graded relevance scores.

Mean Reciprocal Rank (MRR)

How to use MRR at K to assess the ranking quality.

Explore topics

DATA Drift detection

5 methods to detect data drift on large datasets

We ran an experiment to help build an intuition on how popular drift detection methods behave. In this guide, we share the key takeaways and the code to run the tests on your data.

See the guide

5 methods to detect drift in ML embeddings

Monitoring embedding drift is relevant for the production use of LLM and NLP models. We ran experiments to compare 5 drift detection methods. Here is what we found.

See the guide

Classification metrics in ml

A complete guide to classification metrics in ML

How to evaluate the quality of a classification model? In this guide, we break down different machine learning metrics for binary and multi-class problems.