Ad hoc visual analysis does not scale well when you have multiple models in production and need to monitor them in an automated fashion.
Still, it can be pretty helpful for the following:
1. Evaluate the tool if you are new to Evidently. Just come check out our examples in the
documentation and get a taste of what Evidently can do.
2. Evaluate an ML model before deployment. Even drift analysis can be handy before you deploy a model in production. You can do that to
learn the past patterns and define your monitoring strategy. You can also evaluate the performance of the model: for example, to
choose between the two models with similar performance.
3. Report-based monitoring. You can generate regular reports, for example, weekly, to
check on your model performance. It might help keep tabs closely on a new model or share the summary with other stakeholders. If you have batch models, you might not need live dashboards at all: just schedule the Evidently reports with a
tool like Airflow.
4. When you are debugging the model decay. If the model drift is detected, you need to drill down the root cause and figure out how to address it. Ad hoc visual analysis is often the first step. You can use the
pre-built Evidently reports as a starting point.
And if you want the live dashboards, don't miss out on our
recent Grafana integration.