MLOps: Automated Pipelines and Model Monitoring

Get oriented with the ML Ops course, its goals, prerequisites, and the tools you will use throughout. Introduce the big picture and the course roadmap

Collect, store, and version datasets for reproducible ML pipelines with traceability from raw data to training-ready sets, and validate them to detect ML-specific quality concerns.

Ingest a dataset from a public API, store it locally, track dataset versions using DVC and Git Validate the dataset using Great Expectations

Learn how feature stores solve train-serve skew by providing consistent feature definitions and serving.

Define feature views in Feast, generate point-in-time training datasets, and retrieve features for inference.

Learn the principles of experiment tracking and how MLflow organizes parameters, metrics, and artifacts.

Instrument a training script to log parameters, metrics, and models to MLflow, then compare runs in the UI.

Learn CI/CD principles applied to ML, including triggers, orchestration, and automated retraining strategies.

Create a CI/CD pipeline that detects new data, retrains a model and validates it

Learn how model registries provide versioning, stage transitions, and governance for production ML models.

Register models, transition them through lifecycle stages, and build applications that load production models.

Learn what containers are, why they solve environment inconsistency, and how Docker packages ML model APIs.

Package a trained model and its FastAPI service into a Docker container, then build, run, and push the image.

Learn how to design model test suites that validate performance, robustness, and fairness before deployment.

Create automated test suites that evaluate model performance on subpopulations and block flawed deployments.

Learn how to deploy containerized models as APIs in production

Deploy a containerised model API to AWS, verify endpoint responses, and apply a cost optimization strategy to reduce serving costs.

Learn how to automate deployment pipelines for ML models, including CI/CD principles, canary release patterns, and automated rollback strategies

Build CI/CD pipeline that automatically deploys a containerized ML model from registry

Learn to monitor deployed ML model APIs for latency, error rates, resource usage and trigger alerts

Implement monitoring for a deployed ML model API, including metrics collection, dashboards, and alerting.

Learn the statistical foundations of data drift, how it differs from concept drift, and why monitoring matters.

Generate drift reports comparing training and production data, identify drifted features, and set up alerts.

Learn explainability methods like SHAP and LIME, the difference between local and global explanations, and their limits.

Create an API endpoint that returns SHAP-based feature importance explanations for individual predictions.

Learn fairness metrics like demographic parity and equalized odds, and how to evaluate bias across groups.

Assess model fairness across subgroups, generate fairness reports, and set up automated regression alerts.

Learn how managed cloud ML services integrate into a cohesive ML Ops lifecycle from training to monitoring.

Create a full ML Ops pipeline on SageMaker with automated training, deployment, auto-scaling, and monitoring.

Review the skills gained throughout the ML Ops course and explore next steps for applying them in production.