MLOps Maturity Levels

Not every organization needs full automation from day one. Understanding maturity levels helps you identify where you are and where to focus improvement efforts.

The Three Levels

Google's MLOps maturity model defines three levels:

Level	Name	Characteristics
0	Manual	Data scientists run notebooks manually
1	ML Pipeline	Automated training, manual deployment
2	CI/CD + CT	Full automation with continuous training

Level 0: Manual Process

Most teams start here. It's fine for experimentation, but not for production.

┌─────────────────────────────────────────────────┐
│                  Level 0: Manual                 │
├─────────────────────────────────────────────────┤
│  Data Scientist                                  │
│  ┌─────────┐  ┌─────────┐  ┌─────────┐         │
│  │Notebook │─▶│ Train   │─▶│ Export  │─▶ Model │
│  └─────────┘  └─────────┘  └─────────┘         │
│                                                  │
│  Engineer manually deploys model                 │
└─────────────────────────────────────────────────┘

Characteristics:

• Manual, script-driven experiments
• No experiment tracking
• Model handoff via files (pickle, ONNX)
• Rare releases (quarterly or less)
• No monitoring or retraining triggers

When it's okay: Proof of concepts, research, single-use models

Level 1: ML Pipeline Automation

Training becomes automated and reproducible.

┌─────────────────────────────────────────────────┐
│              Level 1: ML Pipeline                │
├─────────────────────────────────────────────────┤
│                                                  │
│  ┌──────┐  ┌──────┐  ┌──────┐  ┌──────┐        │
│  │ Data │─▶│Train │─▶│Validate│─▶│Model │       │
│  └──────┘  └──────┘  └──────┘  └──────┘        │
│      │                              │            │
│      └──────── Orchestrator ────────┘            │
│           (Kubeflow, Airflow)                    │
│                                                  │
│  Manual deployment trigger                       │
└─────────────────────────────────────────────────┘

Characteristics:

• Automated data validation
• Experiment tracking (MLflow, W&B)
• Reproducible training pipelines
• Feature store integration
• Manual deployment decisions

Key additions:

• Pipeline orchestration (Kubeflow, Airflow)
• Data and model versioning (DVC)
• Feature stores (Feast)
• Model registry (MLflow)

Level 2: CI/CD + Continuous Training

Full automation with production-grade practices.

┌─────────────────────────────────────────────────┐
│         Level 2: CI/CD + Continuous Training     │
├─────────────────────────────────────────────────┤
│                                                  │
│  ┌──────────────────────────────────────┐       │
│  │         ML Pipeline (Automated)       │       │
│  │  Data → Train → Validate → Register   │       │
│  └──────────────────────────────────────┘       │
│                      │                           │
│                      ▼                           │
│  ┌──────────────────────────────────────┐       │
│  │         CI/CD Pipeline                │       │
│  │  Test → Build → Deploy → Monitor      │       │
│  └──────────────────────────────────────┘       │
│                      │                           │
│                      ▼                           │
│  ┌──────────────────────────────────────┐       │
│  │      Continuous Training Triggers     │       │
│  │  Schedule │ Data drift │ Performance  │       │
│  └──────────────────────────────────────┘       │
└─────────────────────────────────────────────────┘

Characteristics:

• Automated retraining triggers
• Model testing in CI
• Canary/shadow deployments
• A/B testing infrastructure
• Comprehensive monitoring and alerting

Assessment Checklist

Use this to evaluate your current level:

Capability	L0	L1	L2
Experiment tracking	❌	✅	✅
Automated training	❌	✅	✅
Data versioning	❌	✅	✅
Feature store	❌	✅	✅
Model registry	❌	✅	✅
CI/CD for ML	❌	❌	✅
Automated retraining	❌	❌	✅
A/B testing	❌	❌	✅
Drift monitoring	❌	❌	✅

Progression Strategy

Don't jump to Level 2 immediately. Progress incrementally:

1. Start with versioning - DVC for data and models
2. Add experiment tracking - MLflow or W&B
3. Build pipelines - Kubeflow or Airflow
4. Integrate feature stores - Feast for consistency
5. Add CI/CD - Automated testing and deployment
6. Enable CT - Automated retraining triggers

Key insight: Most organizations benefit most from going from Level 0 to Level 1. The ROI is highest there.

Next, we'll explore infrastructure patterns for training vs serving workloads. :::