Working with ML Engineering Teams

ML engineers think differently than traditional software engineers. Understanding their world helps you collaborate effectively.

The PM-ML Communication Gap

What You Say	What They Hear	Better Way to Say It
"Can we make this smarter?"	Vague, undefined request	"Can we improve precision from 85% to 90%?"
"Why is this taking so long?"	They're slow	"What are the blockers? How can I help unblock?"
"Users want better results"	Undefined success	"Users report 20% of recommendations are irrelevant"
"Just use AI for this"	Naively simple	"Would ML be appropriate here? What would it take?"

Key ML Terms for PMs

Model Metrics

Term	Definition	Why It Matters
Precision	Of all predicted positives, how many were correct?	High precision = fewer false alarms
Recall	Of all actual positives, how many did we catch?	High recall = fewer missed cases
F1 Score	Balance of precision and recall	Overall model quality
Accuracy	Overall correctness	Can be misleading with imbalanced data

PM Rule: You usually can't maximize both precision AND recall. Know which matters more for your use case.

Data Concepts

Term	Definition	Why It Matters
Training data	Data used to teach the model	Quality in = quality out
Validation data	Data to tune the model	Prevents overfitting
Test data	Data for final evaluation	Never seen during training
Data drift	When real data changes over time	Model performance degrades
Ground truth	The correct labels	Defines what "right" means

Development Stages

Stage	What Happens	Typical Duration
Data exploration	Understand what data exists	1-2 weeks
Feature engineering	Create inputs for the model	2-4 weeks
Model training	Teach the model	1-2 weeks
Evaluation	Test model quality	1 week
Deployment	Put into production	1-2 weeks
Monitoring	Track ongoing performance	Ongoing

What ML Teams Need from PMs

1. Clear Problem Definition

Bad: "We need AI to improve search"

Good:

• "Users abandon search when no results match their intent"
• "Success = users click on a result within top 5"
• "Current baseline: 60% click-through on top 5"
• "Target: 80% click-through on top 5"

2. Labeled Data (or Help Getting It)

ML models need examples of "correct" answers.

How you can help:

• Provide historical data with outcomes
• Set up labeling processes
• Define edge cases and how to handle them
• Prioritize data quality initiatives

3. Realistic Timelines

ML development is iterative, not linear.

Phase	What Can Go Wrong	Buffer
Data preparation	Data is messier than expected	2x time
Model development	First model doesn't work	3-5 iterations
Production deployment	Integration challenges	2x time

4. Tolerance for Uncertainty

ML can't guarantee outcomes. Help stakeholders understand:

• "We'll aim for 90% accuracy but may land at 85%"
• "First version will be MVP, we'll iterate"
• "We need production data to truly optimize"

Questions to Ask in ML Kickoffs

About data:

• "What data do we need that we don't have?"
• "How much labeled data exists?"
• "What's our data refresh strategy?"

About approach:

• "Are we using pre-trained models or training from scratch?"
• "What's the simplest baseline we can compare against?"
• "What are the biggest technical risks?"

About success:

• "What accuracy can we realistically expect?"
• "How will we know if the model is degrading?"
• "What's our rollback plan?"

Red Flags in ML Projects

Red Flag	What It Means	Action
"We need more data" (repeatedly)	Fundamental problem with approach	Re-evaluate if ML is right
"The model works in testing but not production"	Data drift or training issues	Investigate data pipeline
No baseline comparison	Can't prove ML adds value	Establish baseline first
"Just a few more weeks"	Scope creep or stuck	Set hard deadline, ship MVP
Accuracy keeps changing	Unstable model	Review evaluation methodology

Setting Up ML Projects for Success

Pre-Kickoff Checklist

• Problem defined with measurable success criteria
• Data inventory completed (what exists, what's missing)
• Baseline established (current solution performance)
• Timeline includes buffer for iteration
• Stakeholders aligned on uncertainty

During Development

• Weekly syncs on progress and blockers
• Review intermediate results (don't wait for final)
• Adjust scope based on learnings
• Document decisions and trade-offs

Post-Launch

• Monitoring dashboards in place
• Alert thresholds defined
• Feedback loop established
• Retraining schedule planned

Key Takeaway

ML teams are partners, not implementers. Invest in understanding their constraints and providing clear requirements. The best AI products come from PM-ML collaboration, not handoffs.

---

Up next: Test your understanding with Module 2 Quiz. :::