AI Product Metrics & Evaluation
Cost Management & ROI
5 min read
AI features can be expensive to run. Understanding the economics helps you make better product decisions.
AI Cost Components
API Costs (Pay-Per-Use)
| Component | What It Means | Typical Cost |
|---|---|---|
| Input tokens | Text you send to the model | $1-15 per 1M tokens |
| Output tokens | Text the model generates | $3-60 per 1M tokens |
| Fine-tuning | Custom model training | $8-25 per 1M training tokens |
| Storage | Storing fine-tuned models | $0-5 per model/day |
Token estimation rule of thumb:
- 1 token ≈ 4 characters (English)
- 100 words ≈ 75-100 tokens
- 1 page of text ≈ 500-800 tokens
Self-Hosted Costs
| Component | Monthly Range |
|---|---|
| GPU infrastructure | $2,000-50,000+ |
| ML engineering time | $15,000-40,000 |
| MLOps tools | $500-5,000 |
| Monitoring/observability | $200-2,000 |
Calculating Cost Per User Action
Step 1: Estimate Token Usage
Average input: 500 tokens
Average output: 200 tokens
Calls per user action: 2 (e.g., generate + refine)
Total tokens per action: (500 + 200) × 2 = 1,400 tokens
Step 2: Apply Pricing
Using GPT-5.4 Mini pricing (illustrative, early 2026):
Input: 500 × 2 = 1,000 tokens × $0.0025/1K = $0.0025
Output: 200 × 2 = 400 tokens × $0.01/1K = $0.004
Total per action: $0.0065
Step 3: Scale to Usage
Monthly active users: 100,000
Actions per user/month: 20
Total actions: 2,000,000
Monthly AI cost: 2,000,000 × $0.0065 = $13,000
Cost per user: $0.13/month
Cost Optimization Strategies
Strategy 1: Choose the Right Model
| Task Complexity | Model Tier | Example |
|---|---|---|
| Simple classification | Small/Fast | GPT-5.4 Mini, Claude Haiku 4.5 |
| Standard generation | Medium | GPT-5.4, Claude Sonnet 4.6 |
| Complex reasoning | Large | GPT-5.4 Pro, Claude Opus 4.6 |
Impact: Switching from GPT-5.4 to GPT-5.4 Mini can reduce costs by 90%+ for simple tasks.
Strategy 2: Optimize Prompts
| Technique | Savings |
|---|---|
| Shorter system prompts | 10-30% |
| Remove unnecessary context | 20-40% |
| Use structured outputs | Varies |
Strategy 3: Cache Responses
If same input → Return cached response
Else → Call API → Cache result
Cache hit rate of 30% = 30% cost reduction
Strategy 4: Batch Processing
Real-time vs batch trade-off:
| Approach | Cost | Latency |
|---|---|---|
| Real-time API | Higher | <1 second |
| Batch API | 50% lower | Hours |
Use batch for: Reports, analytics, background processing
Strategy 5: Fallback to Cheaper Models
Try: Small model (fast, cheap)
If confidence < threshold:
Escalate to: Large model (slow, expensive)
Typical result: 70-80% handled by cheap model = major savings.
Calculating ROI
ROI Formula for AI Features
ROI = (Value Generated - Total Cost) / Total Cost × 100%
Value Sources
| Value Type | How to Measure |
|---|---|
| Revenue increase | Conversion lift × Average order value |
| Cost reduction | Hours saved × Labor cost |
| Efficiency gain | Tasks automated × Previous cost |
| Risk reduction | Prevented losses |
Example: AI Customer Support
Costs:
- AI API costs: $5,000/month
- Implementation: $20,000 (one-time)
- Maintenance: $2,000/month
- Year 1 total: $20,000 + ($7,000 × 12) = $104,000
Value:
- Support tickets resolved by AI: 40%
- Previous cost per ticket: $15
- Monthly tickets: 10,000
- Savings: 10,000 × 0.4 × $15 = $60,000/month
- Year 1 total: $720,000
ROI:
ROI = ($720,000 - $104,000) / $104,000 × 100%
ROI = 592%
Presenting AI Business Cases
What Executives Want to Know
| Question | Your Answer Should Include |
|---|---|
| "What does it cost?" | Upfront + ongoing, per-user breakdown |
| "What's the return?" | Specific metrics, conservative estimates |
| "What's the risk?" | Failure scenarios, mitigation plans |
| "When do we break even?" | Timeline to positive ROI |
| "What if it doesn't work?" | Rollback plan, sunk costs limited |
Business Case Template
AI Feature: [Name]
Investment Required:
- Development: $X (one-time)
- Operations: $Y/month
- Year 1 Total: $Z
Expected Value:
- [Metric 1]: $A/month (conservative)
- [Metric 2]: $B/month (conservative)
- Year 1 Total: $C
ROI: X%
Break-even: Month N
Risks & Mitigations:
- Risk 1: [Mitigation]
- Risk 2: [Mitigation]
Decision Needed: [Specific ask]
Key Takeaway
AI ROI depends on matching the right model to the task, optimizing usage, and clearly connecting costs to business value. Always start with the business outcome, work backward to the AI investment needed.
Up next: Test your understanding with Module 3 Quiz. :::