Insights Agent & Analytics

LangSmith's Insights Agent automatically analyzes your traces to discover patterns, failure modes, and optimization opportunities you might miss manually.

What is the Insights Agent?

The Insights Agent is an AI assistant that:

• Analyzes patterns across thousands of traces
• Identifies failure modes and anomalies
• Suggests improvements to your LLM application
• Answers questions about your production data

Note: Insights Agent is available on Plus and Enterprise plans.

Accessing Insights

In the LangSmith UI:

1. Navigate to your project
2. Select the Insights tab
3. Ask questions in natural language

Example Questions

Ask the Insights Agent:

Question Type	Example
Performance	"Which queries have the highest latency?"
Failures	"What are the most common error patterns?"
Quality	"Which responses received negative feedback?"
Trends	"How has response quality changed this week?"
Comparison	"Compare performance between model versions"

Automatic Pattern Discovery

The Insights Agent can identify:

Discovered Patterns:
├── Queries containing "refund" have 40% lower satisfaction
├── Responses over 500 tokens take 3x longer
├── Weekend traffic shows different topic distribution
└── Model timeout rate increased 15% after deployment

Analytics Dashboard

Beyond the Insights Agent, LangSmith provides:

Trace Analytics

• Volume: Requests over time
• Latency: Response time distribution
• Errors: Failure rates and types
• Cost: Token usage and spend

Filtering and Segmentation

Filter traces by:

# Example filters in the UI
project = "production"
model = "gpt-4o"
latency > 2000ms
feedback_score < 3
contains_error = true

Custom Tags for Segmentation

Add tags to enable better analytics:

@traceable(tags=["customer_support", "tier_1", "english"])
def handle_support_request(query: str) -> str:
    # Your logic here
    pass

Then segment analytics by these tags in the dashboard.

Alerting

Set up alerts for:

• Latency spikes above threshold
• Error rate increases
• Quality score drops
• Cost anomalies

Alerts can notify via:

• Email
• Slack
• Webhooks

Best Practices

1. Tag consistently: Use standard tags across your application
2. Review weekly: Check Insights for emerging patterns
3. Set baselines: Establish normal metrics before alerting
4. Act on insights: Create action items from discoveries

Tip: Start by asking the Insights Agent "What are the biggest issues in my application this week?"

Next, we'll explore how to evaluate multi-turn conversations in LangSmith. :::