Security Testing & Next Steps

Staying Current

2 min read

AI security evolves rapidly. New attack techniques emerge weekly, and defenses must keep pace. This lesson covers resources, communities, and strategies for staying current.

Why Continuous Learning Matters

┌─────────────────────────────────────────────────────────────┐
│                  AI Security Timeline                        │
│                                                             │
│   2023 Q1: First major prompt injection papers              │
│   2023 Q2: Bing Chat jailbreaks go viral                    │
│   2023 Q3: Indirect injection attacks documented            │
│   2023 Q4: OWASP LLM Top 10 v1.0 released                  │
│   2024 Q1: Multimodal injection attacks emerge              │
│   2024 Q2: Advanced jailbreak techniques proliferate        │
│   2024 Q3: AI agents introduce new attack surfaces          │
│   2024 Q4: OWASP LLM Top 10 v2025 released                 │
│   2025 Q1: ? (New attacks emerge constantly)                │
│                                                             │
│   Your defenses from 6 months ago may be obsolete today    │
└─────────────────────────────────────────────────────────────┘

Essential Resources

Official Standards & Guidelines

Resource Description URL
OWASP LLM Top 10 Authoritative vulnerability list genai.owasp.org/llm-top-10
NIST AI RMF Risk management framework ai.nist.gov
MITRE ATLAS AI attack framework atlas.mitre.org
EU AI Act Regulatory requirements digital-strategy.ec.europa.eu

Security Research & News

Resource Focus Area
Lakera Blog Prompt injection, LLM security
Simon Willison's Blog Practical AI security insights
The Gradient AI/ML research summaries
AI Safety Newsletter Weekly safety updates
r/MachineLearning Community discussions

Tools & Frameworks

Tool Purpose
NeMo Guardrails Production guardrails
LLaMA Guard Safety classification
Garak LLM vulnerability scanner
Rebuff Prompt injection detection
LangChain Security Framework-level security

Building Your Learning Routine

from dataclasses import dataclass
from typing import List
from enum import Enum

class Frequency(Enum):
    DAILY = "daily"
    WEEKLY = "weekly"
    MONTHLY = "monthly"

@dataclass
class LearningActivity:
    name: str
    frequency: Frequency
    time_minutes: int
    sources: List[str]

# Recommended learning routine
learning_routine = [
    LearningActivity(
        name="News & Alerts",
        frequency=Frequency.DAILY,
        time_minutes=15,
        sources=[
            "OWASP announcements",
            "Security researcher Twitter/X",
            "AI safety RSS feeds",
        ]
    ),
    LearningActivity(
        name="Deep Dives",
        frequency=Frequency.WEEKLY,
        time_minutes=60,
        sources=[
            "New attack technique papers",
            "Tool documentation updates",
            "Security blog posts",
        ]
    ),
    LearningActivity(
        name="Hands-On Practice",
        frequency=Frequency.WEEKLY,
        time_minutes=120,
        sources=[
            "CTF challenges",
            "Red team practice",
            "Tool experimentation",
        ]
    ),
    LearningActivity(
        name="Standards Review",
        frequency=Frequency.MONTHLY,
        time_minutes=60,
        sources=[
            "OWASP updates",
            "NIST guidelines",
            "Industry reports",
        ]
    ),
]

def calculate_weekly_time(routine: List[LearningActivity]) -> int:
    """Calculate total weekly learning time."""
    weekly_minutes = 0
    for activity in routine:
        if activity.frequency == Frequency.DAILY:
            weekly_minutes += activity.time_minutes * 5  # Weekdays
        elif activity.frequency == Frequency.WEEKLY:
            weekly_minutes += activity.time_minutes
        elif activity.frequency == Frequency.MONTHLY:
            weekly_minutes += activity.time_minutes / 4

    return round(weekly_minutes)

total_weekly = calculate_weekly_time(learning_routine)
print(f"Recommended weekly learning: {total_weekly} minutes ({total_weekly/60:.1f} hours)")

Tracking New Threats

Setting Up Alerts

# RSS feeds to monitor
SECURITY_FEEDS = [
    "https://genai.owasp.org/feed/",
    "https://www.lakera.ai/blog/rss.xml",
    "https://simonwillison.net/atom/everything/",
    "https://lilianweng.github.io/index.xml",
]

# Keywords to track
ALERT_KEYWORDS = [
    "prompt injection",
    "jailbreak",
    "LLM vulnerability",
    "AI safety",
    "guardrails bypass",
    "model extraction",
    "training data leak",
]

# Set up Google Alerts or similar for these terms
def generate_alert_queries() -> List[str]:
    """Generate search queries for alerts."""
    queries = []
    for keyword in ALERT_KEYWORDS:
        queries.append(f'"{keyword}" site:arxiv.org OR site:github.com')
        queries.append(f'"{keyword}" (CVE OR vulnerability OR attack)')
    return queries

Vulnerability Tracking

from datetime import datetime, timedelta
from typing import Optional

@dataclass
class VulnerabilityNote:
    id: str
    name: str
    discovered: datetime
    severity: str
    affects_your_stack: bool
    mitigation_status: str
    notes: str

class SecurityTracker:
    """Track security developments relevant to your stack."""

    def __init__(self):
        self.vulnerabilities: List[VulnerabilityNote] = []
        self.stack_components = []

    def add_component(self, component: str):
        """Track a component in your stack."""
        self.stack_components.append(component)

    def log_vulnerability(self, vuln: VulnerabilityNote):
        """Log a new vulnerability."""
        self.vulnerabilities.append(vuln)

    def get_unmitigated(self) -> List[VulnerabilityNote]:
        """Get vulnerabilities without mitigation."""
        return [
            v for v in self.vulnerabilities
            if v.mitigation_status != "mitigated" and v.affects_your_stack
        ]

    def weekly_report(self) -> str:
        """Generate weekly security report."""
        week_ago = datetime.now() - timedelta(days=7)
        recent = [
            v for v in self.vulnerabilities
            if v.discovered > week_ago
        ]

        report = f"# Weekly Security Report\n\n"
        report += f"## New Vulnerabilities: {len(recent)}\n\n"

        for vuln in recent:
            report += f"- **{vuln.name}** ({vuln.severity})\n"
            report += f"  - Affects you: {vuln.affects_your_stack}\n"
            report += f"  - Status: {vuln.mitigation_status}\n\n"

        unmitigated = self.get_unmitigated()
        if unmitigated:
            report += f"## Action Required: {len(unmitigated)} unmitigated\n"

        return report

# Usage
tracker = SecurityTracker()
tracker.add_component("OpenAI GPT-4")
tracker.add_component("NeMo Guardrails")
tracker.add_component("LangChain")

# Log new vulnerability
tracker.log_vulnerability(VulnerabilityNote(
    id="2025-001",
    name="New indirect injection vector",
    discovered=datetime.now(),
    severity="high",
    affects_your_stack=True,
    mitigation_status="investigating",
    notes="Affects RAG systems with web scraping"
))

print(tracker.weekly_report())

Community Involvement

Activity Benefit
Join OWASP Access to latest standards, networking
Attend conferences DEF CON AI Village, AI security summits
Contribute to open source Hands-on experience, visibility
Share your findings Build reputation, help community
Join Discord/Slack Real-time discussions, early warnings

Key Takeaway: AI security is a moving target. Dedicate regular time to learning, set up alerts for new threats, and engage with the community to stay ahead of attackers. :::

Quiz

Module 6: Security Testing & Next Steps

Take Quiz