GitHub Actions Basics for ML

GitHub Actions is the most popular CI/CD platform for open-source ML projects. Let's understand its core concepts and how they apply to ML workflows.

Core Concepts

# .github/workflows/ml-pipeline.yml
name: ML Pipeline              # Workflow name

on:                            # Trigger events
  push:
    branches: [main]
  pull_request:
    branches: [main]
  workflow_dispatch:           # Manual trigger

jobs:                          # Collection of jobs
  train:                       # Job name
    runs-on: ubuntu-latest     # Runner type
    steps:                     # Sequential steps
      - uses: actions/checkout@v4
      - name: Train model
        run: python train.py

Key Components Explained

Component	Purpose	ML Use Case
Workflow	Complete automation pipeline	Full training → deploy flow
Job	Independent unit of work	Training, validation, deployment
Step	Single task within a job	Run script, upload artifact
Runner	Server executing the job	CPU for tests, GPU for training
Action	Reusable step package	checkout, setup-python

Workflow Triggers for ML

Different triggers suit different ML scenarios:

on:
  # Code changes
  push:
    branches: [main, develop]
    paths:
      - 'src/**'
      - 'configs/**'
      - 'requirements.txt'

  # Pull request validation
  pull_request:
    branches: [main]

  # Scheduled retraining
  schedule:
    - cron: '0 2 * * 0'  # Every Sunday 2 AM

  # Manual trigger with inputs
  workflow_dispatch:
    inputs:
      model_version:
        description: 'Model version to train'
        required: true
        default: 'v1.0'

Setting Up Python for ML

jobs:
  train:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'  # Cache pip packages

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

      - name: Run training
        run: python train.py

Environment Variables and Secrets

Store sensitive data securely:

jobs:
  train:
    runs-on: ubuntu-latest
    env:
      # Non-sensitive config
      MODEL_NAME: fraud-detector
      EXPERIMENT_NAME: ci-training

    steps:
      - name: Train with secrets
        run: python train.py
        env:
          # Sensitive data from secrets
          MLFLOW_TRACKING_URI: ${{ secrets.MLFLOW_URI }}
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_KEY }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET }}

Setting Up Secrets

1. Go to repository → Settings → Secrets and variables → Actions
2. Click "New repository secret"
3. Add secrets like MLFLOW_URI, AWS_KEY

Job Dependencies

Control execution order:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - run: pytest tests/

  train:
    needs: test  # Wait for test to pass
    runs-on: ubuntu-latest
    steps:
      - run: python train.py

  validate:
    needs: train  # Wait for training
    runs-on: ubuntu-latest
    steps:
      - run: python validate.py

  deploy:
    needs: [train, validate]  # Wait for both
    runs-on: ubuntu-latest
    steps:
      - run: ./deploy.sh

Conditional Execution

Run steps based on conditions:

steps:
  - name: Deploy to production
    if: github.ref == 'refs/heads/main'
    run: ./deploy-prod.sh

  - name: Deploy to staging
    if: github.ref == 'refs/heads/develop'
    run: ./deploy-staging.sh

  - name: Run expensive tests
    if: github.event_name == 'push'
    run: pytest tests/integration/ --slow

  - name: Notify on failure
    if: failure()
    run: |
      curl -X POST $SLACK_WEBHOOK \
        -d '{"text": "Pipeline failed!"}'

Uploading and Downloading Artifacts

Share files between jobs:

jobs:
  train:
    runs-on: ubuntu-latest
    steps:
      - name: Train model
        run: python train.py --output models/

      - name: Upload model artifact
        uses: actions/upload-artifact@v4
        with:
          name: trained-model
          path: models/
          retention-days: 30

  deploy:
    needs: train
    runs-on: ubuntu-latest
    steps:
      - name: Download model
        uses: actions/download-artifact@v4
        with:
          name: trained-model
          path: models/

      - name: Deploy model
        run: ./deploy.sh models/model.pkl

Workflow Outputs

Pass data between jobs:

jobs:
  train:
    runs-on: ubuntu-latest
    outputs:
      accuracy: ${{ steps.train.outputs.accuracy }}
      model_path: ${{ steps.train.outputs.model_path }}

    steps:
      - name: Train and output metrics
        id: train
        run: |
          python train.py
          echo "accuracy=$(cat metrics.txt | grep accuracy)" >> $GITHUB_OUTPUT
          echo "model_path=models/model.pkl" >> $GITHUB_OUTPUT

  validate:
    needs: train
    runs-on: ubuntu-latest
    steps:
      - name: Check accuracy threshold
        run: |
          if (( $(echo "${{ needs.train.outputs.accuracy }} < 0.85" | bc -l) )); then
            echo "Accuracy below threshold!"
            exit 1
          fi

Complete ML Workflow Example

name: ML Pipeline

on:
  push:
    branches: [main]
  workflow_dispatch:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'
      - run: pip install -r requirements.txt
      - run: pytest tests/unit/

  train:
    needs: test
    runs-on: ubuntu-latest
    outputs:
      accuracy: ${{ steps.metrics.outputs.accuracy }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'
      - run: pip install -r requirements.txt
      - run: python train.py
      - id: metrics
        run: echo "accuracy=$(cat metrics.json | jq .accuracy)" >> $GITHUB_OUTPUT
      - uses: actions/upload-artifact@v4
        with:
          name: model
          path: models/

  validate:
    needs: train
    runs-on: ubuntu-latest
    steps:
      - name: Check accuracy
        run: |
          echo "Model accuracy: ${{ needs.train.outputs.accuracy }}"

Key Insight: Start with a simple workflow, then add complexity. You don't need every feature from day one.

Next, we'll explore training pipelines with matrix builds and GPU runners. :::