GitOps & Continuous Delivery

GitOps Principles

3 min read

GitOps is a set of practices that uses Git as the single source of truth for declarative infrastructure and applications. For platform teams, GitOps provides auditability, consistency, and developer-friendly workflows.

What is GitOps?

GitOps applies DevOps best practices to infrastructure automation:

Traditional Deployment:
┌─────────────────────────────────────────────────────────┐
│  Developer → CI Pipeline → kubectl apply → Cluster      │
│                                                         │
│  Problems:                                              │
│  ├── Who deployed what and when?                       │
│  ├── What's the current desired state?                 │
│  ├── How do we rollback?                               │
│  └── Drift between environments                        │
└─────────────────────────────────────────────────────────┘

GitOps:
┌─────────────────────────────────────────────────────────┐
│  Developer → Git Commit → GitOps Agent → Cluster        │
│                 │                                       │
│           ┌─────┴─────┐                                │
│           │ Git Repo  │ ← Single source of truth       │
│           │ (desired  │                                │
│           │  state)   │                                │
│           └───────────┘                                │
│                                                         │
│  Benefits:                                              │
│  ├── Full audit trail (git history)                   │
│  ├── Easy rollbacks (git revert)                      │
│  ├── PR-based reviews for changes                     │
│  └── Automatic drift correction                        │
└─────────────────────────────────────────────────────────┘

GitOps Principles

The four principles defined by the OpenGitOps project:

# GitOps Principles
principles:

  1_declarative:
    description: "System state is described declaratively"
    example: "YAML manifests, Helm charts, Kustomize"
    benefit: "Reproducible, versionable configuration"

  2_versioned:
    description: "Desired state is stored in Git"
    example: "All changes go through Git commits"
    benefit: "Audit trail, rollback capability"

  3_pulled_automatically:
    description: "Agents pull changes from Git"
    example: "ArgoCD, Flux continuously sync"
    benefit: "No external access to cluster needed"

  4_continuously_reconciled:
    description: "Agents ensure actual matches desired"
    example: "Drift detection and correction"
    benefit: "Self-healing infrastructure"

Repository Structure

Organize your GitOps repository for multi-environment deployments:

# Recommended GitOps repository structure
gitops-repo/
├── apps/                        # Application configurations
│   ├── base/                    # Base configurations (shared)
│   │   ├── user-service/
│   │   │   ├── deployment.yaml
│   │   │   ├── service.yaml
│   │   │   └── kustomization.yaml
│   │   └── order-service/
│   │       └── ...
│   └── overlays/                # Environment-specific overrides
│       ├── development/
│       │   ├── user-service/
│       │   │   ├── kustomization.yaml
│       │   │   └── patch-replicas.yaml
│       │   └── kustomization.yaml
│       ├── staging/
│       │   └── ...
│       └── production/
│           └── ...
├── infrastructure/              # Infrastructure configurations
│   ├── crossplane/
│   │   ├── xrds/
│   │   ├── compositions/
│   │   └── claims/
│   └── platform/
│       ├── monitoring/
│       ├── ingress/
│       └── cert-manager/
├── clusters/                    # Cluster-specific configurations
│   ├── production/
│   │   ├── apps.yaml           # ArgoCD Application
│   │   └── infrastructure.yaml
│   └── staging/
│       └── ...
└── README.md

Kustomize for Configuration Management

Kustomize enables environment-specific overrides without duplication:

# apps/base/user-service/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - deployment.yaml
  - service.yaml
  - configmap.yaml
commonLabels:
  app: user-service
# apps/base/user-service/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-service
spec:
  replicas: 1
  selector:
    matchLabels:
      app: user-service
  template:
    metadata:
      labels:
        app: user-service
    spec:
      containers:
        - name: user-service
          image: user-service:latest
          ports:
            - containerPort: 8080
          resources:
            requests:
              memory: "128Mi"
              cpu: "100m"
            limits:
              memory: "256Mi"
              cpu: "200m"
# apps/overlays/production/user-service/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../../base/user-service
patches:
  - path: patch-replicas.yaml
  - path: patch-resources.yaml
images:
  - name: user-service
    newTag: v1.2.3
# apps/overlays/production/user-service/patch-replicas.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-service
spec:
  replicas: 3
# apps/overlays/production/user-service/patch-resources.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-service
spec:
  template:
    spec:
      containers:
        - name: user-service
          resources:
            requests:
              memory: "512Mi"
              cpu: "500m"
            limits:
              memory: "1Gi"
              cpu: "1000m"

Branch Strategy

Different strategies for managing environments:

# Branch Strategies for GitOps
strategies:

  environment_branches:
    structure:
      main: "Production"
      staging: "Staging"
      development: "Development"
    workflow: "Promote via merge/PR"
    pros: "Clear separation"
    cons: "Branch management overhead"

  directory_based:
    structure:
      main_branch:
        - overlays/development
        - overlays/staging
        - overlays/production
    workflow: "Single branch, directory per env"
    pros: "Simpler branch model"
    cons: "All envs in one branch"
    recommended: true

  repo_per_environment:
    structure:
      gitops-dev: "Development configs"
      gitops-staging: "Staging configs"
      gitops-prod: "Production configs"
    workflow: "Promote via repo sync"
    pros: "Strong isolation"
    cons: "Multiple repos to manage"

Secrets Management

Handle secrets securely in GitOps:

# Secrets Options for GitOps
secrets:

  sealed_secrets:
    tool: "Bitnami Sealed Secrets"
    approach: "Encrypt secrets in Git"
    example: |
      apiVersion: bitnami.com/v1alpha1
      kind: SealedSecret
      metadata:
        name: db-credentials
      spec:
        encryptedData:
          password: AgBy8hC...encrypted...

  external_secrets:
    tool: "External Secrets Operator"
    approach: "Reference secrets from external store"
    example: |
      apiVersion: external-secrets.io/v1beta1
      kind: ExternalSecret
      metadata:
        name: db-credentials
      spec:
        secretStoreRef:
          name: aws-secrets-manager
          kind: SecretStore
        target:
          name: db-credentials
        data:
          - secretKey: password
            remoteRef:
              key: production/db/password

  sops:
    tool: "Mozilla SOPS"
    approach: "Encrypt files with KMS/GPG"
    integration: "ArgoCD + KSOPS plugin"

GitOps Tools Comparison

# GitOps Tools
tools:

  argocd:
    model: "Pull-based"
    ui: "Rich web UI"
    multi_cluster: "Built-in"
    sync: "Application CRDs"
    best_for: "Platform teams, enterprise"

  flux:
    model: "Pull-based"
    ui: "CLI-focused"
    multi_cluster: "Via Kustomization"
    sync: "GitRepository + Kustomization CRDs"
    best_for: "GitOps purists, minimal overhead"

  comparison:
    argocd_strengths:
      - "Better UI for visibility"
      - "ApplicationSets for scale"
      - "Easier onboarding"
    flux_strengths:
      - "Smaller footprint"
      - "More GitOps-native"
      - "Helm controller included"

In the next lesson, we'll set up ArgoCD and explore ApplicationSets for managing deployments at scale. :::