Database migrations in GitOps need a Job, not a pipeline

A database migration has two simple requirements:

• it runs once;
• it runs before the new version of your application starts.

In a classic deploy pipeline that’s easy: add a migration step before the deploy step. With GitOps there’s no pipeline. Flux watches a Git repository and makes the cluster match it. There’s no “before” unless you create one.

The tempting options that don’t work

Run migrations when the app starts. Your app probably runs multiple replicas, so several pods would try to migrate the same database at the same time. Some frameworks lock the database to prevent that. Many don’t, and you don’t want to find out which one yours is during a production deploy.

Use an initContainer. Same problem: every replica runs it.

Keep a pipeline just for migrations. Now half of your deploy is GitOps and half isn’t, and they don’t know about each other.

The pattern: a Job with its own Kustomization

Run the migration as a normal Kubernetes Job. Put it in a separate Flux Kustomization, and make the application’s Kustomization depend on it:

Git commit (new image tag)
        │
        ▼
Kustomization: myapp-migrate ── runs Job, waits until it completes
        │  dependsOn
        ▼
Kustomization: myapp ── rolls out the new version

Flux won’t start rolling out the application until the migration has finished successfully.

The migration Job

The Job runs your application image with the migration command:

# myapp-migrate/job.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: migrate-db
spec:
  backoffLimit: 2
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: migrate-db
          image: myapp
          command: ["make", "migrate"]

With a small kustomization.yaml to set the namespace and image:

# myapp-migrate/kustomization.yaml
namespace: myapp
resources:
  - job.yaml
images:
  - name: myapp
    newName: ghcr.io/acme/myapp
    newTag: "1.4.2"

Use the same image tag as the application, so the migration and the code that expects it always ship together. If you use Flux image automation, let it update both.

The two Flux Kustomizations

Here’s where it comes together:

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: myapp-migrate
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./myapp-migrate
  prune: true
  force: true     # recreate the Job when its image changes
  wait: true      # only "ready" once the Job has completed
  timeout: 5m
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: myapp
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./myapp
  prune: true
  dependsOn:
    - name: myapp-migrate

Three settings do the work:

• force: true. A Job’s pod template is immutable, so Flux can’t simply update the image of an existing Job. With force, Flux deletes and recreates it when an immutable field changes, which starts a fresh migration run.
• wait: true. Flux marks the Kustomization as ready only when its resources are healthy. For a Job, that means completed.
• dependsOn. The application waits until myapp-migrate is ready. If the migration fails, the new version never rolls out, and the old one keeps running.

What to watch out for

This pattern guarantees order, not magic. Keep two things in mind.

Migrations must be backwards compatible. While the migration runs, and during the rollout afterwards, the old version of your app is still serving traffic against the new schema. Use the expand-and-contract approach: add columns first, remove old ones in a later release.

A failed migration blocks the deploy. That’s the point, but someone needs to notice. Make sure Flux alerts reach a channel people actually read, so a stuck myapp-migrate doesn’t go unseen for a day.

That’s all it takes. No extra tooling, no pipeline next to your GitOps setup, just a Job, a dependency and two flags.