How to fix CrashLoopBackOff in Kubernetes

CrashLoopBackOff is not an error. It is Kubernetes telling you it has given up restarting your container for now and is sitting out a waiting period before it tries again. The actual failure already happened, and the status has replaced it.

That distinction is the whole reason people get stuck. You cannot fix CrashLoopBackOff, because nothing is broken called CrashLoopBackOff. You fix whatever made the container exit, and the status resolves itself.

The 30-second version

kubectl describe pod <pod> | grep -A5 "Last State"
kubectl logs <pod> --previous

The first command gives you the exit code. The second gives you what the container said on its way out — --previous is essential, because kubectl logs without it reads the container that is currently starting, which usually has nothing in it yet.

Those two commands answer the question in most cases. Everything below is what to do with the answer.

Read the exit code first, logs second

The exit code narrows the problem from “something is wrong” to one of about five things, and it takes two seconds to read.

Zoom

Exit code 0 is the one that confuses people

A container that exits 0 is still restarted if restartPolicy is Always, which is the default and is what a Deployment always uses. So a perfectly successful one-shot script in a Deployment produces an endless, entirely healthy-looking crash loop.

If the workload is genuinely a task that finishes, it should be a Job, not a Deployment. If it is meant to stay up, then your entrypoint is exiting when you think it is serving.

Exit code 137 is two different problems

137 is 128 + 9 — SIGKILL. The container was killed, not asked to leave. There are two common reasons and they need opposite fixes, so check which one:

kubectl get pod <pod> -o jsonpath='{.status.containerStatuses[*].lastState.terminated.reason}'

If that prints OOMKilled, the container exceeded its memory limit. Raise resources.limits.memory, or find out why the process is using more than you budgeted. Note that OOMKilled is reported against the container’s limit — a pod can be OOMKilled on a node with plenty of free memory.

If it prints Error instead, the most likely culprit is a liveness probe. The kubelet restarts a container whose liveness probe fails, and if the probe’s initialDelaySeconds is shorter than your app’s real startup time, the kubelet kills it before it has ever finished booting — forever. This produces a crash loop in an application with no bug in it at all.

The fix for slow starters is a startupProbe, not a longer initialDelaySeconds:

startupProbe:
  httpGet: { path: /healthz, port: 8080 }
  failureThreshold: 30
  periodSeconds: 10   # allows up to 5 minutes to start
livenessProbe:
  httpGet: { path: /healthz, port: 8080 }
  periodSeconds: 10   # only begins once the startup probe has succeeded

Liveness probes are disabled while a startup probe is still running, which is exactly the behaviour you want and cannot get by tuning delays.

When the logs are empty

An empty --previous log is information, not a dead end. It means the container died before the application produced output — so the application is probably not the problem. In rough order of likelihood:

1. A missing environment variable or secret. A configMapKeyRef or secretKeyRef pointing at something that does not exist stops the container before it starts. This usually shows as CreateContainerConfigError rather than CrashLoopBackOff, so if you see that status, stop reading here and go check your references.
2. A volume that did not mount. Check the pod’s Events, not its logs.
3. The entrypoint is wrong. Exit 127 and no logs is the signature.
4. The process writes to a file, not stdout. Nothing is broken; you are looking in the wrong place. kubectl exec into a working replica and check.

Events carry things logs never will:

kubectl get events --field-selector involvedObject.name=<pod> --sort-by=.lastTimestamp

Why you wait longer each time

The kubelet backs off exponentially between restarts: 10s, 20s, 40s, 80s, 160s, then capped at 5 minutes. The counter resets once a container has run successfully for long enough — so a container that stays up recovers its fast-restart budget.

This is why a pod that has been failing all night takes five minutes to show you the result of your fix. It is not that your fix did not work. It is that the kubelet is still in its backoff window. If you want the answer now, delete the pod and let the controller create a fresh one with a clean backoff.

Kubernetes 1.33 added an alpha feature gate, ReduceDefaultCrashLoopBackOffDecay, which starts the sequence at 1s and caps it at 60s instead. Unless someone has explicitly turned it on for your nodes, five minutes is what you get.

Stopping it happening again

• Set memory limits from observed usage, not from guesses. Most OOMKilled loops come from a limit copied out of another service’s manifest.
• Give every slow-starting service a startupProbe. It costs nothing and removes an entire category of self-inflicted restart.
• Do not let one-shot work run under a Deployment. Jobs exist for this.
• Alert on restartCount increasing, not on pods being unready. A container restarting every four minutes and passing its readiness check in between is invisible to most dashboards and will be reported to you by a user.

A note on what the status column is actually telling you

CrashLoopBackOff is computed by kubectl from state.waiting.reason on one specific container — and kubectl walks the container list in reverse, so in a pod with a sidecar you may be reading the sidecar’s status rather than your application’s. If the exit codes you find do not match the container you think is failing, that is why. We wrote up how kubectl decides what to put in the STATUS column in full, because it explains a surprising amount of confusing output.

If the container never started at all, the status you are looking at is more likely ImagePullBackOff — a different problem with a similar-sounding name, covered in ImagePullBackOff and ErrImagePull.