What kubectl rollout history CHANGE-CAUSE is really showing you

CHANGE-CAUSE is not a record of what a revision changed. It is a free-text annotation on the Deployment, which the controller copies onto whichever ReplicaSet happens to be current when it next syncs.

Three consequences follow, and all three are routinely mistaken for bugs:

• The text can describe a change that is not in that revision.
• Two revisions can carry the same text.
• Setting it does not create a revision, but it does overwrite one.

Nothing validates the string against the pod template. It is a comment field with the authority of a changelog and none of the guarantees.

The fastest check

Never trust the column. Ask what the revision actually contains:

kubectl rollout history deploy/<name> --revision=3

Or read every revision, its cause and its real image in one go — the ReplicaSets are the history:

kubectl get rs -l app=<label> -o custom-columns='NAME:.metadata.name,REV:.metadata.annotations.deployment\.kubernetes\.io/revision,CAUSE:.metadata.annotations.kubernetes\.io/change-cause,IMAGE:.spec.template.spec.containers[0].image'

That second command is the one to keep. It puts the claim and the fact side by side.

Reproducing the wrong answer

On a Kubernetes 1.36.1 cluster, a deployment on busybox:1.36. Change the image, then write the change-cause afterwards — the order almost everyone uses, because you annotate once you know the change worked:

kubectl set image deploy/histdemo busybox=busybox:1.37
kubectl annotate deploy/histdemo kubernetes.io/change-cause="upgrade to busybox 1.37" --overwrite

REVISION  CHANGE-CAUSE
1         <none>
2         upgrade to busybox 1.37

That looks right, and it is only right by luck: the annotation landed on revision 2 because revision 2 was still the current ReplicaSet. Now do the next upgrade with the annotation first, which is the order the documentation implies:

kubectl annotate deploy/histdemo kubernetes.io/change-cause="upgrade to 1.36.1" --overwrite
kubectl set image deploy/histdemo busybox=busybox:1.36.1

REVISION  CHANGE-CAUSE
1         <none>
2         upgrade to 1.36.1
3         upgrade to 1.36.1

Revision 2 has just been retroactively relabelled. It contains busybox:1.37 and now claims to be the 1.36.1 upgrade:

NAME                  REV   CAUSE               IMAGE
histdemo-6c9777576d   1     <none>              busybox:1.36
histdemo-756d7dd695   2     upgrade to 1.36.1   busybox:1.37
histdemo-85488f6cc    3     upgrade to 1.36.1   busybox:1.36.1

Two revisions, one story, and the older one is describing an image it never ran. Neither ordering is safe: annotate first and you relabel the outgoing revision, annotate afterwards and you are racing the controller.

Why it happens

The annotation is not on the pod template. It is on the Deployment’s own metadata, and the Deployment controller propagates the Deployment’s annotations down to the ReplicaSet it is currently managing on every sync.

Zoom

So kubectl rollout history is reading a field that arrived from somewhere else, at a time it does not record. Sequence it out and the double-label stops being mysterious:

Zoom

The pod template is what defines a revision — change it and you get a new ReplicaSet with the next revision number. The annotation is not part of the template, which is why changing it alone produces no new revision:

kubectl annotate deploy/histdemo kubernetes.io/change-cause="totally unrelated text" --overwrite

REVISION  CHANGE-CAUSE
1         <none>
2         totally unrelated text

Same two revisions. Revision 2’s history has simply been rewritten, silently, by a command that changed nothing about what is running. There is no append, no timestamp, and no previous value kept.

--record is deprecated and was never better

The flag that used to populate this automatically still works and still warns:

Flag --record has been deprecated, --record will be removed in the future

It has the same defect with a friendlier face. It writes the command line you typed, which is a record of your intent rather than of the resulting template — and it writes it even when the command fails. On a test deployment, a kubectl set image that errored with unable to find container named "app" still replaced the existing revision’s CHANGE-CAUSE with the failed command. The revision did not change. Its description did.

A command line is also frequently useless as history: kubectl apply -f . records nothing about what was in the directory.

This has been reported upstream as a bug, and the outcome is worth knowing. In kubectl#1869 (refiled from kubernetes#141263, which was closed within a day as a support question), a reporter showed CHANGE-CAUSE naming the wrong image after a --recorded command was followed by an unrecorded one. The SIG CLI contributor assigned to investigate agreed on 22 August 2026: “yes this is a valid bug”. On 9 September 2026 a maintainer closed it without a fix:

The recommended path is to always use kubectl apply with server side apply instead of using imperative commands (rollout, create, etc.) which can be considered as legacy commands.

So this is not going to be fixed in kubectl. The behaviour described here is the behaviour you should plan around, and upstream’s own view is that rollout and create are legacy commands.

Rolling back renumbers the thing you rolled back to

The other surprise is that revisions are not stable identifiers. rollout undo does not create a copy of an old revision — it reuses that ReplicaSet and gives it the next number.

kubectl rollout undo deploy/histdemo --to-revision=1

REVISION  CHANGE-CAUSE
2         upgrade to 1.36.1
3         upgrade to 1.36.1
4         <none>

Revision 1 is gone from the history. It is now revision 4, running the same busybox:1.36 it always ran, and its change-cause is empty because the Deployment’s annotation was cleared by the undo. A runbook that says “roll back to revision 1” is unusable after the first rollback, and any incident note quoting a revision number is only valid until the next one.

Revisions also expire. revisionHistoryLimit defaults to 10, so old ReplicaSets — and the only copies of their templates and change-causes — are garbage collected without warning.

Making it actually useful

Nothing here is fixable by using the field more carefully; it is fixable by not depending on it.

• Put the change-cause in the manifest, not in a follow-up command. If the annotation is applied in the same kubectl apply as the template change, the two arrive together and the race disappears. This is the single change that makes the column trustworthy.
• Record something derivable from the template. An image tag or a Git SHA can be checked against the ReplicaSet later. “Fix the login bug” cannot.
• Add your own template annotation as well. An annotation under spec.template.metadata.annotations is part of the revision, so it can never be retroactively rewritten and never applies to the wrong ReplicaSet — at the cost of triggering a rollout when it changes, which for a build-SHA annotation is what you want anyway.
• Identify revisions by ReplicaSet name, not number. histdemo-756d7dd695 is stable for the life of the object. revision 2 is not.
• Raise revisionHistoryLimit if you rely on history at all. Ten rollouts is a quiet week for an active service.

Reading a rollout while it is happening

For the state of a rollout in progress rather than the record of past ones, the useful signals are elsewhere: kubectl rollout status, the ready-over-desired counts, and the restart counts on the new pods. A new ReplicaSet whose pods are restarting is a failing rollout regardless of what the history says, and what RESTARTS 3 (5m ago) is actually counting covers how to read that number correctly across a multi-container pod.

If the new pods never start at all, the reason is in the container status rather than in the rollout: an image that will not pull is covered in ImagePullBackOff and ErrImagePull, and a missing ConfigMap or Secret key in CreateContainerConfigError and CreateContainerError.