Concepts

Rollouts

A Rollout deploys the builds of one OCI artifact into one environment, one build at a time, and holds further deploys when a build fails its health checks. The artifact is either your container image or the rendered manifests for that environment.

On this page
  1. How a build is chosen
  2. Rollout resource
  3. Rollout status
  4. What a Rollout propagates
  5. When a deploy is held
  6. How a deploy is verified
  7. When a bake fails
  8. Related

How a build is chosen

A Rollout reads its releases from a ReleaseSource, which lists the tags of an OCI repository. Every release newer than the deployed one is a candidate. The Rollout deploys the newest candidate that every gate allows, by writing the version to the Flux source it owns: a Kustomization substitution variable or an OCIRepository tag. It then bakes the build and records the outcome in status.history.

Kuberik deploys one version at a time. If 1.2.0 is deploying or baking when 1.3.0 and 1.4.0 arrive, both wait as candidates. When 1.2.0 succeeds, the Rollout deploys the newest candidate the gates allow, 1.4.0, and skips 1.3.0.

Rollout resource

rollout.yamlYAML
apiVersion: kuberik.com/v1alpha1
kind: Rollout
metadata:
  name: hello-world
  namespace: hello-world
spec:
  releaseSourceRef:
    name: hello-world
  versionHistoryLimit: 5
  deployTimeout: 10m  # healthy within 10m
  bakeTime: 15m
  healthCheckSelector:
    selector:
      matchLabels:
        app: hello-world
FieldPurpose
releaseSourceRefThe ReleaseSource that provides new versions, in the Rollout’s namespace.
releasesImagePolicyDeprecated and ignored. Use releaseSourceRef.
versionHistoryLimitHow many past deployments to keep in status.history (default 10).
bakeTimeHow long the build must stay free of health-check errors once the bake starts (e.g., 5m, 1h).
deployTimeoutHow long to wait for the health checks to become healthy after a deploy. When it passes before the bake starts, the deploy fails. Unset means wait indefinitely.
healthCheckSelectorLabel selector (and optional namespace selector) for the HealthCheck resources to evaluate.
wantedVersionPins the Rollout to one tag: it deploys that tag and nothing else until you clear the field. See Operate with kubectl.

Rollout status

The History tab of a staging rollout: the Deploys card groups each deploy by day, with its build, its commit message, when it turned healthy and how long ago it ranThe History tab of a staging rollout: the Deploys card groups each deploy by day, with its build, its commit message, when it turned healthy and how long ago it ran
Terminal
kubectl -n hello-world get rollout hello-world -o yaml
  • history: deployments, newest first. history[0].version.tag is the running version; each entry records its message, triggeredBy and bakeStatus (Deploying, InProgress, Succeeded, Failed or Cancelled).
  • candidates: the newest releases newer than the deployed one (20 by default, set by the controller flag --release-candidates), plus any tag a gate, the pin or an action annotation names, up to 64. Gates and selection see every release; this list is what the status shows. gated: true means every gate lets the release through; blockedBy names the gates that hold it.
  • gates: each gate that targets this Rollout, whether it is passing, and the versions it allows.
  • conditions include Ready, GatesPassing, DeploymentBlocked (a health check is unhealthy), BakeFailureDisabled (see When a bake fails), and ReleasesUpdated and ReleaseSourceReady (whether releases can be read from the ReleaseSource).

What a Rollout propagates

A Rollout does not care what its versions point at. Annotate a Kustomization and Kuberik substitutes the version into it; annotate an OCIRepository and Kuberik moves its tag. Gating, health checks and bake time work the same way in both cases. See Propagation Modes for the trade-offs and for running both at once.

When a deploy is held

Before an automatic deploy, the build must be allowed by every gate that targets the Rollout. A gate either lets a build through or holds it; a held build is evaluated again whenever a gate changes, and nothing is rejected for good. Gates never hold deploys you start by hand. On a Rollout’s first deploy they never hold either: it deploys the newest release the gates allow, or the newest release if they allow none.

Automatic deploys also wait while any selected health check reports Unhealthy. The Rollout then sets DeploymentBlocked=True with the name of the check. A check that is still Pending holds nothing, and a Rollout’s first deploy and deploys you start by hand go ahead regardless.

RolloutGates filter the release candidates; the newest allowed version deploysThree release candidates, 1.6.0, 1.5.0 and 1.4.0, reach the RolloutGates. A gate whose allowedVersions lists only 1.5.0 and 1.4.0 holds 1.6.0. The newest allowed candidate, 1.5.0, deploys. When no candidate is allowed, or a gate has passing: false, the Rollout is held and re-evaluated when a gate changes; there is no polling and no reject state. Notes: a first deploy is never held, it takes the newest allowed release or else the newest; on a deploy by hand gates are skipped; an Unhealthy HealthCheck also holds automatic deploys and sets DeploymentBlocked=True.RELEASE CANDIDATES1.6.01.5.01.4.0RolloutGatepassing: trueallowedVersions:- 1.5.0- 1.4.0Rolloutdeploys 1.5.0,the newest allowedHELDevaluated again whena gate changesotherwise: no candidateallowed, orpassing: falseFirst deploy:never held; the newest allowed release, else the newest.Deploy by hand (wantedVersion, force-deploy):gates are skipped.A HealthCheck is Unhealthy:held,DeploymentBlocked=True. RolloutGates filter the release candidates; the newest allowed version deploysThree release candidates, 1.6.0, 1.5.0 and 1.4.0, reach the RolloutGates. A gate whose allowedVersions lists only 1.5.0 and 1.4.0 holds 1.6.0. The newest allowed candidate, 1.5.0, deploys. When no candidate is allowed, or a gate has passing: false, the Rollout is held and re-evaluated when a gate changes; there is no polling and no reject state. Notes: a first deploy is never held, it takes the newest allowed release or else the newest; on a deploy by hand gates are skipped; an Unhealthy HealthCheck also holds automatic deploys and sets DeploymentBlocked=True.RELEASE CANDIDATES1.6.01.5.01.4.0RolloutGatepassing: trueallowedVersions: [1.5.0, 1.4.0]Rolloutdeploys 1.5.0,newest allowedHELDnothing allowed;evaluated againon any changeFirst deploy:never held.Deploy by hand:gates skipped.HealthCheck Unhealthy:held,DeploymentBlocked=True.
RolloutGates filter the release candidates; the newest allowed one deploys, otherwise the Rollout is held until a gate changes.

How a deploy is verified

After a deploy, the history entry starts in Deploying. The bake starts (InProgress) once every selected health check is Healthy and has changed state since the deploy, so a check that was healthy before the deploy does not count until it reports on the new build. The deploy:

  • succeeds when bakeTime passes after the bake started, with no error;
  • fails when a health check reports an error at any point after the deploy, before or during the bake;
  • fails when deployTimeout passes before the bake starts;
  • is cancelled when a deploy you start by hand replaces it.
How a deploy is verified: the bake status of the newest history entryA deploy starts in Deploying. When every selected HealthCheck is Healthy after the deploy it moves to InProgress and the bake time runs. When bakeTime passes without an error it is Succeeded. A HealthCheck error after the deploy fails it from Deploying or InProgress, and so does deployTimeout passing before the bake starts. A Failed bake goes back to Deploying with the rollout.kuberik.com/retry annotation. A deploy started by hand cancels a Deploying or InProgress bake, and BakeFailureDisabled=True suppresses failing.DEPLOYINGwaits for healthINPROGRESSbakeTimerunsSUCCEEDEDnext version may goevery checkHealthybakeTimepassedFAILEDautomatic deploysstop herea HealthCheckerrorsdeployTimeoutrollout.kuberik.com/retryCancelled:a deploy by hand replaces a Deploying or InProgress bake.BakeFailureDisabled=True:the bake is never marked Failed. How a deploy is verified: the bake status of the newest history entryA deploy starts in Deploying. When every selected HealthCheck is Healthy after the deploy it moves to InProgress and the bake time runs. When bakeTime passes without an error it is Succeeded. A HealthCheck error after the deploy fails it from Deploying or InProgress, and so does deployTimeout passing before the bake starts. A Failed bake goes back to Deploying with the rollout.kuberik.com/retry annotation. A deploy started by hand cancels a Deploying or InProgress bake, and BakeFailureDisabled=True suppresses failing.DEPLOYINGwaits for healthHealthChecksHealthyINPROGRESSbakeTimerunsbakeTimepassedSUCCEEDEDnext version may goFAILEDautomaticdeploys stoperrordeployTimeoutrollout.kuberik.com/retryerror:a HealthCheck errors after the deploy.Cancelled:a deploy by hand replaces the bake.BakeFailureDisabled=True:never Failed.
The bake status of the newest deploy: health starts the bake, bakeTime ends it, any HealthCheck error fails it.

A Rollout with no bakeTime, deployTimeout or healthCheckSelector marks each deploy Succeeded straight away.

When a bake fails

Kuberik does not roll back. When a bake fails, the failed build keeps running, the Rollout sets Ready=False, and no newer build deploys automatically until a person pins a version, deploys a build by hand, sets rollout.kuberik.com/unblock-failed, or retries the bake. Operate with kubectl shows each of these.

Part of the Change version dialog deploying the held build 4.45.0-45 of checkout-api to production: the rule it would override, After staging, the four commits to deploy, the Pin build switch, a required reason, the typed confirmation and the Deploy buttonPart of the Change version dialog deploying the held build 4.45.0-45 of checkout-api to production: the rule it would override, After staging, the four commits to deploy, the Pin build switch, a required reason, the typed confirmation and the Deploy button

When a deploy starts, the Rollout sets the BakeFailureDisabled condition for it. The condition is True, and health-check errors and deployTimeout do not fail that deploy, in two cases:

ReasonWhen
PreviousBakeFailedThe previous deploy did not end with a successful bake: it failed, or this deploy cancelled its bake.
DeployedWithUnhealthyHealthChecksA deploy you started by hand began while a selected health check was Unhealthy.

The bake still has to start: it starts the first time every check is Healthy after the deploy, and the deploy succeeds bakeTime later. Errors during that window do not fail it, so a recovery deploy is not verified the way a normal one is. If the checks never turn healthy, it stays in Deploying and the next build has to be deployed by hand. The condition holds until the next deploy starts.

Related