Helm Release Parsing

Helm 3 sometimes feels like little more than just a template rendering engine coupled with a very thin wrapper around kubectl apply -f. It does of course bill itself as a Package Manager (and this branding does make some sense. But perhaps more in the way of Gentoo’s ebuild packages, than like DEB or RPM packages…) but anyone who has to do serious work with Helm often finds themselves doing more template writing and reading, than simply running helm install <some repo>/<some chart>.

Some casaul users may also have encountered the helm list and helm history commands which report about the existing releases and revisions on the current Kubernetes cluster. These of course hint that Helm does more than just render and apply Kubernetes manifests, but in fact tracks something of state and changes to that state.

Some may even have dared to use helm rollback to really get hands on with Helm’s state management features.

Unfortunately, while I have found that Helm is a very effective state manager the Helm CLI tool itself leaves a lot to be desired in the way of actually harnessing the power of state management. But that doesn’t mean we can’t take matters into our own hands to get more out of Helm’s state management.

The Release Object

Helm 3 stores state in Kubernetes Secret objects within the namespace that a Helm Release (install/upgrade) is performed. Each revision (or release number) is stored in its own Secret object in the format:

sh.helm.release.v1.<HELM RELEASE NAME>.v<REVISION NUMBER>

These objects however may seem of little use to anything but the Helm tool itself at first glance, but with a little processing, one can find a trove of information in these objects which can be used to do more interesting things.

Extracting Release Data

Like all Opaque K8s Secrets, the release data is Base 64 encoded. However, for reasons I can only speculate about, they are actually Base 64 encoded twice. But if you then decode them twice you will be met with a binary stream of data which upon inspection will show itself to be actually GZIP compressed JSON.

export NAMESPACE="<MY NAMESPACE>"
export HELM_RELEASE_NAME="<MY RELEASE>"
# Use this to simply use the latest release:
export REVISION_NUMBER=$(helm list --no-headers -n "${NAMESPACE}" | grep "^${HELM_RELEASE_NAME}" | awk '{print $3}')
# Or specify a specific number manually:
#export REVISION_NUMBER=5
kubectl \
  -n "$NAMESPACE" \
  get secret/sh.helm.release.v1.${HELM_RELEASE_NAME}.v${REVISION_NUMBER} \
  -o json \
    | jq -r .data.release \
    | base64 -d \
    | base64 -d \
    | gzip -d

Get Applied Manifests

# Extract Release Data command...
#...| gzip -d \
    | jq -r '.manifest' > manifests.yaml

Comparing Rendered Manifests

The Manifests stored in the release are the same as the output of the helm template command.

This means that if you can compare the manifests produced by helm template with the manifests of the current release to get a diff of the changes you would make by applying an upgrade to your helm release by comparing the two.

helm template my-release my-repo/my-chart -f my-override-values.yaml > new-manifests.yaml
# manifests.yaml is what was extracted from the latest release above.
diff -u manifests.yaml new-manifests.yaml > release-changes.diff

Redacting Secrets

If your manifests contain secrets, and you are going to be exporting or printing them somewhere you don’t want to leak these secret to, you can use yq to redact the sensitive information.

# Extract Release Data command...
#...| gzip -d \
    | jq -r '.manifest' \
    | yq eval '(select(.kind == "Secret") | (select(has("data")) | .data[] |= "***[REDACTED]***"), (select(has("stringData")) | .stringData[] |= "***[REDACTED]***")) // .' > manifests.yaml

Extracting a Helm Chart from a Release

Due to a series of unfortunate choices, I once faced the situation where I deleted my whole locally-developed Helm Chart, with no backup.

Or at least I thought that I had no backup. But in reality I had already deployed the Helm Chart to a Kubernetes cluster, and so that meant the chart was actually still backed up.

Using the following, commands I was able to essentially re-hydrate the entire chart in nearly the exact state it was when the release/revision was created:

Template Files

# Extract Release Data command...
#...| gzip -d \
    | jq .chart.templates \
    | jq -r '.[] | @sh "mkdir -p $(dirname \(.name)) && echo \(.data) | base64 -d > \(.name)"' \
    | xargs -I {} sh -c {}

Default Values Files

# Extract Release Data command...
#...| gzip -d \
    | jq .chart.values \
    | yq -p json -o yaml > values.yaml

Chart.yaml

# Extract Release Data command...
#...| gzip -d \
    | jq .chart.metadata \
    | yq -p json -o yaml > Chart.yaml

Other Files

# Extract Release Data command...
#...| gzip -d \
    | jq .chart.files \
    | jq -r '.[] | @sh "mkdir -p $(dirname \(.name)) && echo \(.data) | base64 -d > \(.name)"' \
    | xargs -I {} sh -c {}