One of the trends of recent cloud computing is continuous deployment. As soon as a new version of a service is available, automated processes have it deployed on staging (or even production) system. There are many ways to achieve this. One of them is to use keel.
Keel is a very simple Kubernetes Operator to automate Helm, DaemonSet, StatefulSet & Deployment updates. It will track your installations and, when an update is published on their repository, it will update the images.
It can be triggered by webhook from Jenkins or GitHub actions or it can periodically poll the repository to see if there are updates.
You can program it to trigger the updates using semantic rules, so that (for example) it will not update to a different major release, but just to a newer patchlevel.
Installation and usage
Installation is very simple: you can install it using kubectl or helm. Helm is particolarely convenient, so we are going to show you that.
A minimal installation is very simple:
helm upgrade --install keel --namespace=keel keel-charts/keel --set helmProvider.enabled="false"
Recent versions of keel also offers an handy console, so you may want to enable that:
cat <<EOF > keel-values.yaml
helm upgrade --install keel --namespace=keel keel-charts/keel -f keel-values.yaml
Keel uses resource (deployment, daemonset or statefulset) annotation to configure its behavior. To allow your deployment (or daemonset or statefulset) to be tracked and update by keel, you therefore need to annotate it. You can simply add this annotations to have keel track resource image repository by polling, and have it updated on new minor releases.
keel.sh/policy: minor # <-- policy name according to https://semver.org/
keel.sh/trigger: poll # <-- actively query registry, otherwise defaults to webhooks
You can choose different update policies:
- all: update whenever there is a version bump or a new prerelease created (i.e.: 1.0.0 -> 1.0.1-rc1)
- major: update major & minor & patch versions
- minor: update only minor & patch versions (ignores major)
- patch: update only patch versions (ignores minor and major versions)
- force: force update even if tag is not semver, ie: latest, optional label: keel.sh/match-tag=true which will enforce that only the same tag will trigger force update.
- glob: use wildcards to match version
There are more setting you can enable, like notifications or different poll schedules. You can check them out at https://keel.sh/docs
One interesting feature of keel is that it can wait for an external approval. When an update is ready to be deployed, a new approval request is created. A human supervisor, or a different software, have to vouch the update in order to be performed.
If you have enabled the web panel, you can check for update approvals using a browser or you can use the rest interface to interact with them. We are going to leverage that for image authentication.
Integration with Codenotary Trustcenter
Codenotary Trustcenter is a trust and integrity management platform, that enables any company, developer, automation engineer, DevOps engineer to secure all stages of any CI/CD pipeline. Assets, like container images, can be notarized, marked as trusted, and then authenticated (checked) to see if what I’m going to run on my cluster is actually trusted.
Keel approval phase is the perfect moment to plug in image authentication. Image is approved only if it is trusted to run.
To do that, we use this simple tool: keel validator.
It is a simple python script, deployed alongside keel, that will poll it to see if there are pending approvals. If there are, it tries to authenticate using
vcn tool from Codenotary.
If the image authenticates, that means it was notarized and signed as trusted, so the update is approved.
Installation is straightforward: grab the
keel-validator.yaml file from the repository, and be sure to have at hand:
- apikey from Codenotary Trustcenter (User Identifier)
- keel admin password (the one you put in keel-values.yaml)
- eventual json token for non-public repositories
- eventual credentials for non-public repositories
Fill in the values for credentials in
Then you can simply deploy keel-validator:
kubectl deploy -n keel -f keel-validator.yaml
If your image repository is protected by simple username/password, you’ll have to modify keel-validator.yaml to use them instead of a json token:
Automation, security and ease of use don’t have to necessarely to be in conflict. It’s just a matter of finding the right tool for the job at hand. Codenotary Trustcenter can be easily integrated in many workflow and provide safety and awareness.