Let’s Encrypt is a nonprofit certificate authority that provides free SSL/TLS certificates, which are essential for enabling secure HTTPS connections on websites. Launched in 2016 by the Internet Security Research Group (ISRG), its mission is to make encrypted connections ubiquitous on the web, enhancing privacy and security for all users. Let’s Encrypt simplifies the process of obtaining and renewing certificates through automation, making it accessible even for those without extensive technical knowledge. By offering a straightforward, cost-free way to secure websites, Let’s Encrypt plays a crucial role in promoting a safer and more privacy-respecting internet.
Focus
For the focus of this discussion, we want to be able to create and manage certificates in an automated way. To be even more blunt – we do not want anyone, especially developers, to be concerned with certificate management. The acts of certificate management is one of the clearest ways a platform engineering mindset can eliminate both technical toil but also reduce cognitive load on the organization. For our OpenShift clusters, there are typically two different types of certificates we want to manage:
- Cluster certificates – these are going to be certificates for the cluster ingress, which is a wildcard certificate, as well as certificates for the api endpoints.
- Application certificates – for each application, there may be a need for specific application certificates, especially if the cluster serves both cluster DNS requests (test.apps.ocp.example.com) but also logical DNS (test.dev.example.com).
Operator Installation
To get the party started, let’s install the OpenShift cert-manager operator.
apiVersion: v1
kind: Namespace
metadata:
name: cert-manager-operator
---
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
name: cert-manager-operator-operator-group
namespace: cert-manager-operator
spec:
targetNamespaces:
- cert-manager-operator
---
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: openshift-cert-manager-operator-subscription
namespace: cert-manager-operator
spec:
channel: stable-v1
installPlanApproval: Automatic
name: openshift-cert-manager-operator
source: redhat-operators
sourceNamespace: openshift-marketplaceCreating a ClusterIssuer
Once the operator is installed, we will now want to configure a ClusterIssuer. In our case, we are going to use Let’s Encrypt as the authority to issue our certificates and we are going to use the acme protocol to manage the lifecycle. However, one of our use cases is going to be managing the wildcard cluster ingress certificate. As part of the acme protocol, we have to be able to prove we own the domain for which we are wanting certificates issued. There are two ways to do this: HTTP or DNS. The HTTP solver allows verification because we stand up a web server to respond to the requests and therefore, we would have to own the DNS to get it to resolve correctly. But the HTTP solver doesn’t work for wildcard certs, so we need to use the DNS solver which will place a TEXT DNS entry temporarily in our domain and then query DNS for the entry.
To achieve this, you will have to use a DNS solver plugin. In our case, we are using Cloudflare for DNS so we will use the Cloudflare DNS solver plugin. But we have to provide API access to the ClusterIssuer to be able to update our DNS, so to accomplish this, you will need to go into your Cloudflare setup and generate an API token with the permissions to update that zone’s DNS records. Once you have that token, we will need to create a secret for it and place it in the cert-manager namespace.
oc create secret generic cloudflare-api-token-secret \
-n cert-manager \
--from-literal=api-token=<token>Now that we have our secret, we can create our ClusterIssuer. The resource is not namespaced, as it applies to the entire cluster.
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-cluster-issuer
spec:
acme:
server: 'https://acme-staging-v02.api.letsencrypt.org/directory'
privateKeySecretRef:
name: acme-account-private-key
solvers:
- dns01:
cloudflare:
apiTokenSecretRef:
key: api-token
name: cloudflare-api-token-secretWildcard Ingress Certificate
With our ClusterIssuer in place, we can now request a wildcard certificate for our cluster ingress.
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: apps-ocp-lab-snimmo-com-certificate
namespace: openshift-ingress
spec:
commonName: apps.ocp.lab.snimmo.com
dnsNames:
- "apps.ocp.lab.snimmo.com"
- "*.apps.ocp.lab.snimmo.com"
secretName: apps-ocp-lab-snimmo-com-tls
isCA: false
issuerRef:
group: cert-manager.io
name: letsencrypt-cluster-issuer
kind: ClusterIssuerHere are some notes about the resource:
- Naming: your organization can setup some fairly easy patterns here to manage the consistency of naming by simply replacing the FQDN’s periods with dashes and then appending some type information.
- Namespacing: We want the certificate to be put into a secret in the
openshift-ingressnamespace because that is where the resources will be updated to refer to the secret.
Next we need to download and add the Let’s Encrypt STAGING root CA to the cluster.
curl https://raw.githubusercontent.com/letsencrypt/website/main/static/certs/staging/letsencrypt-stg-root-x1.pem > /home/snimmo/Downloads/letsencryptrootca.pem
oc create configmap custom-ca \
--from-file=ca-bundle.crt=/home/snimmo/Downloads/letsencryptrootca.pem \
-n openshift-config
oc patch proxy/cluster --type=merge \
--patch='{"spec":{"trustedCA":{"name":"custom-ca"}}}'Once the certificate is issued, we can then update the default ingress controller to use the wildcard tls we just created.
oc patch ingresscontroller.operator default \
--type=merge -p \
'{"spec":{"defaultCertificate": {"name": "apps-ocp-lab-snimmo-com-tls"}}}' \
-n openshift-ingress-operatorAPI Server Certificate
We also want to replace the certificate for API access to the OpenShift cluster. This certificate is located in the openshift-config namespace so our Certificate manifest goes there.
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: api-ocp-lab-snimmo-com-certificate
namespace: openshift-config
spec:
commonName: api.ocp.lab.snimmo.com
dnsNames:
- "api.ocp.lab.snimmo.com"
secretName: api-ocp-lab-snimmo-com-tls
isCA: false
issuerRef:
group: cert-manager.io
name: letsencrypt-cluster-issuer
kind: ClusterIssuerAfter applying the manifest, you can watch the Certificate request progress using the following command.
oc get certificate -w -n openshift-configOnce the certificate is “Ready”, then we can apply the patch to the apiserver to swap out the certificates.
oc patch apiserver cluster --type=merge \
-p '{"spec":{"servingCerts": {"namedCertificates": [{"names": ["api.ocp.lab.snimmo.com"], "servingCertificate": {"name": "api-ocp-lab-snimmo-com-tls"}}]}}}' Application Certificates
The cert-manager operator allows you to use annotations to produce the certificates. However, we need to take a small extra step to configure the cluster to enable this functionality on OpenShift Route objects. The documentation for this install is located at https://github.com/cert-manager/openshift-routes?tab=readme-ov-file#installation
oc apply -f https://github.com/cert-manager/openshift-routes/releases/latest/download/cert-manager-openshift-routes.yamlOnce this component is installed, we can utilize the existing ClusterIssuer to manage the certificates by simply utilizing some annotations on the route.
apiVersion: route.openshift.io/v1
kind: Route
metadata:
name: busybox
namespace: default
annotations:
cert-manager.io/common-name: apps.ocp.lab.snimmo.com
cert-manager.io/issuer-name: letsencrypt-cluster-issuer
cert-manager.io/issuer-kind: ClusterIssuer
spec:
host: busybox-default.apps.ocp.lab.snimmo.com
to:
kind: Service
name: busybox
weight: 100
port:
targetPort: 8080
tls:
termination: edge
wildcardPolicy: NoneWhen this is applied, it is automatically updated to include all the correct certificate information.
snimmo@snimmo-mac notes % oc get route busybox -o yaml
apiVersion: route.openshift.io/v1
kind: Route
metadata:
annotations:
cert-manager.io/certificate-revision: "1"
cert-manager.io/common-name: apps.ocp.lab.snimmo.com
cert-manager.io/issuer-kind: ClusterIssuer
cert-manager.io/issuer-name: letsencrypt-cluster-issuer
creationTimestamp: "2024-05-16T16:54:59Z"
name: busybox
namespace: default
resourceVersion: "10695190"
uid: bcbb9a0c-ab4d-4f70-9664-cfd85a52a4a9
spec:
host: busybox-default.apps.ocp.lab.snimmo.com
port:
targetPort: 8080
tls:
certificate: |
-----BEGIN CERTIFICATE-----
MIIFIzCCBAugAwIBAgISAxfYjt9t38FE6NX5rcVLtnTEMA0GCSqGSIb3DQEBCwUA
...
SxAs2GWpP38NvIU2zmH0weUYhFoOxRI=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAw
...
MldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqX
nLRbwHOoq7hHwg==
-----END CERTIFICATE-----
key: |
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAvpOTJY0AII6wH6qgsiRNFDHd9nGYDFyphmUUzZv/Xlz+0JFz
...
KkX9cMbCVLj1H/F2X8dt1Zt7ZR9q8wF/wdjQQo7RWTE9uITTXegQ
-----END RSA PRIVATE KEY-----
termination: edge
to:
kind: Service
name: busybox
weight: 100
wildcardPolicy: None
status:
ingress:
- conditions:
- lastTransitionTime: "2024-05-16T16:54:59Z"
status: "True"
type: Admitted
host: busybox-default.apps.ocp.lab.snimmo.com
routerCanonicalHostname: router-default.apps.ocp.lab.snimmo.com
routerName: default
wildcardPolicy: None
One comment
Comments are closed.