TLS certificate expiry monitoring that actually catches outages

Why checking the renewal job is not enough

An automated renewal job can report success while users see an expired certificate. These are the failure paths that a calendar reminder or a job status check will miss:

The principle follows from the table: test what's actually being served, from where clients are, and validate it the way a client would.

Inspect a live endpoint

These openssl commands are worth keeping to hand. -servername matters, because without SNI a shared endpoint may return a different certificate.

HOST=api.internal.example
echo | openssl s_client -connect "$HOST:443" -servername "$HOST" 2>/dev/null \
  | openssl x509 -noout -subject -issuer -enddate -ext subjectAltName

To check for expiry in a script, use -checkend with a number of seconds. It exits non-zero if the certificate will have expired by then:

echo | openssl s_client -connect "$HOST:443" -servername "$HOST" 2>/dev/null \
  | openssl x509 -noout -checkend $((21 * 86400)) \
  && echo "more than 21 days left" || echo "expires within 21 days"

To validate the chain and hostname, as a client would, and show the whole chain:

echo | openssl s_client -connect "$HOST:443" -servername "$HOST" \
  -verify_hostname "$HOST" -CAfile /etc/pki/tls/certs/ca-bundle.crt -showcerts 2>/dev/null \
  | grep -E 'Verification|Verify return code|s:|NotAfter'

Not every TLS service starts with a TLS handshake. Protocols that upgrade the connection need -starttls:

openssl s_client -connect db.internal.example:5432 -starttls postgres
openssl s_client -connect mail.internal.example:587 -starttls smtp

A probe that opens a TCP port and reports "up" will not see any certificate on those services, so make sure your probing covers them.

Build the inventory

You cannot alert on a certificate you don't know about. A public certificate may appear in Certificate Transparency logs, but internal CA certificates are recorded nowhere except your own systems, so assemble a list from several directions.

Kubernetes TLS secrets. This lists the leaf expiry for every TLS secret you are permitted to read:

kubectl get secrets -A --field-selector type=kubernetes.io/tls \
  -o jsonpath='{range .items[*]}{.metadata.namespace}{"\t"}{.metadata.name}{"\t"}{.data.tls\.crt}{"\n"}{end}' \
  | while IFS=$'\t' read -r ns name crt; do
      end=$(printf '%s' "$crt" | base64 -d | openssl x509 -noout -enddate | cut -d= -f2)
      printf '%s/%s\t%s\n' "$ns" "$name" "$end"
    done

This shows the first certificate in each bundle, and only those in secrets of that type. Certificates stored as Opaque secrets or in ConfigMaps need a separate search.

Cluster internals. Control plane certificates matter as much as anything you run. On kubeadm clusters, kubeadm certs check-expiration lists them. OpenShift manages and rotates its platform certificates, including the service CA, so the task there is to watch the cluster operators for degradation rather than to rotate by hand. Check the OpenShift docs for your version on the rotation schedule, particularly if clusters can be shut down for long periods, since rotation can be missed while a cluster is off.

Webhook and API service CA bundles. A caBundle in a MutatingWebhookConfiguration or APIService has its own expiry, and a failed webhook certificate can stop pod creation.

Java keystores. Java applications frequently hold certificates in a keystore, which your file-based searches will miss:

keytool -list -v -keystore truststore.jks | grep -E 'Alias name|Valid from'

Linux hosts. Search for certificate files in the usual paths, and also read server configuration, since a referenced file may be in an unexpected location.

Often-forgotten certificates. Check load balancers and appliances, VPN gateways, SAML signing certificates, code signing certificates, client certificates for partner APIs, LDAPS on directory servers, and mail relays. Also check embedded devices. Add a named owner to each entry. A list with no owners will not be kept up to date.

Fill in the gaps by scanning internal address ranges for TLS ports, subject to your security team's approval. A scan will find endpoints that nobody thought to register.

Lifetimes are getting shorter

The CA/Browser Forum has adopted a schedule that reduces the maximum validity of publicly trusted TLS certificates:

The schedule applies to publicly trusted certificates, not to a private internal CA, which sets its own policy. Its consequence is simple arithmetic. At 47 days, you must renew about 8 times a year (365 ÷ 47 = 7.8, and more when renewing early). A manual process that was annoying once a year becomes untenable, so automate renewal now. ACME, the protocol defined in RFC 8555, is the standard for this, and it also works with private CAs that support it.

Setting alert thresholds

A fixed "30 days" alert does not suit every lifetime. With a 47-day certificate on healthy automation, 30 days remaining is normal, and alerting then would be pure noise. Base thresholds on when renewal should have already happened. A common convention is to renew when about one third of the lifetime remains.

The table is an example of that reasoning, and the numbers are rounded:

For manually renewed certificates, give more notice, because humans need procurement, change windows and approvals. Sixty to ninety days is common for those.

The important distinction: an alert means automation has failed, or a person needs to begin a manual job. It should never be routine. If this alert fires regularly, fix the renewal rather than tuning the threshold. The alert fatigue guide covers that discipline more generally.

Besides the time remaining, also alert on these:

• The certificate served changed unexpectedly, or a different issuer appeared
• The hostname or chain failed validation
• The renewal job has failed, even with plenty of time left
• A renewed certificate on disk differs from the one that is being served

Rotation without an outage

Monitoring catches problems, and rotation design prevents them.

• Reload. Confirm that each service actually picks up a new certificate. Some reload on a signal, some watch the file and some need a restart. A rolling restart is a sound fallback. Note that Kubernetes updates mounted secrets but does not reload your process, and subPath mounts never update.
• Overlap the CA. When you rotate a CA, publish the new CA to all trust stores first, then issue from it, and retire the old one last. Skipping the first step is the classic self-inflicted outage.
• Test the failure. In a test environment, deploy an expired certificate on purpose. You'll learn what the errors look like, and whether your alerts work.
• Plan for air-gapped sites. Public ACME endpoints aren't reachable from closed networks, so run an internal CA, and include its root in your air-gapped install trust configuration.

Clients may need care as well. Certificate pinning breaks at rotation by design, so if you pin, pin to something that lasts longer than the leaf, and ship updates ahead of time.

Probing with Wrentrix

Wrentrix's synthetic monitoring runs HTTP(S) and TCP probes, and provides TLS certificate expiry and response-time alerts, all from inside your network, so internal endpoints are covered as well as public ones. It works from what you configure, so the inventory work above still applies. The alerts guide on noise applies to these alerts too.

Frequently asked questions

How do I check when a TLS certificate expires from the command line?

Run echo | openssl s_client -connect host:443 -servername host 2>/dev/null | openssl x509 -noout -enddate. This reports the certificate actually being served, which may differ from the file on disk. Use -checkend with a number of seconds for a scriptable check.

How do I monitor internal certificates that public scanners can't see?

Run probes from inside the network, and build an inventory from Kubernetes secrets, keystores, server configuration and internal port scans. Certificate Transparency logs generally do not include certificates from a private CA. Give each entry an owner so that alerts reach someone who can act on them.

How many days before expiry should a certificate alert fire?

It depends on the lifetime and on whether renewal is automated. With automation, alert after the point when renewal should already have happened, such as when less than a third of the lifetime remains, with a page much closer to expiry. For manual renewal, allow 60 to 90 days to cover procurement and change windows.

Why is my site serving an old certificate after renewal?

The server process probably loaded the old certificate at startup and hasn't reloaded it. Another possibility is a load balancer or another replica that holds its own copy. Probe each member directly, and add a reload step to the renewal process.

Do the 47-day certificate rules apply to internal certificates?

The CA/Browser Forum's schedule covers publicly trusted TLS certificates. A private CA used only within your organisation sets its own lifetimes, subject to your clients' behaviour and your own policies. Many organisations are still shortening internal lifetimes, since automation built for one case often serves the other.