Renew AWX Certificates

From OISecWiki

AWX, the open-source upstream project for Ansible Tower/Ansible Autiomation Platform, can be deployed on Kubernetes distributions like K3S using the AWX Operator. Renewing SSL/TLS certificates for the AWX web interface typically involves updating the Kubernetes secret that holds the certificate and key, as the AWX Operator manages the deployment through a Custom Resource (CR). This guide focuses on manual renewal of custom or self-signed certificates, assuming an existing AWX installation with Ingress enabled for external access.

Prerequisites

  • An existing AWX deployment on K3S managed by the AWX Operator.
  • Access to the Kubernetes cluster (e.g., via kubectl).
  • New SSL/TLS certificate (.crt or .pem) and private key (.key) ready. These could be renewed from a Certificate Authority (CA) or self-generated.
  • Knowledge of the current TLS secret name used in your AWX CR (commonly something like awx-tls-secret).

If your setup uses cert-manager for automatic certificate management, renewal is handled automatically by cert-manager. This guide covers manual updates.

Step 1: Identify the TLS Secret

Inspect your AWX Custom Resource to find the TLS secret name. Run: 

kubectl get awx -n <your-awx-namespace> -o yaml

Look for fields under spec related to Ingress and TLS, such as:

  • ingress_type: ingress
  • ingress_hosts (with tls_secret)
  • ingress_tls_secret (deprecated, but may be used in older setups)

Example from AWX spec: 

spec:
  ingress_type: ingress
  ingress_hosts:
  - hostname: awx.example.com
    tls_secret: awx-tls-secret

Note the tls_secret value (e.g., awx-tls-secret).

Step 2: Generate or Obtain New Certificates

If using self-signed certificates, generate new ones with OpenSSL: 

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout new.key -out new.crt -subj "/CN=awx.example.com"

Replace awx.example.com with your AWX hostname. For production, obtain certificates from a trusted CA like Let's Encrypt.

Step 3: Update the Kubernetes Secret

Replace the contents of the existing TLS secret with the new certificate and key. Use kubectl apply to avoid deleting the secret (which could cause brief downtime).

First, create a YAML manifest for the new secret: 

kubectl create secret tls awx-tls-secret \
  --cert=new.crt \
  --key=new.key \
  -n <your-awx-namespace> \
  --dry-run=client -o yaml | kubectl apply -f -

This updates the secret in place.

If your setup uses kustomize (common in AWX on K3S examples), update the certificate files in your kustomization directory and re-apply: 

kubectl apply -k /path/to/your/kustomization

Step 4: Verify the Update

The AWX Operator and Ingress controller (e.g., Traefik in K3S) should automatically detect the secret change and apply the new certificate. Check the Ingress resource: 

kubectl get ingress -n <your-awx-namespace>
kubectl describe ingress <ingress-name> -n <your-awx-namespace>

Access the AWX web interface in a browser and verify the certificate details (e.g., expiration date).

If the update doesn't propagate, restart the Ingress controller pods or force reconciliation by editing the AWX CR (e.g., add a dummy annotation and apply).

Step 5: Troubleshooting

  • If AWX pods restart or fail, check logs: kubectl logs deployment/awx-operator-controller-manager -n <operator-namespace>.
  • For self-signed certs, ensure clients trust the new certificate.
  • If using cert-manager, refer to its documentation for renewal issues: Certificate Resource.

Examples

In a typical AWX on K3S setup using the kurokobo/awx-on-k3s repository, certificates are generated in the base directory. Update tls.crt and tls.key, then re-apply kubectl apply -k base.

For advanced configurations, such as trusting custom CAs for Git access, mount additional secrets as described in the official docs.

References

This article is based on AWX Operator version 2.x and K3S v1.28+. Always check the latest documentation for changes.

Retrieved from "https://wiki.oisec.net/mediawiki/index.php?title=Renew_AWX_Certificates&oldid=662"