Automatic DNS With external-dns, UniFi, and Cilium BGP
Every homelab reaches the point where you deploy a new service, it gets an IP, and then you trot off to the router UI to add yet another DNS record by hand. I did that for years. This is the setup that finally made me stop: Cilium announces LoadBalancer IPs to my UniFi gateway over BGP, and external-dns writes the DNS records into UniFi automatically. Deploy an HTTPRoute, get a routable IP and a resolvable name, touch nothing.
This post is intentionally short and mostly configuration. In the Migrating Away From GitHub article I already covered how the Cilium Gateway API entry point itself is set up, so I won’t repeat that here.
BGP between UniFi and Cilium
Cilium hands out LoadBalancer IPs from a pool, but those IPs live on no physical network. Something has to tell the router where they are, and that something is BGP. UniFi gateways run FRR under the hood, and since UniFi Network 9.x you can upload an FRR config directly in the UI under Settings, Routing, BGP.
Mine peers with every K8s node using a private ASN, iBGP style:
router bgp 64512
bgp router-id 10.42.0.1
bgp log-neighbor-changes
neighbor 10.42.3.1 remote-as 64512
neighbor 10.42.3.1 description "debian-3001"
neighbor 10.42.3.2 remote-as 64512
neighbor 10.42.3.2 description "debian-3002"
! ...one stanza per node...
address-family ipv4 unicast
neighbor 10.42.3.1 activate
neighbor 10.42.3.2 activate
exit-address-family
On the Cilium side, the BGP control plane needs to be enabled in the Helm values:
bgpControlPlane:
enabled: true
Then a pool to allocate LoadBalancer IPs from:
apiVersion: "cilium.io/v2alpha1"
kind: CiliumLoadBalancerIPPool
metadata:
name: "lb-pool-1"
spec:
blocks:
- cidr: "10.128.0.1/24"
And the three resources that make the announcing happen. The cluster config says who to peer with:
apiVersion: cilium.io/v2
kind: CiliumBGPClusterConfig
metadata:
name: bgp-cluster-config
spec:
nodeSelector:
matchLabels:
bgp-policy: a
bgpInstances:
- name: instance-64512
localASN: 64512
peers:
- name: peer-router
peerASN: 64512
peerAddress: 10.42.0.1
peerConfigRef:
name: cilium-peer
The peer config sets timers and points at the advertisement:
apiVersion: cilium.io/v2
kind: CiliumBGPPeerConfig
metadata:
name: cilium-peer
spec:
timers:
connectRetryTimeSeconds: 120
holdTimeSeconds: 90
keepAliveTimeSeconds: 30
families:
- afi: ipv4
safi: unicast
advertisements:
matchLabels:
advertise: bgp
And the advertisement itself says what to announce, in my case LoadBalancer IPs of all services. The NotIn selector with a value nobody will ever use is the standard trick for “match everything”, because the selector is mandatory:
apiVersion: cilium.io/v2
kind: CiliumBGPAdvertisement
metadata:
name: bgp-advertisements
labels:
advertise: bgp
spec:
advertisements:
- advertisementType: Service
service:
addresses:
- LoadBalancerIP
selector:
matchExpressions:
- key: somekey
operator: NotIn
values:
- never-used-value
The nodes also need the label the cluster config selects on:
kubectl label node debian-3001 bgp-policy=a
Verification is one command:
$ cilium bgp peers
Node Local AS Peer AS Peer Address Session State ...
debian-3001 64512 64512 10.42.0.1 established
From that moment, every LoadBalancer service in the cluster gets an IP from the pool that is reachable from anywhere on my network.
external-dns writing into UniFi
Routable IPs are only half of the job, I still want names. UniFi is already the DNS server for my LAN, so instead of running yet another DNS service in the cluster, external-dns writes records straight into it through the external-dns-unifi-webhook provider from the home-operations folks.
I deploy it with the official Helm chart through Argo CD, but the interesting part is just the values:
fullnameOverride: external-dns
# Publish HTTPRoute hostnames -> Cilium gateway LB IP into UniFi DNS.
# `crd` adds DNSEndpoint resources for records with no Gateway API source.
sources:
- gateway-httproute
- crd
domainFilters:
- mydomain
# sync (not upsert-only) so deleting an HTTPRoute also removes its
# record; ownership TXT records limit deletion to records it created.
policy: sync
txtOwnerId: k8s
provider:
name: webhook
webhook:
image:
repository: ghcr.io/home-operations/external-dns-unifi-webhook
tag: "0.10.9"
env:
- name: UNIFI_HOST
value: https://10.42.0.1
- name: UNIFI_API_KEY
valueFrom:
secretKeyRef:
name: unifi-dns
key: UNIFI_API_KEY
The API key comes from the UniFi console under Control Plane, Integrations, API keys, and lives in a plain secret:
apiVersion: v1
kind: Secret
metadata:
name: unifi-dns
namespace: external-dns
type: Opaque
stringData:
UNIFI_API_KEY: <redacted>
With the gateway-httproute source, every HTTPRoute attached to my Cilium Gateway gets its hostname published to UniFi, pointing at the gateway’s LoadBalancer IP, the very one BGP just made reachable. Deploying a new service is now a Deployment, a Service, and an HTTPRoute, and DNS sorts itself out.
Records that don’t live in the cluster
Not everything I want in DNS runs in K8s. Proxmox hosts, Home Assistant, the NFS box. For those, the crd source lets me declare records as a DNSEndpoint resource, so even my “static” DNS entries are YAML in git instead of clicks in a UI:
apiVersion: externaldns.k8s.io/v1alpha1
kind: DNSEndpoint
metadata:
name: custom-records
namespace: external-dns
spec:
endpoints:
- dnsName: homeassistant.mydomain
recordType: A
recordTTL: 300
targets:
- 10.42.1.100
- dnsName: proxmox.mydomain
recordType: A
recordTTL: 300
targets:
- 10.42.0.10
The DNSEndpoint CRD itself is not installed by the chart, you grab it from the external-dns repository
and apply it yourself. In my case Argo CD handles that too. The Application is a multi-source one: the first source is the Helm chart with the values shown above, the second is a path in my git repository holding the CRD, the DNSEndpoint resources, and the secret:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: external-dns
namespace: argocd
spec:
project: infrastructure
sources:
- chart: external-dns
repoURL: https://kubernetes-sigs.github.io/external-dns/
targetRevision: 1.21.1
helm:
releaseName: external-dns
values: |
# values from above
- repoURL: git@git.mydomain:ivan/k8s-apps.git
targetRevision: main
path: external-dns
destination:
server: https://kubernetes.default.svc
namespace: external-dns
syncPolicy:
automated:
selfHeal: true
syncOptions:
- CreateNamespace=true
Argo CD applies everything in that path as plain manifests, CRD included, so the chart and the resources it needs land together in one sync.
One note on policy: sync: it means external-dns will also delete records, which sounds scary for a DNS server that holds the whole household together. The ownership TXT records, tagged with the txtOwnerId, make sure the controller only manages records it created itself. My manually created records from the before times are safe until I get around to converting them.
Closing thoughts
The whole thing is around a hundred lines of YAML plus one FRR config, and I have not opened the DNS page in the UniFi UI since. The router learns routes from the cluster and the cluster writes names into the router, which feels like the two of them finally agreed to talk to each other without me playing interpreter .