CoreDNS in Kubernetes with Helm

Since my last post I have moved almost everything in my home lab to a three-node Kubernetes cluster. I ported my DNS environment without major changes but with PiHole running as a pod within Kubernetes. PiHole is not only primary DNS for my personal devices but it’s also authoritative DNS for the home environment. Authoritative DNS isn’t PiHole’s core competency so it doesn’t do it well and was causing some complexity. To improve this situation I’ve installed CoreDNS as the first step of my DNS redesign. CoreDNS will be authoritative and will consider PiHole upstream. CoreDNS has an official Helm chart which has some confusing aspects for my relevant values.

Normally when I install a Helm chart, I write a values file from scratch. This particular Helm chart has a significant amount of boilerplate value information which, when absent, may cause missing information in the templated output. Some of this boilerplate is important for readiness and liveness tests so I copied the values file and made the necessary modifications. Unfortunately, the Helm chart isn’t obvious if your configuration goes beyond the basics. I wanted my configuration to look something like this:

.:53 {
	forward . 1.1.1.1
	log
}

subdomain.kevinbreit.net:53 {
	file /etc/coredns/subdomain.db
}

Then the subdomain.db file would use BIND formatted syntax to define the domains in the zone. Using most of the default values gave me an error stating a service can’t support multiple protocols. This design is common since DNS supports both TCP and UDP requests. I found an issue that described my exact scenario and it suggests I force CoreDNS to disable TCP. In other words, the Helm chart won’t create separate services for both TCP and UDP. The values syntax to configure this is

      servers:
        - zones:
            - zone: .
              scheme: dns://
              use_tcp: false

This resolved the dual-protocol support error. If I choose to deploy DNS-over-HTTPS(DOH) I will need to reconsider, but for now, it works.

CoreDNS’s relevant values structure looks like

# Default zone is what Kubernetes recommends:
# https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#coredns-configmap-options
servers:
- zones:
  - zone: .
  port: 53
  plugins:
  - name: errors
  # Serves a /health endpoint on :8080, required for livenessProbe
  - name: health
    configBlock: |-
      lameduck 5s
  # Serves a /ready endpoint on :8181, required for readinessProbe
  - name: ready
  # Required to query kubernetes API for data
  - name: kubernetes
    parameters: cluster.local in-addr.arpa ip6.arpa
    configBlock: |-
      pods insecure
      fallthrough in-addr.arpa ip6.arpa
      ttl 30
  # Serves a /metrics endpoint on :9153, required for serviceMonitor
  - name: prometheus
    parameters: 0.0.0.0:9153
  - name: forward
    parameters: . /etc/resolv.conf
  - name: cache
    parameters: 30
  - name: loop
  - name: reload
  - name: loadbalance

# Complete example with all the options:
# - zones:                 # the `zones` block can be left out entirely, defaults to "."
#   - zone: hello.world.   # optional, defaults to "."
#     scheme: tls://       # optional, defaults to "" (which equals "dns://" in CoreDNS)
#   - zone: foo.bar.
#     scheme: dns://
#     use_tcp: true        # set this parameter to optionally expose the port on tcp as well as udp for the DNS protocol
#                          # Note that this will not work if you are also exposing tls or grpc on the same server
#   port: 12345            # optional, defaults to "" (which equals 53 in CoreDNS)
#   plugins:               # the plugins to use for this server block
#   - name: kubernetes     # name of plugin, if used multiple times ensure that the plugin supports it!
#     parameters: foo bar  # list of parameters after the plugin
#     configBlock: |-      # if the plugin supports extra block style config, supply it here
#       hello world
#       foo bar

Notice how zones is a list within the servers key with each zone defined within zones. The plugin stanza lives within the servers list and not within the individual zones. My first attempts at adding zone stanzas to Corefile were based on the zones list. While I did get it working, it wasn’t clean and made little sense. Instead, the proper way to add zone stanzas is to add items to the server list. Here is an abbreviated values section.

servers:
- zones:
  - zone: .
    scheme: dns://
    use_tcp: false
  port: 53
  plugins:
...
  - name: forward
    parameters: . 1.1.1.1
  - name: cache
    parameters: 30
...
- zones:
  - zone: subdomain.kevinbreit.net
    scheme: dns://
    use_tcp: false
  port: 53
  plugins:
  - name: file
    parameters: /etc/coredns/subdomain.db subdomain.kevinbreit.net

I’ll need to delve into Corefile details to understand why zones isn’t what defines the stanzas but the rendered output is

data:
  Corefile: |-
    dns://.:53 {
        errors
        health {
            lameduck 5s
        }
        ready
        prometheus 0.0.0.0:9153
        forward . 1.1.1.1
        cache 30
        loop
        reload
        loadbalance
    }
    dns://subdomain.kevinbreit.net:53 {
        file /etc/coredns/subdomain.db subdomain.kevinbreit.net
    }

This output is exactly what I was looking for. Thankfully, defining the zone file itself isn’t too confusing.

zoneFiles:
  - filename: home.db
    domain: subdomain.kevinbreit.net
    contents: |
      $ORIGIN subdomain.kevinbreit.net.
...

If you’re looking to use the Helm chart with multiple zones and an RFC 1035 file format, this is a good template to follow. CoreDNS’s Helm chart documentation could be more intuitive but changes would be a major release as it would break backward compatibility.