CKA concept notes: Gateway API and Ingress

Based on: commands checked on a kubeadm cluster v1.35.9, Gateway API v1.6.1 CRDs (standard channel), NGINX Gateway Fabric v2.7.2, ingress-nginx chart 4.15.1 (controller 1.15.1)

Ingress resource basics (host/path rules, pathType, TLS) are in CKAD concept notes: Service, Ingress, NetworkPolicy.

Contents

• Gateway API
• Installation
• Gateway and HTTPRoute
• Reading status
• Other namespaces
• NodePort and externalTrafficPolicy
• Ingress controllers
• Ingress vs Gateway API
• Easily confused
• Exam tips
• References

Gateway API

An official Kubernetes API (gateway.networking.k8s.io) built as Ingress’s successor. It isn’t part of Kubernetes itself, so you install its CRDs separately.

It splits what used to be crammed into one Ingress into resources by role.

GatewayClass (nginx) ◀── Gateway (web-gw, :80) ◀── HTTPRoute (app) ──▶ Service app-v1, app-v2

Installation

1. the Gateway API CRDs
2. a Gateway controller (an implementation). The controller creates the GatewayClass, or you create it yourself

kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/standard?ref=v2.7.2" \
  | kubectl apply -f -
helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --version 2.7.2 \
  --create-namespace -n nginx-gateway --set nginx.service.type=NodePort

$ kubectl get gatewayclass
NAME    CONTROLLER                                   ACCEPTED
nginx   gateway.nginx.org/nginx-gateway-controller   True

• the CRDs come in a standard channel (GA and beta) and an experimental channel. The kustomization above is standard (bundle-version: v1.6.1)
• I had already installed the v1.6.3 CRDs and then applied v1.6.1, and it went through as is. A ValidatingAdmissionPolicy safe-upgrades.gateway.networking.k8s.io is installed along with the CRDs
• --set nginx.service.type=NodePort: the default is LoadBalancer, so on a cluster without an LB implementation I changed it to NodePort

Gateway and HTTPRoute

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: web-gw
  namespace: gw
spec:
  gatewayClassName: nginx
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      hostname: "*.example.com"
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: app
  namespace: gw
spec:
  parentRefs:
    - name: web-gw
  hostnames: [app.example.com]
  rules:
    - matches:
        - path: { type: PathPrefix, value: /v2 }
      backendRefs:
        - { name: app-v2, port: 80 }
    - matches:
        - headers:
            - { name: x-canary, value: "true" }
      backendRefs:
        - { name: app-v2, port: 80 }
    - backendRefs:
        - { name: app-v1, port: 80, weight: 80 }
        - { name: app-v2, port: 80, weight: 20 }

When you create a Gateway, NGINX Gateway Fabric creates a data plane per Gateway (a Deployment + Service web-gw-nginx).

$ kubectl get gateway,httproute
gateway.gateway.networking.k8s.io/web-gw   nginx   10.111.208.129   True (PROGRAMMED)
httproute.gateway.networking.k8s.io/app    ["app.example.com"]

Request results (Host: app.example.com):

• weight is a ratio, so 100 requests won’t split exactly 80:20
• matching: path (Exact, PathPrefix, RegularExpression), headers, queryParams, method. Within one matches item it’s AND; multiple items are OR
• filters for header modification, redirects, URL rewrites, mirroring

Reading status

With the Gateway API, when something goes wrong the reason almost always ends up in a status condition.

kubectl get gateway web-gw -o jsonpath='{range .status.conditions[*]}{.type}={.status} {end}'
# Accepted=True Programmed=True

kubectl get httproute app -o jsonpath='{range .status.parents[0].conditions[*]}{.type}={.status} {.reason}{"\n"}{end}'
# Accepted=True Accepted
# ResolvedRefs=True ResolvedRefs

After changing a backend name to a Service that doesn’t exist (app-v3):

Accepted=True Accepted
ResolvedRefs=False BackendNotFound: spec.rules[0].backendRefs[0].name: Not found: "app-v3"

That path (/v2) returns 500. The other rules keep working.

Other namespaces

When the Route is in another namespace: allowedRoutes

A listener’s allowedRoutes defaults to {"namespaces":{"from":"Same"}}. If an HTTPRoute in the other namespace names gw/web-gw as its parent:

Accepted=False NotAllowedByListeners: The Route is not allowed by any listener

  listeners:
  - name: http
    ...
    allowedRoutes:
      namespaces:
        from: All        # Same | All | Selector

After the change, Accepted=True.

When the backend is in another namespace: ReferenceGrant

If an HTTPRoute in the gw namespace names a Service in the other namespace as its backend:

ResolvedRefs=False RefNotPermitted: spec.rules[0].backendRefs[0].namespace: Forbidden:
Backend ref to Service other/echo not permitted by any ReferenceGrant

The permission goes in the backend’s (other) namespace.

apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
  name: allow-gw-routes
  namespace: other
spec:
  from:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      namespace: gw
  to:
    - group: ""
      kind: Service

After applying it, ResolvedRefs=True, and /v2 requests go to the Service in other. The v1.6 CRDs also have ReferenceGrant v1, but I verified with v1beta1 here.

NodePort and externalTrafficPolicy

The Gateway data plane Service was created with externalTrafficPolicy: Local. The data plane Pod was only on cka-w2, so requests to cka-w1’s NodePort timed out with no response (000), and only requests to cka-w2 got 200.

Ingress controllers

An Ingress resource on its own does nothing. You need a controller. I installed ingress-nginx with Helm.

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm install ingress-nginx ingress-nginx/ingress-nginx -n ingress-nginx --create-namespace \
  --set controller.service.type=NodePort
kubectl get ingressclass
# nginx   k8s.io/ingress-nginx

kubectl create ingress a --class=nginx --rule="a.example.com/*=a:80"

• this chart’s controller Service is externalTrafficPolicy: Cluster. Both workers’ NodePorts responded
• with Host: a.example.com you get the backend’s response; with no Host, the controller’s default 404
• an Ingress created without --class (b) has CLASS <none> and returns 404. Without a default IngressClass (the ingressclass.kubernetes.io/is-default-class: "true" annotation), no controller handles it

Ingress vs Gateway API

Easily confused

Exam tips

Quick commands

kubectl get gatewayclass
kubectl get gateway,httproute -A
kubectl describe httproute <name>          # Status.Parents.Conditions
kubectl get httproute <name> -o yaml | grep -A20 "^status"
kubectl get ingressclass
kubectl create ingress <name> --class=<class> --rule="host/path*=svc:port"

• Gateway API YAML has no kubectl create generator. Copy from the examples on gateway-api.sigs.k8s.io or the Gateway API page on kubernetes.io
• Ingress → Gateway API migration tasks: move the Ingress host to hostnames, path to matches.path, backend to backendRefs, and connect to the Gateway with parentRefs
• if a Route won’t attach, check the Gateway listener’s hostname, allowedRoutes, and protocol first

kubectl explain

kubectl explain gateway.spec.listeners
kubectl explain httproute.spec.rules.matches
kubectl explain httproute.spec.rules.backendRefs
kubectl explain referencegrant.spec

References

• Gateway API (kubernetes.io)
• Gateway API docs
• HTTPRoute
• ReferenceGrant
• Ingress Controllers
• NGINX Gateway Fabric: Install with Helm
• Service: externalTrafficPolicy
• CKA Curriculum v1.35