[Kubernetes (K8S)] Access Service or Pod with 80 or 443 port outside macOS Docker-Desktop Kubernetes (K8S)

Posted on 2020-10-31 Edited on 2023-09-02 In Cloud Native , Kubernetes (K8S) Views: Word count in article: 6.3k Reading time ≈ 6 mins.

There are some ways accessing Service with 80 or 443 port outside macOS Docker-Desktop Kubernetes (K8S), include LoadBalancer, NodePort, and Port Forward(kubectl port-forward, socat, Packet Filter(pf), etc.)

LoadBalancer

On cloud providers which support external load balancers, setting the type field to LoadBalancer provisions a load balancer for your Service. The actual creation of the load balancer happens asynchronously, and information about the provisioned balancer is published in the Service’s .status.loadBalancer field.

On macOS you can specify Sevice type loadBalancer which loadBalancerIP is within 127.0.0.1/24 subnet(such as 127.0.0.240).

# nginx.LoadBalancer.yaml

apiVersion: apps/v1beta2
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1
        ports:
        - name: http
          containerPort: 80

---
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  ports:
  - name: http
    port: 8080
    protocol: TCP
    targetPort: 30080
  - name: http
    port: 443
    protocol: TCP
    targetPort: 32443
  selector:
    app: nginx
  type: LoadBalancer
  loadBalancerIP: 127.0.0.240

Then, apply the manifest file.

1	$ kubectl apply -f nginx.LoadBalancer.yaml

Check Service status.

1
2
3

kubectl get svc
NAMESPACE NAME      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
default   nginx  LoadBalancer   10.104.166.92    localhost     80:30080/TCP,443:32443/TCP   2d9h

You can see the EXTERNAL-IP will be assign to localhost.

Once the service is up and running, on Mac this will only be routable by visiting either 127.0.0.1 or localhost. You can go to it on a browser at http://127.0.0.1, https://127.0.0.1, http://localhost or https://localhost.

NodePort

Change kube-apiserver --service-node-port-range parameters

Exposes the Service on each Node’s IP at a static port (the NodePort). A ClusterIP Service, to which the NodePort Service routes, is automatically created. You’ll be able to contact the NodePort Service, from outside the cluster, by requesting :.

If you set the type field to NodePort, the Kubernetes control plane allocates a port from a range specified by --service-node-port-range flag (default: 30000-32767). Each node proxies that port (the same port number on every Node) into your Service. Your Service reports the allocated port in its .spec.ports[*].nodePort field.

The following way to change kube-apiserver --service-node-port-range parameters for Docker-desktop Kubernetes (K8S) on macOS:

$ docker run -it --privileged --pid=host debian nsenter -t 1 -m -u -n -i sh

# or
$ docker run --rm -it --privileged --pid=host walkerlee/nsenter -t 1 -m -u -i -n sh

Edit kube-apiserver.yaml (it’s one of static pods, they are created by kubelet using files in /etc/kubernetes/manifests)

1	$ vi /etc/kubernetes/manifests/kube-apiserver.yaml

Add the following line to the pod spec:

# kube-apiserver.yaml/etc/kubernetes/manifests/kube-apiserver.yaml

spec:
  containers:
  - command:
    - kube-apiserver
    - --advertise-address=192.168.65.3
    ...
    - --service-node-port-range=1-65535   # <-- add this line
    ...

Save and exit. Remember to restart Docker-Desktop, then make kube-apiserver restart with new parameters.
Check kube-apiserver with new parameters.

1 2	$ kubectl describe pod kube-apiserver-docker-desktop -n kube-system \| grep service-node-port-range --service-node-port-range=1-65535

Forward port

kubectl port-forward

kubectl port-forward allows using resource name, such as a service name, to select a matching service to forward a local port to a port on the Service

1
2
3

$ kubectl port-forward service/nginx 80:32080

$ kubectl port-forward service/nginx 80:32443

You can go to it on a browser at http://127.0.0.1, https://127.0.0.1, http://localhost or https://localhost.

socat

socat - http://www.dest-unreach.org/socat/ socat is a relay for bidirectional data transfer between two independent data channels. Each of these data channels may be a file, pipe, device (serial line etc. or a pseudo terminal), a socket (UNIX, IP4, IP6 - raw, UDP, TCP), an SSL socket, proxy CONNECT connection, a file descriptor (stdin etc.), the GNU line editor (readline), a program, or a combination of two of these. These modes include generation of “listening” sockets, named pipes, and pseudo terminals.

socat can be used, e.g., as TCP port forwarder (one-shot or daemon), as an external socksifier, for attacking weak firewalls, as a shell interface to UNIX sockets, IP6 relay, for redirecting TCP oriented programs to a serial line, to logically connect serial lines on different computers, or to establish a relatively secure environment (su and chroot) for running client or server shell scripts with network connections.

You can use socat to forward 80 or 43 ports to Kubernetes (K8S) Service NodePort 32080 or 32443.

# forward 80 port
socat -d -lf socat_ssh.log TCP4-LISTEN:80,reuseaddr,fork TCP4:127.0.0.1:32080

# forward 443 prot
socat -d -lf socat_ssh.log TCP4-LISTEN:443,reuseaddr,fork TCP4:127.0.0.1:32443

You can go to it on a browser at http://127.0.0.1, https://127.0.0.1, http://localhost or https://localhost.

pf

Packet Filter - https://www.openbsd.org/faq/pf/ (from here on referred to as PF) is OpenBSD’s system for filtering TCP/IP traffic and doing Network Address Translation. PF is also capable of normalizing and conditioning TCP/IP traffic, as well as providing bandwidth control and packet prioritization. PF has been a part of the GENERIC kernel since OpenBSD 3.0.

Use macOS PF firewall to forward 80 or 43 ports to Kubernetes (K8S) Service NodePort 32080 or 32443.

Add an anchor

# /etc/pf.conf

scrub-anchor "com.apple/*"
nat-anchor "com.apple/*"
nat-anchor "custom"
rdr-anchor "com.apple/*"
rdr-anchor "custom"
dummynet-anchor "com.apple/*"
anchor "com.apple/*"
anchor "custom"
load anchor "com.apple" from "/etc/pf.anchors/com.apple"

# add kuberneters pf rules.
load anchor "custom" from "/etc/pf.anchors/kuberneters"

Add pf rules.

1
2
3

# /etc/pf.anchors/kuberneters
rdr inet proto tcp from any to any port 80 -> 127.0.0.1 port 32080
rdr inet proto tcp from any to any port 443 -> 127.0.0.1 port 32443

Enable pf rules.