MetalLBをKubernetesで使ってみる
環境
- Ubuntu 22.04.3 LTS
- minikube version: v1.32.0
MetalLB
Kubernetesはロードバランサーの機能を提供していないので、ローカルでServiceのtype: LoadBalancerとしてもEXTERNAL-IPが割当らずPending状態となってしまう。 MetalLBはローカルでローロバランサー機能を提供しtype: LoadBalancerとした場合に問題なくKubernetesにネットワークロードバランサーの機能を追加するためのソフトウェア。
- MetalLB
minikubeでMetalLBを利用する
MetalLBを有効にする
$ minikube addons enable metallb ! metallb is a 3rd party addon and is not maintained or verified by minikube maintainers, enable at your own risk. ! metallb does not currently have an associated maintainer. - Using image quay.io/metallb/speaker:v0.9.6 - Using image quay.io/metallb/controller:v0.9.6 * The 'metallb' addon is enabled $ minikube addons list |-----------------------------|----------|--------------|--------------------------------| | ADDON NAME | PROFILE | STATUS | MAINTAINER | |-----------------------------|----------|--------------|--------------------------------| | ambassador | minikube | disabled | 3rd party (Ambassador) | | auto-pause | minikube | disabled | minikube | | cloud-spanner | minikube | disabled | Google | | csi-hostpath-driver | minikube | disabled | Kubernetes | | dashboard | minikube | disabled | Kubernetes | | default-storageclass | minikube | enabled ? | Kubernetes | | efk | minikube | disabled | 3rd party (Elastic) | | freshpod | minikube | disabled | Google | | gcp-auth | minikube | disabled | Google | | gvisor | minikube | disabled | minikube | | headlamp | minikube | disabled | 3rd party (kinvolk.io) | | helm-tiller | minikube | disabled | 3rd party (Helm) | | inaccel | minikube | disabled | 3rd party (InAccel | | | | | [info@inaccel.com]) | | ingress | minikube | disabled | Kubernetes | | ingress-dns | minikube | disabled | minikube | | inspektor-gadget | minikube | disabled | 3rd party | | | | | (inspektor-gadget.io) | | istio | minikube | disabled | 3rd party (Istio) | | istio-provisioner | minikube | disabled | 3rd party (Istio) | | kong | minikube | disabled | 3rd party (Kong HQ) | | kubeflow | minikube | disabled | 3rd party | | kubevirt | minikube | disabled | 3rd party (KubeVirt) | | logviewer | minikube | disabled | 3rd party (unknown) | | metallb | minikube | enabled ? | 3rd party (MetalLB) | | metrics-server | minikube | disabled | Kubernetes | | nvidia-device-plugin | minikube | disabled | 3rd party (NVIDIA) | | nvidia-driver-installer | minikube | disabled | 3rd party (Nvidia) | | nvidia-gpu-device-plugin | minikube | disabled | 3rd party (Nvidia) | | olm | minikube | disabled | 3rd party (Operator Framework) | | pod-security-policy | minikube | disabled | 3rd party (unknown) | | portainer | minikube | disabled | 3rd party (Portainer.io) | | registry | minikube | disabled | minikube | | registry-aliases | minikube | disabled | 3rd party (unknown) | | registry-creds | minikube | disabled | 3rd party (UPMC Enterprises) | | storage-provisioner | minikube | enabled ? | minikube | | storage-provisioner-gluster | minikube | disabled | 3rd party (Gluster) | | storage-provisioner-rancher | minikube | disabled | 3rd party (Rancher) | | volumesnapshots | minikube | disabled | Kubernetes | |-----------------------------|----------|--------------|--------------------------------|
minikubeを起動
$ minikube start --cpus 4 --memory 4096 * minikube v1.32.0 on Ubuntu 22.04 (hyperv/amd64) * Using the docker driver based on existing profile * Starting control plane node minikube in cluster minikube * Pulling base image ... * Restarting existing docker container for "minikube" ... * Preparing Kubernetes v1.28.3 on Docker 24.0.7 ... * Configuring bridge CNI (Container Networking Interface) ... * Verifying Kubernetes components... - Using image gcr.io/k8s-minikube/storage-provisioner:v5 - Using image quay.io/metallb/speaker:v0.9.6 - Using image quay.io/metallb/controller:v0.9.6 * Enabled addons: storage-provisioner, metallb, default-storageclass * Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
Kubernetesで環境を構築
Serviceの所でEXTERNAL-IPがついていることを確認。 EXTERNAL-IPがついているのでMetalLBが動作していることがわかる。
$ kubectl apply -f deployment-nginx.yaml deployment.apps/nginx created $ kubectl apply -f service-nginx.yaml service/nginx created configmap/config unchanged $ kubectl get all NAME READY STATUS RESTARTS AGE pod/nginx-56cfb68995-7z4rr 1/1 Running 0 41s pod/nginx-56cfb68995-knk7v 1/1 Running 0 41s pod/nginx-56cfb68995-mw4gs 1/1 Running 0 41s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h16m service/nginx LoadBalancer 10.98.136.238 192.168.1.240 80:31742/TCP 37s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/nginx 3/3 3 3 42s NAME DESIRED CURRENT READY AGE replicaset.apps/nginx-56cfb68995 3 3 3 42s
EXTERNAL-IPでアクセス
ホストに到達できないと表示される
# ホストからkubernetesのEXTERNAL-IPへアクセス $ curl http://192.168.1.240 curl: (7) Failed to connect to 192.168.1.240 port 80 after 3058 ms: No route to host
アクセスできない原因はEXTERNAL-IP経由でアクセスするにはminikube経由でなければいけないから。
ホスト → minikube → EXTERNAL-IP経由でなければアクセスできない
minikube経由でhttp://192.168.1.240へアクセスできるか確認。 問題がなくアクセスできる
# minikubeにsshでログインしてcurlを実行
$ minikube ssh -- curl http://192.168.1.240
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
ポートフォワードして接続
minikubeにsshで接続するには2つの方法がある。 1個目はminikubeの機能でsshを使ってminikubeにログイン。 2個目は通常のsshでのログイン。(今回はこっちを使う)
minikube ssh
$ minikube ssh docker@minikube:~$
sshを使ってminikubeへログイン
下記のYAMLのserverの所に書かれているIPでsshでアクセスする。
古い情報だとid: docker, pass: tcuserを使ってsshでログインできるとあるが、現在はパスワードでのsshログインは禁止されている。
このため、ログインするにはid_rsaを利用する。
$ kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority: /home/admin/.minikube/ca.crt
extensions:
- extension:
last-update: Sun, 04 Feb 2024 21:28:22 JST
provider: minikube.sigs.k8s.io
version: v1.32.0
name: cluster_info
server: https://192.168.49.2:8443 ← これがminikuberにログインするためのIP
name: minikube
contexts:
- context:
cluster: minikube
extensions:
- extension:
last-update: Sun, 04 Feb 2024 21:28:22 JST
provider: minikube.sigs.k8s.io
version: v1.32.0
name: context_info
namespace: default
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate: /home/admin/.minikube/profiles/minikube/client.crt
client-key: /home/admin/.minikube/profiles/minikube/client.key
# 鍵の場所を確認
$ minikube ssh-key
/home/admin/.minikube/machines/minikube/id_rsa
# ログイン
$ ssh docker@192.168.49.2 -i /home/admin/.minikube/machines/minikube/id_rsa
docker@minikube:~$ exit
logout
Connection to 192.168.49.2 closed.
# ホストの8080ポートをminikuberの192.168.1.240:80ポートへポートフォワードする
$ ssh -N docker@192.168.49.2 -i /home/admin/.minikube/machines/minikube/id_rsa -L 8080:192.168.1.240:80
# 別にもう一個ターミナルを開いて
# ホストの8080にcurlでアクセスしてレスポンスが返ってくることを確認
$ curl http://127.0.0.1:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
$
参考ファイル
deployment.yaml
apiVersion: apps/v1 kind: Deployment metadata: name: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1 ports: - name: http containerPort: 80
service.yaml
apiVersion: v1 kind: Service metadata: name: nginx spec: ports: - name: http port: 80 protocol: TCP targetPort: 80 selector: app: nginx type: LoadBalancer --- apiVersion: v1 kind: ConfigMap metadata: namespace: metallb-system name: config data: config: | address-pools: - name: my-ip-space protocol: layer2 addresses: - 192.168.1.240/28
参考にしたサイト
- kubernetes: minikubeで立ち上げたIngressにアクセスできなかった問題を解決した
https://qiita.com/kenshow-blog/items/d5ac12637b7fb16a3d55 - オンプレK8sで使えるGoogle製External Load Balancer: MetalLB
https://qiita.com/tmatsu/items/f45f0ca07b4f8489df85
