-
[Kubernetes] Visual하게 DashBoard를 설치, 접속해보자인프라/Kubernetes 2020. 11. 26. 16:37
kubernetes/dashboard
General-purpose web UI for Kubernetes clusters. Contribute to kubernetes/dashboard development by creating an account on GitHub.
github.com
# kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.4/aio/deploy/recommended.yaml namespace/kubernetes-dashboard created serviceaccount/kubernetes-dashboard created service/kubernetes-dashboard created secret/kubernetes-dashboard-certs created secret/kubernetes-dashboard-csrf created secret/kubernetes-dashboard-key-holder created configmap/kubernetes-dashboard-settings created role.rbac.authorization.k8s.io/kubernetes-dashboard created clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created deployment.apps/kubernetes-dashboard created service/dashboard-metrics-scraper created deployment.apps/dashboard-metrics-scraper createdrecommended.yaml apply!
# kubectl get services -n kubernetes-dashboard NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE dashboard-metrics-scraper ClusterIP 10.102.59.174 <none> 8000/TCP 15m kubernetes-dashboard ClusterIP 10.96.151.107 <none> 443/TCP 15mdashboard-metrics-scraper 같은경우 metrics-server를 미리 설치해서 그런지 자동으로 dashboard에서 사용가능하도록 연결이 된 것 같다.
kubectl proxy다른 터미널로 proxy 실행
# curl http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/ <!-- Copyright 2017 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> <!doctype html> <html lang="en"> <head> <meta charset="utf-8"> <title>Kubernetes Dashboard</title> <link rel="icon" type="image/png" href="assets/images/kubernetes-logo.png"/> <meta name="viewport" content="width=device-width"> <link rel="stylesheet" href="styles.c3ed2dcd657a389ecc4d.css"></head> <body> <kd-root></kd-root> <script src="runtime.6304db2809b97aa812ee.js" defer=""></script><script src="polyfills-es5.8f06d415489cadffc1de.js" nomodule="" defer=""></script><script src="polyfills.36db5820637aca3bd1e6.js" defer=""></script><script src="scripts.e296fd4cf14eea7ea0bd.js" defer=""></script><script src="main.17bd8ead409f8f047d6a.js" defer=""></script></body> </html>api를 통하여 html을 얻을 수 있다면 성공적으로 dashboard가 실행되고 있다는 뜻이다!
위의 curl은 kube-proxy를 통해서 apiserver로 https:kubernetes-dashboard: 요청을 보내 html 응답을 반환해준 경우이다.
그러나 난 kube-proxy 통하지 않고 direct로 kubernetes-dashboard로 접속을 하고싶다.
kubectl edit services kubernetes-dashboard -n kubernetes-dashboardkubernetes-dashboard service를 수정해보자
# Please edit the object below. Lines beginning with a '#' will be ignored, # and an empty file will abort the edit. If an error occurs while saving this file will be # reopened with the relevant failures. # apiVersion: v1 kind: Service . . . spec: . . . type: NodePort status: loadBalancer: {}자세한 설정은 생략하고 spec.type이 ClusterIP로 되어있을텐데 NodePort로 변경하자! wq! 로 저장하고 나오면
# kubectl get services -n kubernetes-dashboard NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE dashboard-metrics-scraper ClusterIP 10.102.59.174 <none> 8000/TCP 19h kubernetes-dashboard NodePort 10.96.151.107 <none> 443:30692/TCP 19h이처럼 NodePort로 바뀌고 {hostip}:30692 포트로 접근이가능하다.
hostip 여기선 192.168.5.1:30692로 접속을 하면 인증서를 따로 등록하지 않아 경고가 발생한다.
인증서부분은 서버에서 생성하고, 접속할 클라이언트 브라우저에도 등록하고 하는 번거로운 과정이 필요하므로 고급을 눌러서 안전하지않지만 접속을 하자.
그럼 이렇게 login 화면으로 접속이 된다. 한번도 접속을 안했으면 빨간글씨가 없을것이다.
apiserver에 접근하여 클러스터 환경의 자원들을 가져와 대시보드를 꾸미려면 권한이 필요하다.
Role-based access control (RBAC)를 기반으로 apiserver에 인증, 권한을 관리하는데 쿠버네티스 대시보드에 로그인하고 올바른 인증을 거쳐야만 정상적으로 대시보드를 사용할 수 있다.
위에서 말하듯 모든 ServiceAccount(bcho.tistory.com/1272 블로그 참조)는 apiserver에 접근하기위하여 secret을 가지고 있고 이 안에는 Bearer Token값이 있다. http 프로토콜을 사용하여 인증하는 여러가지 방법 중 쿠버네티스 대시보드에 접속하기위해서 Bearer Token을 사용해 볼것이다.
# kubectl get serviceaccount -A NAMESPACE NAME SECRETS AGE default default 1 5d21h kube-node-lease default 1 5d21h kube-public default 1 5d21h kube-system admin-user 1 41m kube-system attachdetach-controller 1 5d21h kube-system bootstrap-signer 1 5d21h kube-system calico-kube-controllers 1 5d21h kube-system calico-node 1 5d21h kube-system certificate-controller 1 5d21h kube-system clusterrole-aggregation-controller 1 5d21h kube-system coredns 1 5d21h kube-system cronjob-controller 1 5d21h kube-system daemon-set-controller 1 5d21h kube-system default 1 5d21h kube-system deployment-controller 1 5d21h kube-system disruption-controller 1 5d21h kube-system endpoint-controller 1 5d21h kube-system endpointslice-controller 1 5d21h kube-system endpointslicemirroring-controller 1 5d21h kube-system expand-controller 1 5d21h kube-system generic-garbage-collector 1 5d21h kube-system horizontal-pod-autoscaler 1 5d21h kube-system job-controller 1 5d21h kube-system kube-proxy 1 5d21h kube-system metrics-server 1 25h kube-system namespace-controller 1 5d21h kube-system node-controller 1 5d21h kube-system persistent-volume-binder 1 5d21h kube-system pod-garbage-collector 1 5d21h kube-system pv-protection-controller 1 5d21h kube-system pvc-protection-controller 1 5d21h kube-system replicaset-controller 1 5d21h kube-system replication-controller 1 5d21h kube-system resourcequota-controller 1 5d21h kube-system service-account-controller 1 5d21h kube-system service-controller 1 5d21h kube-system statefulset-controller 1 5d21h kube-system token-cleaner 1 5d21h kube-system ttl-controller 1 5d21h kubernetes-dashboard default 1 19h kubernetes-dashboard kubernetes-dashboard 1 19h현재 나의 Master Node에는 이렇게 많은 ServiceAccount가 생성되어 있다.
# kubectl describe serviceaccount kubernetes-dashboard -n kubernetes-dashboard Name: kubernetes-dashboard Namespace: kubernetes-dashboard Labels: k8s-app=kubernetes-dashboard Annotations: <none> Image pull secrets: <none> Mountable secrets: kubernetes-dashboard-token-xq564 Tokens: kubernetes-dashboard-token-xq564 Events: <none>kubernetes-dashboard serviceaccount에 대해 살펴보니 secret token은 kubernetes-dashboard-token-xq564라고 한다.
kubernetes-dashboard에 대해서 secret을 살펴보자
# kubectl describe secret kubernetes-dashboard-token-xq564 -n kubernetes-dashboard Name: kubernetes-dashboard-token-xq564 Namespace: kubernetes-dashboard Labels: <none> Annotations: kubernetes.io/service-account.name: kubernetes-dashboard kubernetes.io/service-account.uid: 9f79057a-9afb-4e72-b52b-1abd61022fe6 Type: kubernetes.io/service-account-token Data ==== ca.crt: 1066 bytes namespace: 20 bytes token: eyJhbGciOiJSUzI1NiIsImtpZCI6IkRpUjl6SDRMUV9qaERqd3pBNkZUUXJIaDI4SVViRm1OZUVReXVjcUpjX3MifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi14cTU2NCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjlmNzkwNTdhLTlhZmItNGU3Mi1iNTJiLTFhYmQ2MTAyMmZlNiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.mxlrXaZlwo6lYHPqnnxnZ6bs5oRn_-UO5xXhhDGHcgTU_uD3UXc8xYp96iv-TgIYAP15wQJ6PAFxs78lRKGbX-WCTa5DhoXqSUZHtEHnNj1Ek3XM_xJGPBJJECBgm71fh599_J0BhTdgpeNMk-6dtmBF0sLecMew3lQESLID_a9o_SVYNB8WEw-lkVOXdXwA2eQbGU0q40UR34U9aIlIn9H3iPoHhJcb2H5pQAxhsulLpHWk8Y6IFUiNVqjD92_v8Wxz4OnU0JJOgAtG9C4ZkRefEq1kGxycSMECBJG6-VmWZNN4alcquHFQ7LgdpbRgpwr6W_YBS6dPYeHHbazJmg확인해보니 token 값이 보인다!
# kubectl describe secret kubernetes-dashboard-token-xq564 -n kubernetes-dashboard | grep token: | awk '{print $2}' eyJhbGciOiJSUzI1NiIsImtpZCI6IkRpUjl6SDRMUV9qaERqd3pBNkZUUXJIaDI4SVViRm1OZUVReXVjcUpjX3MifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi14cTU2NCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjlmNzkwNTdhLTlhZmItNGU3Mi1iNTJiLTFhYmQ2MTAyMmZlNiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.mxlrXaZlwo6lYHPqnnxnZ6bs5oRn_-UO5xXhhDGHcgTU_uD3UXc8xYp96iv-TgIYAP15wQJ6PAFxs78lRKGbX-WCTa5DhoXqSUZHtEHnNj1Ek3XM_xJGPBJJECBgm71fh599_J0BhTdgpeNMk-6dtmBF0sLecMew3lQESLID_a9o_SVYNB8WEw-lkVOXdXwA2eQbGU0q40UR34U9aIlIn9H3iPoHhJcb2H5pQAxhsulLpHWk8Y6IFUiNVqjD92_v8Wxz4OnU0JJOgAtG9C4ZkRefEq1kGxycSMECBJG6-VmWZNN4alcquHFQ7LgdpbRgpwr6W_YBS6dPYeHHbazJmggrep 과 awk를 사용하면 편하게 token 값만 뽑을 수 있다.
저 값을 입력해 주면 로그인이 될 것이다.
접속이 되었다!! 하지만 뭔가 이상하다 ... 대시보드가 정상적으로 동작을 안하는지 아무것도 표시되지 않는다.
알림을 보니 kubernetes-dashboard serviceaccount는 뭔가 자원에 접근할만한 "default" namespace에 속해있지 않다는 둥, roles, rolebindings API group에 속해있지 않다고 막 뜬다.
Using RBAC Authorization
Role-based access control (RBAC) is a method of regulating access to computer or network resources based on the roles of individual users within your organization. RBAC authorization uses the rbac.authorization.k8s.io API group to drive authorization decis
kubernetes.io
# kubectl describe clusterrole kubernetes-dashboard Name: kubernetes-dashboard Labels: k8s-app=kubernetes-dashboard Annotations: <none> PolicyRule: Resources Non-Resource URLs Resource Names Verbs --------- ----------------- -------------- ----- nodes.metrics.k8s.io [] [] [get list watch] pods.metrics.k8s.io [] [] [get list watch]clusterrole을 통해서 접근가능한 Resources를 확인가능한데, kubernetes-dashboard는 metrics 만 get, list, watch 가능하다고 한다.
# Please edit the object below. Lines beginning with a '#' will be ignored, # and an empty file will abort the edit. If an error occurs while saving this file will be # reopened with the relevant failures. # apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"rbac.authorization.k8s.io/v1","kind":"ClusterRole","metadata":{"annotations":{},"labels":{"k8s-app":"kubernetes-dashboard"},"name":"kubernetes-dashboard"},"rules":[{"apiGroups":["metrics.k8s.io"],"resources":["pods","nodes"],"verbs":["get","list","watch"]}]} creationTimestamp: "2020-11-25T09:12:00Z" labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard resourceVersion: "412630" selfLink: /apis/rbac.authorization.k8s.io/v1/clusterroles/kubernetes-dashboard uid: 936d856f-d702-4f9a-9c76-903662f3b78b rules: - apiGroups: - metrics.k8s.io resources: - pods - nodes verbs: - get - list - watchkubectl edit를 통해 들어가보면 rules.apiGroups가 metrics.k8s.io 만 등록되어 있다.
그래서 kubernetes-dashboard secret token으로는 metrics.k8s.io 라는 api밖에 사용을 하지못한다.
일단 어떤 clusterrole이 전체 Resources에 접근이 가능한지 알아보자.
# kubectl get clusterrole NAME CREATED AT admin 2020-11-20T07:31:11Z calico-kube-controllers 2020-11-20T07:32:41Z calico-node 2020-11-20T07:32:41Z cluster-admin 2020-11-20T07:31:11Z edit 2020-11-20T07:31:11Z kubeadm:get-nodes 2020-11-20T07:31:15Z kubernetes-dashboard 2020-11-25T09:12:00Z system:aggregate-to-admin 2020-11-20T07:31:11Z system:aggregate-to-edit 2020-11-20T07:31:11Z system:aggregate-to-view 2020-11-20T07:31:11Z system:aggregated-metrics-reader 2020-11-25T03:20:11Z system:auth-delegator 2020-11-20T07:31:11Z system:basic-user 2020-11-20T07:31:11Z system:certificates.k8s.io:certificatesigningrequests:nodeclient 2020-11-20T07:31:11Z system:certificates.k8s.io:certificatesigningrequests:selfnodeclient 2020-11-20T07:31:11Z system:certificates.k8s.io:kube-apiserver-client-approver 2020-11-20T07:31:11Z system:certificates.k8s.io:kube-apiserver-client-kubelet-approver 2020-11-20T07:31:11Z system:certificates.k8s.io:kubelet-serving-approver 2020-11-20T07:31:11Z system:certificates.k8s.io:legacy-unknown-approver 2020-11-20T07:31:11Z system:controller:attachdetach-controller 2020-11-20T07:31:11Z system:controller:certificate-controller 2020-11-20T07:31:11Z system:controller:clusterrole-aggregation-controller 2020-11-20T07:31:11Z system:controller:cronjob-controller 2020-11-20T07:31:11Z system:controller:daemon-set-controller 2020-11-20T07:31:11Z system:controller:deployment-controller 2020-11-20T07:31:11Z system:controller:disruption-controller 2020-11-20T07:31:11Z system:controller:endpoint-controller 2020-11-20T07:31:11Z system:controller:endpointslice-controller 2020-11-20T07:31:11Z system:controller:endpointslicemirroring-controller 2020-11-20T07:31:11Z system:controller:expand-controller 2020-11-20T07:31:11Z system:controller:generic-garbage-collector 2020-11-20T07:31:11Z system:controller:horizontal-pod-autoscaler 2020-11-20T07:31:11Z system:controller:job-controller 2020-11-20T07:31:11Z system:controller:namespace-controller 2020-11-20T07:31:11Z system:controller:node-controller 2020-11-20T07:31:11Z system:controller:persistent-volume-binder 2020-11-20T07:31:11Z system:controller:pod-garbage-collector 2020-11-20T07:31:11Z system:controller:pv-protection-controller 2020-11-20T07:31:11Z system:controller:pvc-protection-controller 2020-11-20T07:31:11Z system:controller:replicaset-controller 2020-11-20T07:31:11Z system:controller:replication-controller 2020-11-20T07:31:11Z system:controller:resourcequota-controller 2020-11-20T07:31:11Z system:controller:route-controller 2020-11-20T07:31:11Z system:controller:service-account-controller 2020-11-20T07:31:11Z system:controller:service-controller 2020-11-20T07:31:11Z system:controller:statefulset-controller 2020-11-20T07:31:11Z system:controller:ttl-controller 2020-11-20T07:31:11Z system:coredns 2020-11-20T07:31:16Z system:discovery 2020-11-20T07:31:11Z system:heapster 2020-11-20T07:31:11Z system:kube-aggregator 2020-11-20T07:31:11Z system:kube-controller-manager 2020-11-20T07:31:11Z system:kube-dns 2020-11-20T07:31:11Z system:kube-scheduler 2020-11-20T07:31:11Z system:kubelet-api-admin 2020-11-20T07:31:11Z system:metrics-server 2020-11-25T03:20:11Z system:node 2020-11-20T07:31:11Z system:node-bootstrapper 2020-11-20T07:31:11Z system:node-problem-detector 2020-11-20T07:31:11Z system:node-proxier 2020-11-20T07:31:11Z system:persistent-volume-provisioner 2020-11-20T07:31:11Z system:public-info-viewer 2020-11-20T07:31:11Z system:volume-scheduler 2020-11-20T07:31:11Z view 2020-11-20T07:31:11Z엄청 많은 clusterrole들이 만들어져 있다. clusterrole은 cluster환경 내에서만 접근가능한 role들이다.
저기서 cluster-admin 이라는 clusterrole이 눈에 보인다.
# kubectl describe clusterrole cluster-admin Name: cluster-admin Labels: kubernetes.io/bootstrapping=rbac-defaults Annotations: rbac.authorization.kubernetes.io/autoupdate: true PolicyRule: Resources Non-Resource URLs Resource Names Verbs --------- ----------------- -------------- ----- *.* [] [] [*] [*] [] [*]모든 Resources에 모든 Verbs를 사용할 수 있다
우리가 사용할 kubernetes-dashboard serviceaccount의 clusterrole인 kubernetes-dashboard도 모두 사용할 수 있도록 변경해보자.
# kubectl edit clusterrole kubernetes-dashboard# Please edit the object below. Lines beginning with a '#' will be ignored, # and an empty file will abort the edit. If an error occurs while saving this file will be # reopened with the relevant failures. # apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"rbac.authorization.k8s.io/v1","kind":"ClusterRole","metadata":{"annotations":{},"labels":{"k8s-app":"kubernetes-dashboard"},"name":"kubernetes-dashboard"},"rules":[{"apiGroups":["metrics.k8s.io"],"resources":["pods","nodes"],"verbs":["get","list","watch"]}]} creationTimestamp: "2020-11-25T09:12:00Z" labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard resourceVersion: "412630" selfLink: /apis/rbac.authorization.k8s.io/v1/clusterroles/kubernetes-dashboard uid: 936d856f-d702-4f9a-9c76-903662f3b78b rules: - apiGroups: #- metrics.k8s.io - '*' resources: #- pods #- nodes - '*' verbs: #- get #- list #- watch - '*'위에 보이는 것처럼 rules에 apiGroups, resources, verbs 에 있던것들을 다 지우고 '*'를 추가해주자
그리고 다시 token을 입력하면
위와 같이 알람없이 정상적으로 값들을 불러올 수 있게 되었다!
이제 dashboard로 Deployments, Service, Role등 모든 설정이 가능하다고 한다.
혹여나 clusterrole을 변경하는것이 신경쓰인다면
아래의 블로그를 참조하여 새로운 serviceaccount를 생성하고 clusterrolebindings를 이용하여 cluster-admin 권한을 부여하여 사용해도 된다.
Docker with Kubernetes #5 - Dashboard 설치
Kubernetes 를 설치하면서 고생했던 것이 두 가지가 있습니다. 하나가 방화멱(Firewalld )문제였고, 다른 하나가 바로 DashBoard 입니다. Kubernetes Dashboard 설치 Dashboard 에서는 모든 일을 할 수 있습니다...
crystalcube.co.kr
혹은 github에 Account생성하는 방법이 있다.
kubernetes/dashboard
General-purpose web UI for Kubernetes clusters. Contribute to kubernetes/dashboard development by creating an account on GitHub.
github.com
'인프라 > Kubernetes' 카테고리의 다른 글
[Kubernetes] Jenkins Pod 설치 (0) 2020.12.02 [Kubernetes] private docker registry Pod 설치 (0) 2020.12.02 [Kubernetes] 설치 명령어 모음 (0) 2020.11.26 [Kubernetes] Horizontal Pod Autoscaler (0) 2020.11.25 [Kubernetes] 쿠버네티스 설치하기 (7) - Performing a Rolling Update (0) 2020.11.23