逃脱只会部署集群系列 —— k8s集群认证、授权、安全控制

我们可以利用调试，获取请求时读取的文件（/root/.kube/）以及请求地址（:6443即的地址）
[root@k8s-master ~]# kubectl get nodes -v=7I1120 12:31:58.06522443018 loader.go:375] Config loaded from file:/root/.kube/configI1120 12:31:58.06889543018 round_trippers.go:420] GET https://192.168.0.121:6443/api?timeout=32sI1120 12:31:58.06896443018 round_trippers.go:427] Request Headers:I1120 12:31:58.06897443018 round_trippers.go:431]Accept: application/json, */*
2、怎么默认授权
init启动完节点后，会默认输出类似下面的提示内容：
... ...Your Kubernetes master has initialized successfully!To start using your cluster, you need to run the following as a regular user:mkdir -p $HOME/.kubesudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/configsudo chown $(id -u):$(id -g) $HOME/.kube/config... ...
这些信息是在告知我们如何配置文件。按照上述命令配置后，节点上的就可以直接使用$HOME/.kube/的信息访问k8s 了。并且，通过这种配置方式，也拥有了整个集群的管理员(root)权限。当然，如果是二进制部署该步骤也是手动创建好admin.conf文件然后拷贝的。
现在讲解下的kube-是如何对来自的访问进行认证()和授权()，以及具备哪些授权。
3、的认证阶段
通过上述得知，利用文件向发起请求，前面提到过的支持通过tls、basic auth、token等方式对客户端发起的请求进行身份校验，从信息来看，显然在请求中使用了tls的方式，即客户端的证书。
我们获取--data的内容进行证书解码：
echo 'LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM4akNDQWRxZ0F3SUJBZ0lJYi8xa3dCNzZ4TVF3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TVRFeE1Ea3hNelV6TUROYUZ3MHlNakV4TURreE16VXpNRGhhTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXcvY0IyK0crRURaWDhveDYKTjMvUk8rQmxFNExUUlMrSzZJZzJrTVJ5WVcxbkZEMHVobDloaTd1OXhzVEt3eVlrdE1OQWFMeFUzbFpLcDAzMApDTnRPdVg4aXFRYVlZOGg0NEc5V3Q1YTNjM00rYitRNnFQZWhwYmhTanVCZnNFU3JOUG5ZTnlnTnZLMFM5T0RPCjdHLzBrVHdJajA3OW9sWS96eEkxQXJBaWVnb2dBbnJ0OUZ0NWZ2eUpuZjYzTW5FdXgyVG5oc2h1YitTc2VSYkgKNU5HWFMzVHNPRXlmQUZqUW95SkRsbkdIU0xLVzJTdVVkODhlVDhCL0x5b0NHN0pNZllmKzVEMjBHN3ROZktKRQpYT2xiZFp0Sldub3pjSTlCMEFjdHlLNWo0NndTOVcyU2V4em9hQVV3VEhDM0Y4Qy9IQlROQ3JrZXpRSU1FQ0hlCnNCYy9Vd0lEQVFBQm95Y3dKVEFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFLT2ZCdFlmWTltcFg3TU9FaEdtditCQk1IeTRleVd3eUFzUwpHSUgxSUNmNnd6aXZhVTJjRnRzektON2xkMy9nNXZwVHJSMUV5akJYK1B6SjFPVmdubFU4c0p0L0ZDdHBPWnVkCmhRbU9tejRqeTBRLzErMnZ4TGhqTXlXbmxLYmtBOFhMRDlqVkpIM21iRS9FOXNIVU03WHNCT2Ixbk14TU5MUTMKSXRQSWRQVWd3UXVOS2ErY0diaXdqMmVJMlgwSTE2aDRvLzJhQ2Z6R0VtMHV0bFRIdmczdkQydlZNTlpYV3lTQQowVUxaZVkyeThyQVViN3FpbHp4YmdIMStCUndiOStvSEovdW80UlNURjlXaVFmcFFiN3RCQU40TGlxOFJRTjY5CmU3MEoxVit4cThkSk9PZFBGVUFUWWsxeUdkbi81bDY1dlAzYnNneitaMjNLTGw3ZmxNcz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=' | base64 -d > 123.crt
在认证阶段，会首先使用---ca-file配置的CA证书去验证提供的证书的有效性,我们可以手动认证下证书是否有效，到这里为止，认证通过，代表这个请求可以进来了，大门开放
[root@k8s-master ~]# openssl verify -CAfile /etc/kubernetes/pki/ca.crt 123.crtkubectl.crt: OK
除了认证身份，还会取出必要的信息供授权阶段使用，文本形式查看证书内容，我们取出请求对应的用户组或者用户发给授权机制：
[root@k8s-master ~]# openssl x509 -in 123.crt -textCertificate:Data:Version: 3 (0x2)Serial Number: 8069716883634046148 (0x6ffd64c01efac4c4)Signature Algorithm: sha256WithRSAEncryptionIssuer: CN=kubernetesValidityNot Before: Nov9 13:53:03 2021 GMTNot After : Nov9 13:53:08 2022 GMT# 重点是这里，我们可以获取到该请求对应的用户组：system:masters，用户：kubernetes-adminSubject: O=system:masters, CN=kubernetes-adminSubject Public Key Info:Public Key Algorithm: rsaEncryptionPublic-Key: (2048 bit)Modulus:00:c3:f7:01:db:e1:be:10:36:57:f2:8c:7a:37:7f:d1:3b:e0:65:13:82:d3:45:2f:8a:e8:88:36:90:c4:
4、的授权阶段
在init初始引导集群启动过程中，创建了许多默认的RBAC规则，在k8s有关RBAC的官方文档中(使用 RBAC 鉴权 | )，我们看到下面一些列表:
其中第一个-admin这个 role 绑定了: group，这和环节传递过来的身份信息不谋而合。此处涉及到RBAC的用户组和角色绑定，不做具体阐述了Using RBAC|。
我们可以看出：请求授权传递的用户组是:，该用户组绑定了-admin的 role角色，并且具有超级用户在平台上的任何资源上执行所有操作。
[root@k8s-master ~]# kubectl describe clusterrolebinding cluster-adminName:cluster-adminLabels:kubernetes.io/bootstrapping=rbac-defaultsAnnotations:rbac.authorization.kubernetes.io/autoupdate: trueRole:Kind:ClusterRoleName:cluster-adminSubjects:KindNameNamespace-----------------Groupsystem:masters[root@k8s-master ~]# kubectl describe clusterrole cluster-adminName:cluster-adminLabels:kubernetes.io/bootstrapping=rbac-defaultsAnnotations:rbac.authorization.kubernetes.io/autoupdate: truePolicyRule:ResourcesNon-Resource URLsResource NamesVerbs---------------------------------------------*.*[][][*][*][][*]
? ? ?整个请求过程如图：
三、的认证授权 1、的认证阶段
老规矩，通过v=7查看请求用到的配置文件，从而获取对应的证书等内容，进程通过查看启动参数即可。
继续，查看启动参数的配置文件，获取证书内容，base64解密证书把请求的用户或者用户组先拎出来，然后看看集群有没有利用binding赋予相应的权限。
【逃脱只会部署集群系列 —— k8s集群认证、授权、安全控制】# 1、读取启动参数配置文件[root@k8s-master ~]# # vim /etc/kubernetes/kubelet.conf # 2、解密配置文件中的证书内容[root@k8s-master ~]# echo 'LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM5akNDQWQ2Z0F3SUJBZ0lJWlNLMS8vdCtvM1V3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TVRFeE1Ea3hNelV6TUROYUZ3MHlNakV4TURreE16VXpNRGhhTURneApGVEFUQmdOVkJBb1RESE41YzNSbGJUcHViMlJsY3pFZk1CMEdBMVVFQXhNV2MzbHpkR1Z0T201dlpHVTZhemh6CkxXMWhjM1JsY2pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTUJsblJzMWkxS0UKTWw3ejgvMWN5VTlwczlIZVVsMllOenh0bEl3ZzFDOUtGUXVuMHV0aEMrNk0xNG9lcFNRVkF4cUZjK2dnQXlsMgpXTEFoZ0U5YVdyU0FWZWNjaHdKd0dianFIYmZBd2UvR3JZMGZrUEtMQlZ0UVJ1Znl2TEovQUluNUdqVkZ0Z2UxClVWa1JtZXBkWGxlUTc4bG5rZExvbGRQMThFYU5OcEdqcjY5Q1VSbjI1Zm8xTmpXSjhOZ3JDOHJxY2YxSHZUaUgKWVFWenRwaHh1MEo5b1Z1K2Ftc3h3RU5pRTBOOHdsS29lNG1NRTUvaTJhRHZ4VU9GNjlRMmE1c0QwQzBXVVJMYwppWWd6bWxsZjRuMldHMkxCRnRpRlpYWmdYTkNyMm91MkcvMjlZN0pIR0V4VjdOOFN0S0tvSEFjcm9FWGFhVTQwCmFHS04vRmVsSUxjQ0F3RUFBYU1uTUNVd0RnWURWUjBQQVFIL0JBUURBZ1dnTUJNR0ExVWRKUVFNTUFvR0NDc0cKQVFVRkJ3TUNNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUJBUUJmZnR4RlB6ODl6TURJVGNOVUpTTURxL2c4NzhsVQpKMmphdjZMRzVZNTlWeWQwMzMzd2ZidG5hY0VtZnY1L0paVDFPZXREdHJVMmJnOTA2VllFR3RDR3laM2d6dDY5Ckc2VFZBY3pBS3VZMGlIcVl6UTRIdFVKSEZyZXY1aDduaXRaNmhiTEVJU3l5VUpvMFlIc1FxRmUzYkduOGg2UVYKMzRoalQ5NXEzRUp3SWV2VGM3UWh2dlBoaEpEMjU2elFTYkR5QzN6VWdMamxDcmNONERkWmw1VjRVU1FUWVpKUApuT01WbGhoTkRBOVMrWEJZN09MMDkxQzJaR0dDRHVrSnpMOVgwaUQ1VHlNazBZbnZhb01yQy9Cblg0U2FKSDFUCm5XV0Vud0lwQTlXNzVvZlJ6ZTRCSndkNmk1eExnQ0MrVkpOK0pIaVRkK2VnQ3ZiN3l2cTI5eTR1Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K' | base64 -d > kubelet.crt# 3、验证证书有效性，OK说明认证阶段通过[root@k8s-master ~]# openssl verify -CAfile /etc/kubernetes/pki/ca.crt kubelet.crtkubelet.crt: OK# 4、或者kubelet请求的用户组和用户[root@k8s-master ~]# openssl x509 -in kubelet.crt -textCertificate:Data:Version: 3 (0x2)Serial Number: 7287587258079552373 (0x6522b5fffb7ea375)Signature Algorithm: sha256WithRSAEncryptionIssuer: CN=kubernetesValidityNot Before: Nov9 13:53:03 2021 GMTNot After : Nov9 13:53:08 2022 GMTSubject: O=system:nodes, CN=system:node:k8s-master
2、的授权阶段
K8s会把O作为Group来进行请求，因此如果有权限绑定给这个组，肯定在的定义中可以找得到。因此尝试去找一下绑定了:nodes组的
[root@k8s-master ~]# kubectl get clusterrolebinding|awk 'NR>1{print $1}'|xargs kubectl get clusterrolebinding -oyaml|grep -n10 system:nodes76-resourceVersion: "172"77-selfLink: /apis/rbac.authorization.k8s.io/v1/clusterrolebindings/kubeadm%3Anode-autoapprove-certificate-rotation78-uid: 5d227fdf-57e9-4003-b410-55acf820b76579-roleRef:80-apiGroup: rbac.authorization.k8s.io81-kind: ClusterRole82-name: system:certificates.k8s.io:certificatesigningrequests:selfnodeclient83-subjects:84-- apiGroup: rbac.authorization.k8s.io85-kind: Group86:name: system:nodes87-- apiVersion: rbac.authorization.k8s.io/v188-kind: ClusterRoleBinding89-metadata:90-creationTimestamp: "2021-11-09T13:53:47Z"91-name: kubeadm:node-proxier92-resourceVersion: "200"93-selfLink: /apis/rbac.authorization.k8s.io/v1/clusterrolebindings/kubeadm%3Anode-proxier94-uid: 97c4ec4b-f745-4060-977f-77ed9432110095-roleRef:96-apiGroup: rbac.authorization.k8s.io
[root@k8s-master ~]# kubectl describe clusterrole system:certificates.k8s.io:certificatesigningrequests:selfnodeclientName:system:certificates.k8s.io:certificatesigningrequests:selfnodeclientLabels:kubernetes.io/bootstrapping=rbac-defaultsAnnotations:rbac.authorization.kubernetes.io/autoupdate: truePolicyRule:ResourcesNon-Resource URLsResource NamesVerbs---------------------------------------------certificatesigningrequests.certificates.k8s.io/selfnodeclient[][][create]
结局有点意外，除了:.k8s.io::外，没有找到相关的，显然和我们的理解不一样。尝试去找Using RBAC| 发现了这么一段 :
:kube-
:kube- user
to theby the .
:-
:kube- user
to theby the kube- .
:kube--
:kube-- user
to theby the. Thebyarein theroles.
:node
None
toby the ,readto all , and writeto all pod. Youuse the Nodeandof the :node role, and allowAPItobased on the Podsto run on them. The :node role onlyforwithfromprior to v1.8.
:node-
:kube-proxy user
to theby the kube-.
大致意思是说：之前会定义:node这个角色，目的是为了可以访问到必要的资源，包括所有的读权限及更新pod状态的写权限。如果1.8版本后，是建议使用 Nodeand来代替这个角色的。

文章插图
我们目前使用1.16，查看一下授权策略：
$ ps axu|grep apiserverkube-apiserver --authorization-mode=Node,RBAC--enable-admission-plugins=NodeRestriction?
查看一下官网对Node 的介绍（Using Node| ）：
Nodeis a -mode thatAPImade by .
In, the nodemay add ortohave theset ofto.
In order to beby the Node ,must use athatthem as being in the :nodes group, with aof :node:
3、授权记住特殊性
如果你看的有点懵，那么记住结论就好了：（1.8版本后）利用证书模式仅作认证，利用NODE模式进行授权，Node 授权器允许执行 API 操作。而且目前看来，只有利用node授权，其他还是都用RBAC授权。
四、及K8S Api调用
前面说，认证可以通过证书，也可以通过使用（服务账户）的方式来做认证。大多数时候，我们在基于k8s做二次开发时都是选择通过 + RBAC 的方式。我们之前访问的时候，是如何做的？
目的：创建一个，创建或者使用一个现有的角色与sa绑定，当sa请求时不通过携带证书，而是通过token的方式成功获取集群内容
1、创建并绑定角色
新建一个名为admin的，并且把名为-admin的这个集群角色的权限授予新建的

[root@k8s-master ~]# catca-api.yamlapiVersion: v1kind: ServiceAccountmetadata:name: adminnamespace: default---kind: ClusterRoleBindingapiVersion: rbac.authorization.k8s.io/v1beta1metadata:name: adminannotations:rbac.authorization.kubernetes.io/autoupdate: "true"roleRef:kind: ClusterRolename: cluster-adminapiGroup: rbac.authorization.k8s.iosubjects:- kind: ServiceAccountname: adminnamespace: default[root@k8s-master ~]# kubectl create -f ca-api.yaml serviceaccount/admin createdclusterrolebinding.rbac.authorization.k8s.io/admin created

2、查看创建sa的token
默认创建sa都会同时创建一个对应的，我们可以直接查看内容获取token

# 查看创建的sa[root@k8s-master ~]# kubectl get sa admin -o yamlapiVersion: v1kind: ServiceAccountmetadata:creationTimestamp: "2021-11-20T09:24:15Z"name: adminnamespace: defaultresourceVersion: "60267"selfLink: /api/v1/namespaces/default/serviceaccounts/adminuid: 7a2db3d5-84eb-4339-a8cd-4b95115f4688secrets:- name: admin-token-bs4sg

# 查看创建ca对应的secret，获取token[root@k8s-master ~]# kubectl get secretNAMETYPEDATAAGEadmin-token-bs4sgkubernetes.io/service-account-token3116sdefault-token-pshvdkubernetes.io/service-account-token310d[root@k8s-master ~]# kubectl describe secret admin-token-bs4sgName:admin-token-bs4sgNamespace:defaultLabels:Annotations:kubernetes.io/service-account.name: adminkubernetes.io/service-account.uid: 7a2db3d5-84eb-4339-a8cd-4b95115f4688Type:kubernetes.io/service-account-tokenData=http://www.kingceram.com/post/===ca.crt:1025 bytesnamespace:7 bytestoken:eyJhbGciOiJSUzI1NiIsImtpZCI6IlJYY0FNa3lvdjVtNERqbzJzVzlJM2RTa2dZQ3pxSnI5NDFxUFZHRk9zbGsifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImFkbWluLXRva2VuLWJzNHNnIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiN2EyZGIzZDUtODRlYi00MzM5LWE4Y2QtNGI5NTExNWY0Njg4Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6YWRtaW4ifQ.xTyQ9ZJFgQkbQx0TLpuueku2DkQc49S3DP8E_3fDqHES615cBaGof-HU77daywO7YW4vRxTk0MGIvPDarnh04gYH37ZSqUGWAF4uJNjrXsKVjly7sq7o-iS_nJef-1zWxr7FBKYjYMbRobMYHPwMdguz4GBK2S9C9IYCzpifo1ERm36WbA7gA6b6ylET8x_5zbORBhNm7FUDBRbpXLGXCyHlA2o6batMc2cAwHog0hIWYZ0HPvZkSjl7E2uIKiG8QefApdgDgck2jWztSyvBB6QGA8buGcIevws4I78GfGA9hgq0tzPuDTHNdFSAkCY4J3lirdQwKT84yWsgqQg4Rw

3、调用
认证即进入大门，授权即确认可以对资源进行哪些操作，所以API调用需要两点：1、调用资源时需要携带证书或者token等鉴权信息；2、注明请求的API接口地址
token我们已经拿到了，API接口以获取的pod信息为例，利用v=7查看

[root@k8s-master ~]# kubectl get po -v=7I1120 17:34:32.08345189976 loader.go:375] Config loaded from file:/root/.kube/configI1120 17:34:32.08835589976 round_trippers.go:420] GET https://192.168.0.121:6443/api/v1/namespaces/default/pods?limit=500I1120 17:34:32.08837289976 round_trippers.go:427] Request Headers:I1120 17:34:32.08837789976 round_trippers.go:431]Accept: application/json;as=Table;v=v1beta1;g=meta.k8s.io, application/jsonI1120 17:34:32.08838689976 round_trippers.go:431]User-Agent: kubectl/v1.16.1 (linux/amd64) kubernetes/d647ddbI1120 17:34:32.09975489976 round_trippers.go:446] Response Status: 200 OK in 11 millisecondsNAMEREADYSTATUSRESTARTSAGEbusybox-5d7b4b65d6-mk6c61/1Running142hhttps://192.168.0.121:6443/api/v1/namespaces/default/pods?limit=500就是需要请求的API接口

curl: (35) Peer reports incompatible or unsupported protocol version.curl 不兼容或不支持的协议版本centos系统解决方法：yum update -y nss curl libcurl

利用：curl -k -H ": +空格+token+空格+API请求地址"调用成功

[root@k8s-master ~]# curl -k-H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IlJYY0FNa3lvdjVtNERqbzJzVzlJM2RTa2dZQ3pxSnI5NDFxUFZHRk9zbGsifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImFkbWluLXRva2VuLWJzNHNnIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiN2EyZGIzZDUtODRlYi00MzM5LWE4Y2QtNGI5NTExNWY0Njg4Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6YWRtaW4ifQ.xTyQ9ZJFgQkbQx0TLpuueku2DkQc49S3DP8E_3fDqHES615cBaGof-HU77daywO7YW4vRxTk0MGIvPDarnh04gYH37ZSqUGWAF4uJNjrXsKVjly7sq7o-iS_nJef-1zWxr7FBKYjYMbRobMYHPwMdguz4GBK2S9C9IYCzpifo1ERm36WbA7gA6b6ylET8x_5zbORBhNm7FUDBRbpXLGXCyHlA2o6batMc2cAwHog0hIWYZ0HPvZkSjl7E2uIKiG8QefApdgDgck2jWztSyvBB6QGA8buGcIevws4I78GfGA9hgq0tzPuDTHNdFSAkCY4J3lirdQwKT84yWsgqQg4Rw" https://192.168.0.121:6443/api/v1/namespaces/default/pods?limit=500{"kind": "PodList","apiVersion": "v1","metadata": {"selfLink": "/api/v1/namespaces/default/pods","resourceVersion": "61327"},"items": [{"metadata": {"name": "busybox-5d7b4b65d6-mk6c6","generateName": "busybox-5d7b4b65d6-","namespace": "default","selfLink": "/api/v1/namespaces/default/pods/busybox-5d7b4b65d6-mk6c6","uid": "7a06c495-bd70-4816-9845-3bd2db69ec7f","resourceVersion": "43135","creationTimestamp": "2021-11-18T14:38:38Z","labels": {"app": "busybox","pod-template-hash": "5d7b4b65d6"},"ownerReferences": [{"apiVersion": "apps/v1","kind": "ReplicaSet","name": "busybox-5d7b4b65d6","uid": "8f36b9d4-9fb2-4642-97da-1dbad70809c3","controller": true,"blockOwnerDeletion": true}]}, 。。。。。