诺普培训《K8S安全技术文章分享》
K8S安全系统-PSPPod Security PolicyPod Security Policy 是 kubernetes中一种集群级别的资源，它定义了用户能否在Pod中使用各种安全相关的特性，PSP可以做哪些事情？
· 允许和拒绝Pod使用宿主节点的PID,IPC,网络命名空间· 允许和拒绝Pod绑定到宿主节点端口· 容器运行时允许和拒绝Pod使用的用户ID· 是否允许特权模式的PODhttps://bbs.hh010.com/data/attachment/forum/202203/07/104227mkkkmk8svsrgmgio.png
这次，诺普培训的邓老师给大家带来了一篇干货满满的K8S技术文章，希望对广大学员们有所帮助。实战如下：
#默认psp没有开启：psp 无效；ubuntu@cks-1:~$ kubectl get pspWarning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMESgatekeeper-admin false RunAsAny MustRunAsNonRoot MustRunAs MustRunAs false configMap,projected,secret,downwardAPIubuntu@cks-1:~$
ubuntu@cks-1:~/LFS260/SOLUTIONS/s_04$ kubectl delete -f gatekeeper.yamlnamespace "gatekeeper-system" deletedresourcequota "gatekeeper-critical-pods" deletedcustomresourcedefinition.apiextensions.k8s.io "configs.config.gatekeeper.sh" deletedcustomresourcedefinition.apiextensions.k8s.io "constraintpodstatuses.status.gatekeeper.sh" deletedcustomresourcedefinition.apiextensions.k8s.io "constrainttemplatepodstatuses.status.gatekeeper.sh" deletedcustomresourcedefinition.apiextensions.k8s.io "constrainttemplates.templates.gatekeeper.sh" deletedserviceaccount "gatekeeper-admin" deletedWarning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+podsecuritypolicy.policy "gatekeeper-admin" deletedrole.rbac.authorization.k8s.io "gatekeeper-manager-role" deletedclusterrole.rbac.authorization.k8s.io "gatekeeper-manager-role" deletedrolebinding.rbac.authorization.k8s.io "gatekeeper-manager-rolebinding" deletedclusterrolebinding.rbac.authorization.k8s.io "gatekeeper-manager-rolebinding" deletedsecret "gatekeeper-webhook-server-cert" deletedservice "gatekeeper-webhook-service" deleteddeployment.apps "gatekeeper-audit" deleteddeployment.apps "gatekeeper-controller-manager" deletedWarning: policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudgetpoddisruptionbudget.policy "gatekeeper-controller-manager" deletedvalidatingwebhookconfiguration.admissionregistration.k8s.io "gatekeeper-validating-webhook-configuration" deleted
ubuntu@cks-1:~$ kubectl get pspWarning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+No resources foundubuntu@cks-1:~$
#在没有开启PSP功能的情况下，创建拒绝提权的POD；ubuntu@cks-1:~$ touch unprivileged-psp.yamlubuntu@cks-1:~$ vim unprivileged-psp.yamlubuntu@cks-1:~$ cat unprivileged-psp.yamlapiVersion: policy/v1beta1kind: PodSecurityPolicymetadata:name: unprivileged-pspspec:privileged: false# Don't allow privileged pods!# The rest fills in some required fields.seLinux: rule: RunAsAnysupplementalGroups: rule: RunAsAnyrunAsUser: rule: RunAsAnyfsGroup: rule: RunAsAnyvolumes:- '*'ubuntu@cks-1:~$
ubuntu@cks-1:~$ kubectl create -f unprivileged-psp.yamlWarning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+podsecuritypolicy.policy/unprivileged-psp createdubuntu@cks-1:~$ kubectl get pspWarning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMESunprivileged-psp false RunAsAny RunAsAny RunAsAny RunAsAny false *ubuntu@cks-1:~$
#在没有开启PSP的前提下还是可以创建提权的pod；ubuntu@cks-1:~$ kubectl create -f privileged-pod.yamlpod/privileged-pod created
ubuntu@cks-1:~$
#此时POD可以正常使用和在POD内部提权到root权限；
#在K8S中开启psp；#删除掉之前创建的POD；ubuntu@cks-1:~$ kubectl delete pods privileged-pod --forcewarning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.pod "privileged-pod" force deletedubuntu@cks-1:~$
#修改kube-apiserver.yaml；root@cks-1:/etc/kubernetes/manifests# cat kube-apiserver.yaml | head -n 30apiVersion: v1kind: Podmetadata:annotations: kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: 172.19.0.11:6443creationTimestamp: nulllabels: component: kube-apiserver tier: control-planename: kube-apiservernamespace: kube-systemspec:containers:- command: - kube-apiserver - --advertise-address=172.19.0.11 - --token-auth-file=/etc/kubernetes/pki/users.txt - --profiling=false - --allow-privileged=true - --authorization-mode=Node,RBAC - --client-ca-file=/etc/kubernetes/pki/ca.crt - --enable-admission-plugins=NodeRestriction,PodSecurityPolicy - --enable-bootstrap-token-auth=true - --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key - --etcd-servers=https://127.0.0.1:2379 - --insecure-port=0 - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.keyroot@cks-1:/etc/kubernetes/manifests#
#重启kubelet服务，使配置生效；root@cks-1:~# systemctl restart kubelet
ubuntu@cks-1:~$ kubectl get pods -n kube-systemNAME READY STATUS RESTARTS AGEcalico-kube-controllers-6b9fbfff44-ltm9g 1/1 Running 3 4dcalico-node-r72x7 1/1 Running 3 4dcalico-node-vt447 1/1 Running 3 3d23hcoredns-558bd4d5db-9w7h6 1/1 Running 3 4dcoredns-558bd4d5db-wv6v2 1/1 Running 3 4detcd-cks-1.example.com 1/1 Running 3 4dkube-controller-manager-cks-1.example.com 1/1 Running 8 4dkube-proxy-7t27k 1/1 Running 3 3d23hkube-proxy-vq295 1/1 Running 3 4dkube-scheduler-cks-1.example.com 1/1 Running 8 4dubuntu@cks-1:~$
#再创建一个psp: 允许提权；ubuntu@cks-1:~$ vim privileged-psp.yamlubuntu@cks-1:~$ cat privileged-psp.yamlapiVersion: policy/v1beta1kind: PodSecurityPolicymetadata:name: privileged-pspspec:privileged: true# Don't allow privileged pods!# The rest fills in some required fields.seLinux: rule: RunAsAnysupplementalGroups: rule: RunAsAnyrunAsUser: rule: RunAsAnyfsGroup: rule: RunAsAnyvolumes:- '*'ubuntu@cks-1:~$
ubuntu@cks-1:~$ kubectl create -f privileged-psp.yamlWarning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+podsecuritypolicy.policy/privileged-psp createdubuntu@cks-1:~$ kubectl get pspWarning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMESprivileged-psp true RunAsAny RunAsAny RunAsAny RunAsAny false *unprivileged-psp false RunAsAny RunAsAny RunAsAny RunAsAny false *ubuntu@cks-1:~$
ubuntu@cks-1:~$ kubectl get pods -n kube-systemNAME READY STATUS RESTARTS AGEcalico-kube-controllers-6b9fbfff44-ltm9g 1/1 Running 3 4dcalico-node-r72x7 1/1 Running 3 4dcalico-node-vt447 1/1 Running 3 4dcoredns-558bd4d5db-9w7h6 1/1 Running 3 4dcoredns-558bd4d5db-wv6v2 1/1 Running 3 4detcd-cks-1.example.com 1/1 Running 3 4dkube-controller-manager-cks-1.example.com 1/1 Running 8 4dkube-proxy-7t27k 1/1 Running 3 4dkube-proxy-vq295 1/1 Running 3 4dkube-scheduler-cks-1.example.com 1/1 Running 8 4dubuntu@cks-1:~$
#创建普通pod和提权pod；ubuntu@cks-1:~$ kubectl create -f normal-pod.yamlpod/normal-pod createdubuntu@cks-1:~$ kubectl create -f privileged-pod.yamlpod/privileged-pod createdubuntu@cks-1:~$
#用户要使用psp策略，必须有use权限；ubuntu@cks-1:~$ kubectl auth can-i use psp/privileged-pspWarning: resource 'podsecuritypolicies' is not namespace scoped in group 'policy'yesubuntu@cks-1:~$
ubuntu@cks-1:~$ kubectl auth can-i use psp/unprivileged-pspWarning: resource 'podsecuritypolicies' is not namespace scoped in group 'policy'yesubuntu@cks-1:~$
#使用Mary在default 中创建 role: admin-pod, verb： get,list,watch, delete,run ,create；ubuntu@cks-1:~$ kubectl create role --resource=pods --verb=get,list,watch,create,delete --dry-run=client admin-pod -o yaml > admin-pod.yamlubuntu@cks-1:~$ kubectl create -f admin-pod.yamlrole.rbac.authorization.k8s.io/admin-pod createdubuntu@cks-1:~$ kubectl get rolesNAME CREATED ATadmin-pod 2022-01-16T13:40:00Zubuntu@cks-1:~$
#将role: admin-pod 绑定给mary；ubuntu@cks-1:~$ kubectl create rolebinding --role=admin-pod --user=mary mary-rb-admin-pod --dry-run=client -o yaml > mary-rb-admin-pod.yamlubuntu@cks-1:~$ kubectl create -f mary-rb-admin-pod.yamlrolebinding.rbac.authorization.k8s.io/mary-rb-admin-pod created
ubuntu@cks-1:~$ lsadmin-pod.yamlkube-bench_0.6.5_linux_amd64.deb mary_csr.yaml privileged-psp.yamlAssessor-CLI LFS260 mary-rb-admin-pod.yamltrivy_0.22.0_Linux-64bit.debCIS-Cat.zip LFS260_V2021-08-10_SOLUTIONS.tar.xznormal-pod.yaml unprivileged-psp.yamlinstall_k8s.shmary privileged-pod.yamlubuntu@cks-1:~$
#测试；ubuntu@cks-2:~$ cd .kube/ubuntu@cks-2:~/.kube$ lscacheconfig.backupubuntu@cks-2:~/.kube$ mv config.backup config.backupmv: 'config.backup' and 'config.backup' are the same fileubuntu@cks-2:~/.kube$ mv config.backup configubuntu@cks-2:~/.kube$
ubuntu@cks-2:~$ kubectl auth can-i use psp/privileged-pspWarning: resource 'podsecuritypolicies' is not namespace scoped in group 'policy'noubuntu@cks-2:~$ kubectl auth can-i use psp/unprivileged-pspWarning: resource 'podsecuritypolicies' is not namespace scoped in group 'policy'noubuntu@cks-2:~$
ubuntu@cks-2:~$ kubectl run --image=nginx mypod -n defaultError from server (Forbidden): pods "mypod" is forbidden: PodSecurityPolicy: unable to admit pod: []ubuntu@cks-2:~$
#创建 privileged-psp-rule；ubuntu@cks-1:~$ kubectl get pspWarning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMESprivileged-psp true RunAsAny RunAsAny RunAsAny RunAsAny false *unprivileged-psp false RunAsAny RunAsAny RunAsAny RunAsAny false *ubuntu@cks-1:~$ cat unprivileged-psp-role.yamlapiVersion: rbac.authorization.k8s.io/v1kind: Rolemetadata:name: unprivileged-psp-rolerules:- apiGroups: ['policy']resources: ['podsecuritypolicies']verbs: ['use']resourceNames:- unprivileged-pspubuntu@cks-1:~$
ubuntu@cks-1:~$ kubectl create -f unprivileged-psp-role.yaml
ubuntu@cks-1:~$ kubectl get role -n defaultNAME CREATED ATadmin-pod 2022-01-16T13:40:00Zunprivileged-psp-role 2022-01-16T13:53:32Zubuntu@cks-1:~$
#将mary与unprivileged-psp-role 绑定；ubuntu@cks-1:~$ vim mary-rb-unprivileged-psp-role.yamlubuntu@cks-1:~$ cat mary-rb-unprivileged-psp-role.yamlapiVersion: rbac.authorization.k8s.io/v1kind: RoleBindingmetadata:creationTimestamp: nullname: mary-rb-unprivileged-psproleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: unprivileged-psp-rolesubjects:- apiGroup: rbac.authorization.k8s.iokind: Username: maryubuntu@cks-1:~$
ubuntu@cks-1:~$ kubectl create -f mary-rb-unprivileged-psp-role.yamlrolebinding.rbac.authorization.k8s.io/mary-rb-unprivileged-psp createdubuntu@cks-1:~$
ubuntu@cks-2:~$ubuntu@cks-2:~$ kubectl auth can-i use psp/unprivileged-pspWarning: resource 'podsecuritypolicies' is not namespace scoped in group 'policy'yesubuntu@cks-2:~$ kubectl auth can-i use psp/privileged-pspWarning: resource 'podsecuritypolicies' is not namespace scoped in group 'policy'noubuntu@cks-2:~$
ubuntu@cks-1:~$ scp normal-pod.yaml privileged-pod.yaml ubuntu@172.19.0.3:~/ubuntu@172.19.0.3's password:normal-pod.yaml 100%247 597.6KB/s 00:00privileged-pod.yaml 100%302 614.1KB/s 00:00ubuntu@cks-1:~$
ubuntu@cks-1:~$ kubectl delete pod normal-pod --forcewarning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.pod "normal-pod" force deletedubuntu@cks-1:~$ kubectl delete pod privileged-pod --forcewarning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.pod "privileged-pod" force deletedubuntu@cks-1:~$
ubuntu@cks-2:~$ kubectl create -f normal-pod.yamlpod/normal-pod created
#测试结果，无法创建提权root的pod，提高pod的安全；ubuntu@cks-2:~$ kubectl create -f privileged-pod.yamlError from server (Forbidden): error when creating "privileged-pod.yaml": pods "privileged-pod" is forbidden: PodSecurityPolicy: unable to admit pod: .securityContext.privileged: Invalid value: true: Privileged containers are not allowed]ubuntu@cks-2:~$
关于诺普https://bbs.hh010.com/data/attachment/forum/202203/07/104238ddzoojr97dojtqbj.png
诺普（深圳）咨询服务有限公司（简称ROPU）是专注于IT运维的最佳实践与传播，提供领先于业界的“IT运维技能及厂商认证培训”，“企业IT定制化内训，以及”高校IT基础性人才的培养，与全球著名授权机构、IT厂商建立长期合作伙伴关系，旨在为企业及运维人员提升IT自动化技能，为员工的技能转型及企业数字化转型提供最具价值的培训服务。
联系我们电话：07755-82558626林老师：18926480845官网：http://www.ropustudy.com/地址：深圳市福田区车公庙苍松大厦南座1115https://bbs.hh010.com/data/attachment/forum/202203/07/104238q2xniiydttrtfipz.jpg
{:6_267:}{:6_267:}{:6_267:}{:6_267:}{:6_267:}
页:
[1]