https://www.jianshu.com/p/cdffaff82a34

 

经过几轮方案和填坑,目前方案应该最简单可靠。

一,经历

说起来,都是泪,从三年前和这个问题作斗争,证书过期和自动续期这个大问题,始终是一个心头的伤。
现在要想到一刀切的方案,还是自己更改Kubeadm源码,全部改成100年,最洒脱。
但,如果线上已运行了这些东东,且是10年1年证书过期的都有,那啷个弄嘛?

二,刺探

先用如下命令,看看k8s的哪些证书何时到期

CERT_DIR=${CERT_DIR:-/etc/kubernetes/pki}
for i in $(find $CERT_DIR -name '*.crt' -o -name '*.pem'); do
 echo $i
    openssl x509 -enddate -in $i -noout
done
for f in $(ls /etc/kubernetes/{admin,controller-manager,scheduler,kubelet}.conf); do
 echo $f
    kubectl --kubeconfig $f config view --raw -o jsonpath='{range .users[*]}{.user.client-certificate-data}{end}' | base64 -d | openssl x509 -enddate -noout
done

输出pki下的证书情况:

/etc/kubernetes/pki/ca.crt
notAfter=Nov 25 01:41:33 2029 GMT
/etc/kubernetes/pki/apiserver.crt
notAfter=Nov 27 01:41:34 2020 GMT
/etc/kubernetes/pki/apiserver-kubelet-client.crt
notAfter=Nov 27 01:41:34 2020 GMT
/etc/kubernetes/pki/etcd/ca.crt
notAfter=Nov 25 01:41:34 2029 GMT
/etc/kubernetes/pki/etcd/server.crt
notAfter=Nov 27 01:41:34 2020 GMT
/etc/kubernetes/pki/etcd/peer.crt
notAfter=Nov 27 01:41:35 2020 GMT
/etc/kubernetes/pki/etcd/healthcheck-client.crt
notAfter=Nov 27 01:41:35 2020 GMT
/etc/kubernetes/pki/apiserver-etcd-client.crt
notAfter=Nov 27 01:41:35 2020 GMT
/etc/kubernetes/pki/front-proxy-ca.crt
notAfter=Nov 25 01:41:36 2029 GMT
/etc/kubernetes/pki/front-proxy-client.crt
notAfter=Nov 27 01:41:36 2020 GMT

输出/etc/kubernetes下的证书情况

/etc/kubernetes/admin.conf
notAfter=Jul 24 02:20:39 2021 GMT
/etc/kubernetes/controller-manager.conf
notAfter=Jul 24 06:16:54 2021 GMT
/etc/kubernetes/kubelet.conf
notAfter=Jul 24 06:17:13 2021 GMT
/etc/kubernetes/scheduler.conf
notAfter=Jul 24 06:16:10 2021 GMT

三,如果只是/etc/kubernetes下面的证书过期,则使用如下方案解决。

1,备份

cp -R /etc/kubernetes /etc/kubernetes$(date "+%Y%m%d")

2,将主要证书文件mv一下,如果不mv,则不能创建新的证书文件

mv /etc/kubernetes/admin.conf /etc/kubernetes/admin.bak 
mv /etc/kubernetes/controller-manager.conf /etc/kubernetes/controller-manager.conf.bak                   
mv /etc/kubernetes/scheduler.conf /etc/kubernetes/scheduler.conf.bak 
mv /etc/kubernetes/kubelet.conf /etc/kubernetes/kubelet.conf.bak 

3,重新生成所有四个证书(这是踩过大坑的,最开始只升级admin,干到凌晨,才查出来还要升级controller-manager,scheduler,后来又忘了kubelet,导致k8s集群两小时不能动弹)。

kubeadm init phase kubeconfig admin     
kubeadm init phase kubeconfig scheduler                                                                                             
kubeadm init phase kubeconfig controller-manager 
kubeadm init phase kubeconfig kubelet     

又或者一条命令搞定
kubeadm init phase kubeconfig all
这里有个注意的细节,在使用kubeadm命令之前,它会到外网查找此K8s集群的版本信息,如果我们的机器是纯企业内网,不能访问外面,这里就会卡住。
BUT,还是可以离线进行的。
先从本集群生成一个config view类型文件。
kubeadm config view > kubeadm.conf
然后,在之后生成证书时,加上这个文件作为–config参数即可。如
kubeadm alpha phase kubeconfig scheduler --config kubeadm.conf
(上面是kueadm 1.10版本的命令,新版本已从alpha转正式命令,-h可找出来)

帮助

如果生疏了,可能看看help命令

kubeadm init phase kubeconfig -h
This command is not meant to be run on its own. See list of available subcommands.

Usage:
  kubeadm init phase kubeconfig [flags]
  kubeadm init phase kubeconfig [command]

Available Commands:
  admin              Generates a kubeconfig file for the admin to use and for kubeadm itself
  all                Generates all kubeconfig files
  controller-manager Generates a kubeconfig file for the controller manager to use
  kubelet            Generates a kubeconfig file for the kubelet to use *only* for cluster bootstrapping purposes
  scheduler          Generates a kubeconfig file for the scheduler to use

Flags:
  -h, --help   help for kubeconfig

Global Flags:
      --log-file string   If non-empty, use this log file
      --rootfs string     [EXPERIMENTAL] The path to the 'real' host root filesystem.
      --skip-headers      If true, avoid header prefixes in the log messages
  -v, --v Level           number for the log level verbosity

Use "kubeadm init phase kubeconfig [command] --help" for more information about a command.

四,如果是/etc/kubernetes/pki下面的证书过期,则使用如下方案解决。

1,仍然先备份哟,备份使得万年船~~
cp -R /etc/kubernetes /etc/kubernetes$(date "+%Y%m%d")
2,先将要过期的证书作更名

mv front-proxy-client.crt front-proxy-client.crt.bak
mv front-proxy-client.key front-proxy-client.key.bak

3,生成k8s的config view,然后使用kubeadm生成新的证书对

kubeadm alpha phase kubeconfig scheduler  --config kubeadm.conf
kubeadm alpha phase certs front-proxy-client --config kubeadm.conf 
kubeadm alpha phase certs front-proxy-client --config kubeadm.conf 

4,依次升级完其它几个要过期的证书,包括与etcd连接的证书对。
5,注意,有三个根证书对,是20年过期的,我没有更新(关键我不清楚更新之后,会发生什么事)。

/etc/kubernetes/pki/ca.crt
notAfter=Oct 27 02:34:13 2028 GMT
/etc/kubernetes/pki/etcd/ca.crt
notAfter=Oct 27 02:34:13 2028 GMT
/etc/kubernetes/pki/front-proxy-ca.crt
notAfter=Oct 27 02:34:15 2028 GMT

6,根据不同版本,查看证书过期的命令还不一样呢,最好再作个重复记录。
查看/etc/kubernetes/pki目录证书过期

CERT_DIR=${CERT_DIR:-/etc/kubernetes/pki}
for i in $(find $CERT_DIR -name '*.crt' -o -name '*.pem'); do
 echo $i
    openssl x509 -enddate -in $i -noout
done

查看/etc/kubernetes/目录下的几个conf里的证书过期

config_file=controller-manager.conf;echo $(grep "client-certificate-data" /etc/kubernetes/${config_file} | awk -F ":" '{print $2}' | grep -v "^$") | base64 -d > key_new.crt; openssl x509 -in key_new.crt -noout -dates

config_file=scheduler.conf;echo $(grep "client-certificate-data" /etc/kubernetes/${config_file} | awk -F ":" '{print $2}' | grep -v "^$") | base64 -d > key_new.crt; openssl x509 -in key_new.crt -noout -dates

config_file=admin.conf;echo $(grep "client-certificate-data" /etc/kubernetes/${config_file} | awk -F ":" '{print $2}' | grep -v "^$") | base64 -d > key_new.crt; openssl x509 -in key_new.crt -noout -dates

config_file=kubelet.conf;echo $(grep "client-certificate-data" /etc/kubernetes/${config_file} | awk -F ":" '{print $2}' | grep -v "^$") | base64 -d > key_new.crt; openssl x509 -in key_new.crt -noout -dates

config_file=front-proxy-client.crt;echo $(grep "client-certificate-data" /etc/kubernetes/${config_f

作者:万州客
链接:https://www.jianshu.com/p/cdffaff82a34
来源:简书
著作权归作者所有。商业转载请联系作者获得授权,非商业转载请注明出处。

发表评论

邮箱地址不会被公开。 必填项已用*标注