# k8s Certificates Renewal Verification

## Change to a Past Time

```bash
/opt/pnt/venv/bin/ansible-playbook -i /opt/pnt/kubespray/inventory/nsp-deployer-default/hosts.yml /opt/pnt/kubespray/change-notes-time.yml -e new_time="2020-02-17 16:27:00"

[root@nsp-k8s-deployer ~]# cat /opt/pnt/kubespray/change-notes-time.yml 
- name: Prepare and reset the required environment for a new build
  hosts: all
  gather_facts: yes
  user: root
  vars:
    new_time: "2021-02-18 12:50:00"
  tasks:
    - name: Disable ntp
      command: "timedatectl set-ntp no"

    - name: Set the new time
      command: "timedatectl set-time '{{ new_time }}'"

    - name: Enable ntp
      command: "timedatectl set-ntp yes"
```

## Deploy a New Cluster

```bash
nsp-kubernetes-deployer.bash -i 135.121.159.105 -l node-labels-standard-3nodes.yml
```

## Change Time to Now

```bash
/opt/pnt/venv/bin/ansible-playbook -i /opt/pnt/kubespray/inventory/nsp-deployer-default/hosts.yml /opt/pnt/kubespray/change-notes-time.yml -e new_time="2021-02-17 16:27:00"
```

## Renew Certificates (On All Master Nodes)

1. https://v1-16.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/
   For 1.16.3, somehow `renew all` does not work well. Renew each component one-by-one.

   ```yaml
   [root@nsp-k8s-deployer ~]# cat /opt/pnt/kubespray/renew-certs.yml 
   - name: Prepare and reset the required environment for a new build
     hosts: "{{ groups['kube-master'] }}"
     gather_facts: yes
     user: root
     vars:
       cluster_components:
       - admin.conf
       - apiserver
       - apiserver-kubelet-client
       - controller-manager.conf
       - front-proxy-client
       - scheduler.conf
     tasks:
       - name: Renew certificates
         command: "kubeadm alpha certs renew {{ item }} --config /etc/kubernetes/kubeadm-config.yaml"
         loop: "{{ cluster_components }}"
   ```

2. On nodes created with `kubeadm init`, prior to kubeadm version 1.17, there is a bug where you manually have to modify the contents of `kubelet.conf`. After `kubeadm init` finishes, you should update `kubelet.conf` to point to the rotated kubelet client certificates by replacing `client-certificate-data` and `client-key-data` with:

   ```yaml
   client-certificate: /etc/kubernetes/pki/apiserver-kubelet-client.crt
   client-key: /etc/kubernetes/pki/apiserver-kubelet-client.key
   ```

3. Restart `kube-apiserver`

   ```bash
   docker stop $(docker ps | grep kube-apiserver | awk {'print $1'}) 
   ```

4. Update `.kube/config` for errors like "error: You must be logged in to the server (Unauthorized)"

   ```yaml
   ...
   current-context: kubernetes-admin@cluster.local
   kind: Config
   preferences: {}
   users:
   - name: kubernetes-admin
     user:
       client-certificate: /etc/kubernetes/pki/apiserver-kubelet-client.crt
       client-key: /etc/kubernetes/pki/apiserver-kubelet-client.key
   ```