{"id":308,"date":"2023-05-01T11:49:17","date_gmt":"2023-05-01T11:49:17","guid":{"rendered":"https:\/\/devopsopen.com\/?p=308"},"modified":"2024-11-11T19:34:46","modified_gmt":"2024-11-11T19:34:46","slug":"installation-k8s","status":"publish","type":"post","link":"https:\/\/devopsopen.com\/index.php\/2023\/05\/01\/installation-k8s\/","title":{"rendered":"Installation K8S"},"content":{"rendered":"<ul>\n<li>\n<p><strong>Architecture<\/strong> <\/p>\n<ul>\n<li><strong>Single-node<\/strong><br \/>\nWith a single-node deployment, all the components run on the same server. This is great for testing, learning, and developing around Kubernetes.<\/li>\n<li><strong>Single head node, multiple workers<\/strong><br \/>\nAdding more workers, a single head node and multiple workers typically will consist of a single node etcd instance running on the head node with the API, the scheduler, and the controller-manager.<\/li>\n<li><strong>Multiple head nodes with HA, multiple workers<\/strong><br \/>\nMultiple head nodes in an HA configuration and multiple workers add more durability to the cluster. The API server will be fronted by a load balancer, the scheduler and the controller-manager will elect a leader (which is configured via flags). The etcd setup can still be single node.<\/li>\n<li><strong>HA etcd, HA head nodes, multiple workers<\/strong><br \/>\nThe most advanced and resilient setup would be an HA etcd cluster, with HA head nodes and multiple workers. Also, etcd would run as a true cluster, which would provide HA and would run on nodes separate from the Kubernetes head nodes.<\/li>\n<\/ul>\n<\/li>\n<li>\n<p><strong>Installation K8S components in all nodes (workers and master)<\/strong><br \/>\nThe aim is to install K8S 1.27 So you should see compabilities.<br \/>\n<strong>Linux<\/strong> : linux version Ubunto 20.04<br \/>\n<strong>Containerd<\/strong> : <a href=\"https:\/\/containerd.io\/releases\/\">https:\/\/containerd.io\/releases\/<\/a> for compatibilities between containerd and Kubernetes.<br \/>\n<strong>disable swap<\/strong><\/p>\n<pre><code>swapoff -a\nvi \/etc\/fstab<\/code><\/pre>\n<p><strong>containerd prerequisites<\/strong><\/p>\n<pre><code>sudo modprobe overlay\nsudo modprobe br_netfilter<\/code><\/pre>\n<p><strong>Setup required sysctl params, these persist across reboots<\/strong><\/p>\n<pre><code>cat &lt;&lt;EOF | sudo tee \/etc\/modules-load.d\/containerd.configure\noverlay\nbr_netfilter\nEOF<\/code><\/pre>\n<pre><code>cat &lt;&lt;EOF | sudo tee \/etc\/sysctl.d\/99-kubernetes-cri.conf\nnet.bridge.bridge-nf-call-iptables  = 1\nnet.ipv4.ip_forward                 = 1\nnet.bridge.bridge-nf-call-ip6tables = 1\nEOF<\/code><\/pre>\n<pre><code>sudo sysctl --system<\/code><\/pre>\n<p><strong>Install containerd<\/strong><\/p>\n<pre><code>sudo apt-get update\nsudo apt-get install -y containerd<\/code><\/pre>\n<p><strong>configure containerd<\/strong><\/p>\n<pre><code>sudo mkdir -p \/etc\/containerd\ncontainerd config default | sudo tee \/etc\/containerd\/config.toml<\/code><\/pre>\n<p><strong>Using the systemd cgroup driver<\/strong><\/p>\n<pre><code>sudo vi \/etc\/containerd\/config.toml\n[plugins.&quot;io.containerd.grpc.v1.cri&quot;.containerd.runtimes.runc]\n...\n[plugins.&quot;io.containerd.grpc.v1.cri&quot;.containerd.runtimes.runc.options]\nSystemdCgroup = true<\/code><\/pre>\n<p><strong>restart containerd with new configuration<\/strong><\/p>\n<pre><code>sudo systemctl restart containerd<\/code><\/pre>\n<p><strong>add google's apt repository gpg key<\/strong><\/p>\n<pre><code>curl -s https:\/\/packages.cloud.google.com\/apt\/doc\/apt-key.gpg | sudo apt-key add -<\/code><\/pre>\n<p><strong>Deprecated<\/strong> visit <a href=\"https:\/\/facsiaginsa.com\/kubernetes\/install-kubernetes-single-master\">https:\/\/facsiaginsa.com\/kubernetes\/install-kubernetes-single-master<\/a><br \/>\n<strong>add the kubernetes  apt repository<\/strong><\/p>\n<pre><code>sudo bash -c &#039;cat &lt;&lt;EOF&gt;\/etc\/apt\/sources.list.d\/kubernetes.list\ndeb https:\/\/apt.kubernetes.io\/ kubernetes-xenial main \nEOF&#039;<\/code><\/pre>\n<p><strong>update the package list and use apt-cache policy to inspect versions available in the repository<\/strong><\/p>\n<pre><code>sudo apt-get update\napt-cache policy kubelet | head -n 20<\/code><\/pre>\n<p><strong>Install the required packages, if needed we can request a specific versions<\/strong><\/p>\n<pre><code>VESRION=1.20.1-00\nsudo echo $VERSION\nsudo apt-get install -y kubelet=$VERSION kubeadm=$VERSION kubectl=$VERSION\nsudo apt-mark hold kubelet kubeadm kubectl containerd<\/code><\/pre>\n<p><strong>check status of ours kubelet and container runtime, containerd should be active<\/strong><\/p>\n<pre><code>sudo systemctl status kubelet.service\nsudo systemctl status containerd.service<\/code><\/pre>\n<\/li>\n<li>\n<p><strong>Installation K8S components master node<\/strong><\/p>\n<ul>\n<li><strong>Configure Cluster<\/strong><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>Kubeadm configuration file<\/p>\n<pre><code>kubeadm config print init-defaults | tee ClusterConfiguration.yaml<\/code><\/pre>\n<p>The ip endpoint for the api server localAPEndpoint.advertiseAdress<\/p>\n<pre><code>sed -i &#039;s\/ advertiseAddress: 1.2.3.4\/ advertiseAddress: 10.2.0.2\/&#039; ClusterConfiguration.yaml<\/code><\/pre>\n<p>Change nodeRegistration.criSocket from docker to containerd<\/p>\n<pre><code>sed -i &#039;s\/  criSocket: \\\/var\\\/run\\\/dockershim\\.sock\/  criSocket: \\\/run\\\/containerd\\\/containerd\\.sock\/&#039; ClusterConfiguration.yaml<\/code><\/pre>\n<p>Set the cgroup driver for kubelet to systemd, it's not set in this file yet, the default is cgroupfs<\/p>\n<pre><code>cat &lt;&lt;EOF | cat &gt;&gt; ClusterConfiguration.yaml\n---\napiVersion: kubelet.config.k8s.io\/v1beta1\nkind: KubeletConfiguration\ncgroupDriver: systemd\nEOF<\/code><\/pre>\n<p>check file before init if all is midfied<\/p>\n<pre><code>vi ClusterConfiguration.yaml<\/code><\/pre>\n<p>Init the cluster with configuration<\/p>\n<pre><code>sudo kubeadm init \\\n  --config=ClusterConfiguration.yaml<\/code><\/pre>\n<blockquote>\n<blockquote>\n<p>Note : you can have the join command. execute it into the worker to join the master<\/p>\n<\/blockquote>\n<\/blockquote>\n<pre><code>  mkdir -p $HOME\/.kube\n  sudo cp -i \/etc\/kubernetes\/admin.conf $HOME\/.kube\/config\n  sudo chown $(id -u):$(id -g) $HOME\/.kube\/config<\/code><\/pre>\n<ul>\n<li><strong>Configure and Install Calico<\/strong><br \/>\nPull down Calico and choose the version compatible with K8S<\/p>\n<pre><code>curl https:\/\/raw.githubusercontent.com\/projectcalico\/calico\/v3.26.1\/manifests\/calico.yaml -o calico.yaml<\/code><\/pre>\n<p>Edit the calico-node DaemonSet and Configure the initial IP Pool prefix by setting<br \/>\nfind the setting for pod network ip adress rang CALICO_IPV4POOL_CIDR and active CALICO_IPV4POOL_CIDR with the range 10.48.0.0\/24  (range pods)<\/p>\n<pre><code>vi calcio.yaml\n`CALICO_IPV4POOL_CIDR` to 10.48.0.0\/24<\/code><\/pre>\n<p>Apply calcio file<\/p>\n<pre><code>kubectl apply -f calico.yaml<\/code><\/pre>\n<\/li>\n<\/ul>\n<ul>\n<li><strong> Join worker with master if you dont apply the command before<\/strong><\/li>\n<\/ul>\n<p>ensure both are to srtart when the system starts up<\/p>\n<pre><code>sudo systemctl enable kubelet.service\nsudo systemctl enable containerd.service<\/code><\/pre>\n<p>if you dont keep the token from the master node<br \/>\ngo to the master and apply thoses commands <\/p>\n<pre><code>kubeadm token list<\/code><\/pre>\n<p>if there is no token or timeout, create new token<\/p>\n<pre><code>kubeadm token create<\/code><\/pre>\n<p>or<\/p>\n<pre><code>kubeadm token create --print-join-command<\/code><\/pre>\n<p>on the control plane you can find the CA cert hash<\/p>\n<pre><code>openssl x509 -pubkey \\\n-in \/etc\/kubernetes\/pki\/ca.crt | openssl rsa \\\n-pubin -outform der 2&gt;\/dev\/null | openssl dgst \\\n-sha256 -hex | sed &#039;s\/\u02c6.* \/\/&#039;<\/code><\/pre>\n<p>in the worker join the master or the control plane by executing this command<\/p>\n<pre><code>sudo kubeadm join 10.2.0.2:6443 --token 5u4jqt.w6xzvlx3zh7qf9gx \\\n    --discovery-token-ca-cert-hash sha256:249324363466a62b77472d633b88e600c58f319e9690a8c0ca699c3e8942deee<\/code><\/pre>\n<p>return to the master and apply commands bellow<br \/>\ncheck the second node existance<\/p>\n<pre><code>kubectl get nodes<\/code><\/pre>\n<p>check if status are ready<\/p>\n<pre><code>kubectl get pods --all-namespaces --watch<\/code><\/pre>\n<ul>\n<li><strong>Check installation<\/strong><\/li>\n<\/ul>\n<p>check if all pods are running<\/p>\n<pre><code>kubectl get pods --all-namespaces\n#or witrh command \nkubectl get pods --all-namespaces --watch\nkubectl get nodes<\/code><\/pre>\n<p>check systemd units<\/p>\n<pre><code>sudo systemctl status kubelet.service<\/code><\/pre>\n<p>check out the static pod manifests on the control plane nodeRegistration<\/p>\n<pre><code>ls \/etc\/kubernetes\/manifets<\/code><\/pre>\n<p>look more closely at API server and etcd's manifests<\/p>\n<pre><code>sudo more \/etc\/kubernetes\/manifests\/etcd.yaml\nsudo more \/etc\/kubernetes\/manifests\/etcd.yaml<\/code><\/pre>\n<p>check out the directory where the kubeconfig files live for each of the control plane pods<\/p>\n<pre><code>ls \/etc\/kubernetes<\/code><\/pre>\n<ul>\n<li><strong>Example of command into Worker node<\/strong><\/li>\n<\/ul>\n<pre><code>    1  ip link\n    2  sudo modprobe overlay\n    3  sudo modprobe br_netfilter\n    4  cat &lt;&lt;EOF | sudo tee \/etc\/modules-load.d\/containerd.configure\n    5  overlay\n    6  br_netfilter\n    7  EOF\n    8  cat &lt;&lt;EOF | sudo tee \/etc\/sysctl.d\/99-kubernetes-cri.conf\n    9  net.bridge.bridge-nf-call-iptables  = 1\n   10  net.ipv4.ip_forward                 = 1\n   11  net.bridge.bridge-nf-call-ip6tables = 1\n   12  EOF\n   13  sudo sysctl --system\n   14  sudo apt-get update\n   15  sudo apt-get install -y containerd\n   16  ls \/etc\/containerd\n   17  sudo mkdir -p \/etc\/containerd\n   18  containerd config default | sudo tee \/etc\/containerd\/config.toml\n   19  sudo vi \/etc\/containerd\/config.toml\n   20  sudo systemctl restart containerd\n   21  ps ea|grep containerd\n   22  curl -s https:\/\/packages.cloud.google.com\/apt\/doc\/apt-key.gpg | sudo apt-key add -\n   23  sudo bash -c &#039;cat &lt;&lt;EOF&gt;\/etc\/apt\/sources.list.d\/kubernetes.list\n   24  deb https:\/\/apt.kubernetes.io\/ kubernetes-xenial main\n   25  EOF&#039;\n   26  sudo apt-get update\n   27  apt-cache policy kubelet | head -n 20\n   28  VESRION=1.27.1-00\n   29  sudo echo $VERSION\n   30  VERSION=1.27.1-00\n   31  sudo echo $VERSION\n   32  sudo apt-get install -y kubelet=$VERSION kubeadm=$VERSION kubectl=$VERSION\n   33  sudo apt-mark hold kubelet kubeadm kubectl containerd\n   34  sudo systemctl status kubelet.service\n   35  sudo systemctl status containerd.service\n   36  sudo vi \/etc\/containerd\/config.toml\n   37  sudo systemctl restart containerd\n   38  sudo systemctl status containerd.service\n       kubeadm join 100.92.88.11:6443 --token nlu90p.5m55dmv3zcfpfmkc         --discovery-token-ca-cert-hash sha256:b89ad12d8803ba97588701c684b2b36aa287d28161e060167a865b6ed68deb4c\n   41  kubectl get node<\/code><\/pre>\n<ul>\n<li><strong>Install nginx Controller<\/strong><br \/>\nCheck compatibilities before installing :<\/p>\n<pre><code>https:\/\/github.com\/kubernetes\/ingress-nginx<\/code><\/pre>\n<\/li>\n<\/ul>\n<p>install the ingress controller<br \/>\ncommand bellow or visit the website ; <a href=\"https:\/\/github.com\/kubernetes\/ingress-nginx\/blob\/main\/docs\/deploy\/index.md#bare-metal-clusters\">https:\/\/github.com\/kubernetes\/ingress-nginx\/blob\/main\/docs\/deploy\/index.md#bare-metal-clusters<\/a><\/p>\n<pre><code>    kubectl create namespace ingress-nginx\n    kubectl apply -f https:\/\/raw.githubusercontent.com\/kubernetes\/ingress-nginx\/controller-v1.8.1\/deploy\/static\/provider\/cloud\/deploy.yaml\n    kubectl get pods --namespace=ingress-nginx<\/code><\/pre>\n<p>to expose services via ingress controller <\/p>\n<pre><code>https:\/\/github.com\/kubernetes\/ingress-nginx\/blob\/main\/docs\/deploy\/baremetal.md<\/code><\/pre>\n<p>the yaml file to expose the ingress controller via a service is :<\/p>\n<pre><code>kubectl apply -f - &lt;&lt;EOF \napiVersion: v1\nkind: Service\nmetadata: \n  name: ingress-nginx\nspec:\n  selector:\n    app.kubernetes.io\/component: controller\n  externalIPs: \n    - 100.57.100.100\n  type: NodePort\n  ports: \n    - \n      name: http\n      port: 80\n      protocol: TCP\n      targetPort: 80\n    - \n      name: https\n      port: 443\n      protocol: TCP\n      targetPort: 443\nEOF<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Architecture Single-node With a single-node deployment, all the components run on the same server. This is great for testing, learning, and developing around Kubernetes. Single head node, multiple workers Adding more workers, a single head node and multiple workers typically will consist of a single node etcd instance running on the head node with the API, the scheduler, and the controller-manager. Multiple head nodes with HA, multiple workers Multiple head nodes in an HA configuration and multiple workers add more durability to the cluster. The API server will be fronted by a load balancer, the scheduler and the controller-manager will\u2026<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_uag_custom_page_level_css":""},"categories":[12],"tags":[],"blocksy_meta":{"styles_descriptor":{"styles":{"desktop":"","tablet":"","mobile":""},"google_fonts":[],"version":5}},"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false},"uagb_author_info":{"display_name":"admin","author_link":"https:\/\/devopsopen.com\/index.php\/author\/admin_bak\/"},"uagb_comment_info":37552,"uagb_excerpt":"Architecture Single-node With a single-node deployment, all the components run on the same server. This is great for testing, learning, and developing around Kubernetes. Single head node, multiple workers Adding more workers, a single head node and multiple workers typically will consist of a single node etcd instance running on the head node with the&hellip;","_links":{"self":[{"href":"https:\/\/devopsopen.com\/index.php\/wp-json\/wp\/v2\/posts\/308"}],"collection":[{"href":"https:\/\/devopsopen.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devopsopen.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devopsopen.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/devopsopen.com\/index.php\/wp-json\/wp\/v2\/comments?post=308"}],"version-history":[{"count":15,"href":"https:\/\/devopsopen.com\/index.php\/wp-json\/wp\/v2\/posts\/308\/revisions"}],"predecessor-version":[{"id":456,"href":"https:\/\/devopsopen.com\/index.php\/wp-json\/wp\/v2\/posts\/308\/revisions\/456"}],"wp:attachment":[{"href":"https:\/\/devopsopen.com\/index.php\/wp-json\/wp\/v2\/media?parent=308"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devopsopen.com\/index.php\/wp-json\/wp\/v2\/categories?post=308"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devopsopen.com\/index.php\/wp-json\/wp\/v2\/tags?post=308"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}