Kubernetes cluster installation.

########### Kubernetes Cluster Setup ######


############ for Master ##############
Step 1: Prepare Hostname, Firewall and SELinux
On your master node, set the hostname and if you don’t have a DNS server, then also update your /etc/hosts file.
# vim /etc/hosts
192.168.56.101        master.example.com master
192.168.56.102        worker01
192.168.56.103        worker02
 
# Install Kubernetes
# sudo modprobe overlay
# sudo modprobe br_netfilter
# cat  <<EOF | sudo tee /etc/sysctl.d/kube.conf
net/bridge/bridge-nf-call-ip6tables = 1
net/bridge/bridge-nf-call-iptables = 1
net/bridge/bridge-nf-call-arptables = 1
EOF
 
disable SElinux and update your firewall rules.
#sudo setenforce 0
#sudo sed -i 's/^SELINUX=.*/SELINUX=permissive/g' /etc/selinux/config
 
# cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
 
# sudo sysctl --system
# sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
# sudo swapoff -a
# yum install -y yum-utils
# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
# yum install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin  kubelet kubectl kubeadm kubernetes-cni
# systemctl enable kubelet
# systemctl enable docker
# service firewalld stop
# systemctl disable firewalld
# rm -f /etc/containerd/config.toml
# systemctl restart containerd
# systemctl enable containerd
# echo -e "192.168.50.10 master\n192.168.50.11 worker1\n192.168.50.12 worker2" >> /etc/hosts
# kubeadm config images pull
# kubeadm init --apiserver-advertise-address=192.168.50.10 --pod-network-cidr=10.244.0.0/16
# mkdir -p $HOME/.kube
# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
# sudo chown $(id -u):$(id -g) $HOME/.kube/config
# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
# systemctl status containerd
# systemctl status kubelet
 
######################## worker01 ##############################
Step 1: Prepare Hostname, Firewall and SELinux
 
On your master node, set the hostname and if you don’t have a DNS server, then also update your /etc/hosts file.
# vim /etc/hosts
192.168.56.101        master.example.com master
192.168.56.102        worker01
192.168.56.103        worker02
 
sudo modprobe overlay
sudo modprobe br_netfilter
cat  <<EOF | sudo tee /etc/sysctl.d/kube.conf
net/bridge/bridge-nf-call-ip6tables = 1
net/bridge/bridge-nf-call-iptables = 1
net/bridge/bridge-nf-call-arptables = 1
EOF
 
 
# cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
 # sudo sysctl --system

 # sudo setenforce 0
 # sudo sed -i 's/^SELINUX=.*/SELINUX=permissive/g' /etc/selinux/config
 # sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
 # sudo swapoff -a
 # yum install -y yum-utils
 # yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
 # yum install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin  kubelet kubectl kubeadm kubernetes-cni
# systemctl enable kubelet
# systemctl enable docker
# service firewalld stop
# systemctl disable firewalld
# rm -f /etc/containerd/config.toml
# systemctl restart containerd
# systemctl status kubelet
# echo -e "192.168.56.101 master\n192.168.56.102 worker01\n192.168.56.103 worker02" >> /etc/hosts

# systemctl status kubelet --> nahi chalega..
# kubeadm join 192.168.56.101:6443 --token al34ik.aocuycxej7cu9e09\

      --discovery-token-ca-cert-hash sha256:79c40e306623488a85aac952930cad89d825069d0801c0c9b9dcca9897bd5a22 --v=5
# systemctl status kubelet --> worker join ke badh chal jayega..
# systemctl status containerd
# systemctl status kubelet
on Master  node command :- 
# sudo kubectl label node <worker01> node-role.kubernetes.io/worker=worker
[root@master ~]# kubectl get nodes
NAME                 STATUS   ROLES           AGE   VERSION
master.example.com   Ready    control-plane   11h   v1.28.2
worker01             Ready    worker          10h   v1.28.2
worker02             Ready    worker          10h   v1.28.2
 
####### Reset  k8s cluster ############
kubeadm reset
#####################################

What is a pod kubernetes ?

 1.What is a pod kubernetes ?

Pods is collection of containers . live together and die together. IP assigned to pod only. All containers in the same pod will communicate to localhost with different ports. Pods are the smallest unit that can be deployed and managed by Kubernetes.
Each Pod has its unique IP Address and ports within the cluster.
Any data saved inside the Pod will disappear without a persistent storage.

Pods create flow diagram :-

2.Pause :

2.Pause : First pause container is created after that the pods is created. Pause container is IP assigned to pods. Pause container that works with pods but will not be visible in pods. The input and output traffic of all the containers running in a pod will go through the Puase container.

3.How many pause container is running on k8s cluster.

# ctr -n k8s.io c ls 

4.How many pause container is running on woker side.

on woker01

on woker02

5.Pod lifecycle:

# kubectl get pods

NAME           READY    STATUS    RESTARTS      AGE
      nginxpod       2/2        Running          0               4m25s

NAME           :- Pod name
      READY 2/2   :- <running status of container>/<total of container in pod>
      STATUS        :-  status of pod.
      RESTARTS  :- restart count.
      AGE              :- running status time.

CrashLoopBackOff

CrashLoopBackOff is a Kubernetes state that indicates a restart loop is happening in a pod. We have to check by describing pod.

6.Container States:

The way Kubernetes maintain Pod phases, it maintains state of each container in the Pod.

Once the scheduler assigns a Pod to a Node, the kubelet starts creating containers for that Pod using a container runtime. There are 3 possible states for the container.

Types of Pod:

i)        Single Container pods.

ii)      Multi Container Pods.

Single Container pods :-

If you need to run a single container in Kubernetes, then you have to create a pod for it which is nothing but Single Container Pod. One or more containers running in single pod and one container as a helper container.

Multi Container pods :-

If you have to run two or more containers in a pod, then the pod created to place these containers is called a Multi Container Pod.

There are three common design patterns for multiple containers into a single pod.

i) Sidecar

ii) Adapter

iii) Ambassador

i) What is Kubernetes Sidecar?

Some great examples are using a sidecar for monitoring and logging and adding an agent for these purposes.

one of the pod will contain primary application while the second pod would contain sidecar container. The main application will write the logs into a log file, and the sidecar container will continuously read this log file and send the output to STDOUT. Now you may configure some logging agent to send these logs to a central log server.

 

# cat sidecar-pod.yaml

apiVersion: v1

kind: Pod

metadata:

  name: sidecar-pod

spec:

  volumes:

  - name: log

    emptyDir: {}

  containers:

  - image: busybox

    name: application

    command: ['sh', '-c', 'while true; do echo Today Date and Time is $(date) INFO Hello world..! >> /var/log/myapp.log && sleep 5; done']

    volumeMounts:

    - name: log

      mountPath: /var/log

  - name: sidecar

    image: busybox

    command: ['sh', '-c', 'tail -fn+1 /var/log/myapp.log']

    volumeMounts:

    - name: log

      mountPath: /var/log

# kubectl apply -f sidecar-pod.yaml

# kubectl get pods

How many container available in pods.

# kubectl get pod -o="custom-columns=NAME:.metadata.name,INIT-CONTAINERS:.spec.initContainers[*].name,CONTAINERS:.spec.containers[*].name"

                                                        or

# kubectl get pods <pods_name> -o jsonpath='{.spec.containers[*].name}'

# kubectl logs -f sidecar-pod -c sidecar 

ii) What is Adapter in kubernetes.

For example, you’re running multiple applications within separate containers, but every application has a different way of outputting log files.

Now, you have a centralized logging system that accepts logs in a particular format only. What can you do in such a situation? Well, you can either change the source code of each application to output a standard log format or use an adapter to standardize the logs before sending it to your central server. That’s where the adapter pattern comes in.

use case

let’s consider that an application that outputs logs in a particular format, that we want to change to something standard.

# vim adaptor.yaml

apiVersion: v1

kind: Pod

metadata:

  name: adapter-pod

  labels:

    app: adapter-app

spec:

  volumes:

  - name: logs

    emptyDir: {}

  containers:

  - name: app-container

    image: alpine

    command: ["/bin/sh"]

    args: ["-c", "while true; do date >> /var/log/app.log; sleep 2;done"]

    volumeMounts:

    - name: logs

      mountPath: /var/log

  - name: log-adapter

    image: alpine

    command: ["/bin/sh"]

    args: ["-c", "tail -f /var/log/app.log|sed -e 's/^/Date /' > /var/log/out.log"]

    volumeMounts:

    - name: logs

      mountPath: /var/log                      

In the manifest, we have an app-container that outputs a stream of dates to a log file. The log-adapter container adds a Date prefix in front. Yes, it’s a very rudimentary example, but enough to get how the adapter works.

# kubectl apply -f adaptor.yaml

# kubectl get pods

# kubectl exec -it adapter-pod -c app-container /bin/sh

# ps -elf | grep "date"

# cd  /var/log 

# cat app.log

# kubectl exec -it adapter-pod -c log-adapter /bin/sh

iii) What is Ambassador ?

If you look carefully in the manifest YAML, you will find there are three containers. The app-container-poller continuously calls http://localhost:81 and sends the content to /usr/share/nginx/html/index.html.

The app-container-server runs nginx and listens on port 80 to handle external requests. Both these containers share a common mountPath. That is similar to the sidecar approach.

There is an ambassador-container running within the pod that listens on localhost:81 and proxies the connection to example.com, so when we curl the app-container-server endpoint on port 80, we get a response from example.com.

# cat ambassador.yaml
apiVersion: v1

kind: Pod

metadata:

  name: ambassador-pod

  labels:

    app: ambassador-app

spec:

  volumes:

  - name: shared

    emptyDir: {}

  containers:

  - name: app-container-poller

    image: yauritux/busybox-curl

    command: ["/bin/sh"]

    args: ["-c", "while true; do curl 127.0.0.1:81 > /usr/share/nginx/html/index.html; sleep 10; done"]

    volumeMounts:

    - name: shared

      mountPath: /usr/share/nginx/html

  - name: app-container-server

    image: nginx

    ports:

      - containerPort: 80

    volumeMounts:

    - name: shared

      mountPath: /usr/share/nginx/html

  - name: ambassador-container

    image: bharamicrosystems/nginx-forward-proxy

    ports:

      - containerPort: 81

How to check container name in pods

# kubectl describe pods <pods_name>

How to check particular container log from pods.

# kubectl logs <pod_name> <container_name>

# kubectl get pods

# kubectl port-forward ambassador-pod 80:80

# curl localhost

Init Containers:

In Kubernetes, an init container is the one that starts and executes before other containers in the same Pod. Used to perform small tasks. If the task is completed after that container will exit.
specialized containers that run before app containers in a Pod. Init containers can contain utilities or setup scripts not present in an app image. You can specify init containers in the Pod specification alongside the containers array (which describes app containers).

The init-container pod will clone from the git web-content directory and the code will be downloaded. The logical name of the web-content directory is web-content-dir, after that the nginx pod will mount from the web-content-dir directory.

# cat init-container-pod.yaml

apiVersion: v1

kind: Pod

metadata:

  name: init-container-demo

spec:

  containers:

  - name: nginx

    image: nginx:alpine

    ports:

    - containerPort: 80

    volumeMounts:

    - name: web-content-dir

      mountPath: /usr/share/nginx/html

  initContainers:

  - name: helper

    image: alpine/git

    command:

    - git

    - clone

    - https://github.com/Praqma/simple-website.git

    - /web-content/

    volumeMounts:

    - name: web-content-dir

      mountPath: "/web-content"

  volumes:

  - name: web-content-dir

    emptyDir: {}

# kubectl describe pods init-container-demo

# kubectl apply -f  init-container-pod.yaml

# kubectl get pods -w

# kubectl get pod -o="custom-columns=NAME:.metadata.name,INIT-CONTAINERS:.spec.initContainername,CONTAINERS:.spec.containers[*].name"

Worker node side image download :- nginx pods delete hota hai to nginx image remove nhi hoti hai.

# ctr -n  k8s.io  c ls

# kubectl get pods -o wide

# curl -l 10.244.1.9

Ephemeral Containers :-

Ephemeral Containers are a feature in Kubernetes that allows users to run diagnostic or debugging containers within an existing Pod for a short duration. These containers are temporary and serve the purpose of troubleshooting, inspecting, or debugging issues within the context of a running Pod.

Diagnostic and Debugging Tools:

Same Network Namespace:

Same Storage Volumes:

No Impact on Application Containers:

Access to Pod Namespace:

# kubectl alpha debug -it <pod-name> --image=<debugger-image:tag>

Static Pods in Kubernetes :-

Static pods in Kubernetes are created and managed directly by the kubelet service on a node, without the need for a Kubernetes API server or scheduler. Kubelet can only create and manage static pods.

For example, kubeadm uses static pods to bring up Kubernetes control plane components like api-server, and controller-manager as static pods on the Master Node.

some pods running in the kube-system namespace are static pods on the master node.

Kubelet config file path :-

# vim /var/lib/kubelet/config.yaml

Static pods path :-

 These yaml files are running as a static pods.

# ls -ltr  /etc/kubernetes/manifests/

Creating a static pods on master node :-

# vim static_pods.yaml

apiVersion: v1

kind: Pod

metadata:

  name: static-web

  labels:

    role: myrole

spec:

  containers:

  - name: web

    image: nginx

    ports:

      - name: web

        containerPort: 80

        protocol: TCP

# cp -rvf    static_pods.yaml  /etc/kubernetes/manifests

# kubectl  get  pods

When deleting static_pods.yaml file static pods are removed.

# rm -rf /etc/kubernetes/manifests/static_pods.yaml

# kubectl  get  pods

On worker node :-

# cp -rvf    static_pods.yaml  /etc/kubernetes/manifests

# kubectl  get  pods

# kubectl describe pods static-web-worker01

 command :-

# kubectl run <pod_name>  --image=<image_name>

# kubectl get pods   (show all pods)

# kubectl run nginx --image=nginx

# kubectl describe pods <pods_name> (describe for pods)

# kubectl get pods --show-labels

# kubectl get pods -w (watch command)

# kubectl get po -o wide (which node, pod is working)

# kubectl delete pod <pod_name>

# kubectl delete pods --all (delete all pods)

# kubectl get pods <pod_name> (How many container are running in pod.)

# kubectl edit pod <pod_name>   (edit pod file update and changes.)

# kubectl explain pod --recursive | less  (help for create yaml file)

# kubectl get pods --all-namespaces

Kubernetes system namespaces use :- kube-system

New Pods default namespaces use :- default

#  kubectl get pods   -n  <namespaces>   (show pods only namespace)

# kubectl explain pod ( help for yaml file parami)

# kubectl  exec  <pod_name>  -c <container_name> -it bash (login container)

# kubectl describe pods nginx (check debug pods and containers)

# kubectl logs nginxpod -c rediscontainer (check logs for container)

# kubectl exec <pod_name> date  (execute date command)

# kubectl exec <pod_name> -c <container-name> date  (execute date command in single container)

# kubectl   logs  <pod_name>   (check logs for pods)

# kubectl   logs  -f   <pods_name>  -c  <container_name>  (check logs for container)   

 

# kubectl get pod -o="custom-columns=NAME:.metadata.name,INIT-CONTAINERS:.spec.initContainers[*].name,CONTAINERS:.spec.containers[*].name" (check container in pods)

# kubectl get pods <pods_name> -o jsonpath='{.spec.containers[*].name}' (check container in pods)

# kubectl exec -it <pods_name> -c <container_name> -- cat /etc/hostname (check hostname container)

# kubectl exec -it <pods_name> -c <container_name> -- ifconfig eth0 (check ip container)

 

Check yaml file (create and apply is same command)

# kubectl create -f first_pod.yaml --dry-run=client

# kubectl apply -f first_pod.yaml --dry-run=client

How to create yaml file ?

# kubectl explain pods

# kubectl explain pod --recursive | less

 

Example for help create pod yaml file

# kubectl run nginx --dry-run=client --image=nginx -o yaml

1.	Pause container	Pause container is IP assigned to pods.
2.	Single pods	Single pods contain a single container.
3.	Multi pods	run two or more containers in a pod is called multi pods.
4.	Side car pods	sidecar for monitoring and logging and adding an agent for these purposes.
5.	Adapter pods	centralized logging system that accepts logs in a particular format only.
6.	Ambassador pods	Ambassador is a Kubernetes-native API Gateway built on Envoy Proxy. Managed entirely via Kubernetes Custom Resource Definitions, Ambassador provides powerful capabilities for traffic management, authentication, and observability. Proxy database connection
7.	Staic pods	Static pods are managed directly by the kubelet on a specific node and are not part of the Kubernetes control plane.
8.	Init container	Init containers are containers that run and complete before the main application containers start.
9.	Ephemeral Contaoners	Ephemeral Containers are a feature in Kubernetes that allows users to run diagnostic or debugging containers within an existing Pod for a short duration.

Kubernetes cluster Components .

 1.Kubernetes cluster Components.

Control Plane Components (Master Node) :-
i)                    kube-apiserver
ii)                  etcd
iii)                kube-scheduler
iv)                kube-controller-manager
v)                  cloud-controller-manager  (optional):

            vi)         Kubelet (Manage static pods like kube-apiserver, kube-controller, kube-scheduler, etcd )

Components Worker Node :-
i)                    kubelet
ii)                  kube-proxy
iii)                Container Runtime
iv)                Optional: CNI Plugins (Container Network Interface):

ON Master Node :-

How many pods are running on k8s cluster.

How many pods are running on worker side.

1.Kube API Server :-

i) Kube-API-Server interacts with API, Its a frontend of the kubernetes control plane. API for almost every operation User interact with these API using a tool call Kubectl or UI through a dashboard.

ii) The Kubernetes API (Application Programming Interface) is a set of HTTP endpoints that allow you to interact with a Kubernetes cluster.

iii) The kube API serves as the communication interface between different components of Kubernetes and enables users to control and manage various aspects of the cluster.

iv) All operations (create, update, delete) are submitted to the API server, and it validates and processes them.

2.ETCD (etcd distributed key-value store):-

etcd is a simple distribute key value store. Kubernetes uses etcd as its database to store all cluster data's. Some of the data stored in etcd is job scheduling information, pods, state information and etc.

etcd is consistent and high-available store.

Fully replicated :- The entire state is available on every node in the cluster.

Secure :- Implements automatic TLS with optional client certificate authentication.

Fast :-  Benchmarked at 10,000 writes per sec.

3. kube-scheduler :-

Scheduler watches the pods and assigns the pods to run on specific hosts.

4.kube-controller-manager :-

1) Node controller - Its responsible for noticing and responding when nodes go down.

2) Replication controllers - It maintains the number of pods. It controls how many identical copies of a pod should be running somewhere on the cluster.

3) Endpoint controllers joins services and pods together.

4) Token controllers - Services account and Token controllers handles access managements.

5) ReplicaSet controllers ensure number of replication of pods running at all time.

6) Deployment controller provides declarative updates for pods and replicasets.

7) Daemonsets controller ensure all nodes run a copy of specific pods.

8) Jobs controller is the supervisor process for pods carrying out batch jobs Services allow the communication.

Namespace

CronJob

StatefulSet

9) PV controller is responsible for managing Persistent Volumes within a cluster.

5.cloud-controller-manager  (optional):

The cloud-controller-manager is a component in Kubernetes that allows cloud providers to integrate their services with a Kubernetes cluster. It provides a separation of concerns between cloud-specific operations and core Kubernetes functionality, making it easier to maintain and extend Kubernetes for different cloud environments.

6.Kubelet :-

Kubelet is the primary node agent runs on each nodes and reads the container manifests which ensures that containers are running and healthy.

7.Kube-proxy :-

Kube-proxy is a process helps us to have network proxy and loadbalancer for the services in a single worker node. It performs network routing for tcp and udp packets, and performs connection folding. Worker nodes can be exposed to internet via kubeproxy.

Pods workflow :-

1.       User request to Kube-Api-Server.

2.       kube-API-Server first creates a pod according to the memory and CPU requirements, only the container remains to be created.

3.       Kube-Scheduler checks the pod's resources, then the kube-scheduler decides the correct node.

4.       The kube-schedule sends requests to the kubelet to schedule pods.

5.       Kubelet makes a request to Docker then the container is created in Pods.

6.       kubelet sends pods status request to kube-api-server.

7.       kube-api-server communicates to ETCD DB and stores pod information in ETCD DB.

Kubectl :-

•  Kubectl is a command line configuration tool (CLI) for Kubernetes used to interact with master node of kubernetes. Kubectl has a config file called kubeconfig, this file has the information about server and authentication information to access the API Server.

Objects can be managed in three ways with Kubectl tools.        

i) Imperative Commands.

ii) Imperative Object Configuration.

iii) Declarative Object Configuration.

i) Imperative Commands :- Kubernetes objects can quickly be created, updated, and deleted directly using imperative commands built into the kubectl command-line tool.

ii) Imperative Object Configuration :- In this approach, we operate directly on live objects in a cluster using kubectl CLI as arguments or flags.

For example :

If the command creates objects via CLI or yaml files.

If you then make changes to the firstpod.yaml file and run the kubectl create -f firstpod.yaml command again, there will be no changes. If you want to change anything, you have to do it using the edit command.

 # kubectl edit  <podname>

Generate POD Manifest YAML file (-o yaml). Don't create it(--dry-run)

# kubectl run nginx --image=nginx --dry-run=client -o yaml

Advantages:

·    Commands are simple, easy to learn and remember.

·    Commands require only a single step to make changes to the cluster.

Disadvantages:

·   Commands cannot be committed to source control system and difficult to review.

·   Commands do not provide an audit trail associated with changes.

·  Commands do not provide a template for creating new objects.

iii) Declarative Object Configuration :- In this approach, we create YAML or JSON files for creating, updating, or deleting Kubernetes objects in a cluster using kubectl apply command. There is no need to change each object individually through commands. Change will happen without disturbance. If you create a yaml manifest file and use the apply command, the object will be created. If you make changes in the firstpod.yaml file and run it again command

 For example :-

# kubectl  apply  -f  firstpod.yaml

# kubectl create -f <filename|url>: # used for creating an object from a specified file

# kubectl replace -f <filename|url>: # used for updating a live object from a specified file

# kubectl delete -f <filename|url>: # used for deleting an object from a specified file

# kubectl get -f <filename|url> -o yaml: # used for viewing info about an object from a specified file

# kubectl apply -f <filename|url> : # used for creating/Updating an object from a specified file

# kubectl apply -f <directory> : # used for creating/Updating all objects specified in the directory

Advantages:

·                Object configurations can be stored in a source control system for reviewing changes before push.

·                Object configuration provides a template for creating new objects.

Disadvantages

·                Object configuration requires basic understanding of the object schema.

·                Object configuration requires the additional step of writing a YAML file.