什么是 Kubernetes?

Kubernetes（通常简称为K8s）是一个开源的容器编排平台，用于自动化部署、扩展和管理容器化应用程序。Kubernetes可以更高效地管理容器化的应用，支持自动扩展、负载均衡、服务发现、配置管理等功能。

Kubernetes 架构

• 控制平面 (Control Plane)： 负责管理工作节点和维护集群状态 所有任务分配的来源
• 工作节点 (Work Plane)： 负责执行由控制平面分配的请求任务

Kubernetes 结构

注意这里是从大到小

• Cluster 集群： 可以变相理解为一个完整的kubernetes区域
• Node 节点： 可以是一台服或多台的服务器虚拟机，提供资源运行容器
• Pod 节点集： 一个或多个容器运行的区域，它们共享资源（如文件等）
• Container 容器： 也就是容器：）

Kubernetes 组件

1. 控制平面组件（Control Plane Components）：

   • kube-apiserver：提供API接口，是Kubernetes控制平面的前端。
   • etcd：分布式键值存储，用于保存集群的所有配置数据。
   • kube-scheduler：负责调度Pod到合适的节点上。
   • kube-controller-manager：运行控制器进程，如节点控制器、副本控制器等。
   • cloud-controller-manager：与云服务提供商交互，管理云资源。

2. 节点组件（Node Components）：

   • kubelet：在每个节点上运行，确保容器在Pod中运行。
   • kube-proxy：维护节点上的网络规则，实现服务和Pod的网络通信。
   • 容器运行时：如Docker、containerd等，负责运行容器。

安装

老入言

在后面（从初始化开始）遇到的一切与网络有关的问题

全 部 都 是

modprobe br_netfilter
echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 1 > /proc/sys/net/ipv4/ip_forward

导致的^^

操作环境

[root@localhost ~]# neofetch 
        #####           root@localhost 
       #######          -------------- 
       ##O#O##          OS: Rocky Linux 9.2 (Blue Onyx) x86_64 
       #######          Host: VMware Virtual Platform None 
     ###########        Kernel: 5.14.0-284.11.1.el9_2.x86_64 
    #############       Uptime: 56 mins 
   ###############      Packages: 1180 (rpm) 
   ################     Shell: bash 5.1.8 
  #################     Resolution: 1694x926 
#####################   DE: GNOME 40.10 
#####################   WM: Mutter 
  #################     WM Theme: Adwaita 
                        Theme: Adwaita [GTK2/3] 
                        Icons: Adwaita [GTK2/3] 
                        Terminal: gnome-terminal 
                        CPU: Intel i5-10500 (2) @ 3.096GHz 
                        GPU: 00:0f.0 VMware SVGA II Adapter 
                        Memory: 1263MiB / 2755MiB 

准备

• 关闭防火墙

  systemctl stop firewalld && systemctl disable firewalld && iptables -F

• 关闭selinux

  sed -i 's/enforcing/disabled/' /etc/selinux/config && setenforce 0

• 关闭swap

  swapoff -a

• 永久关闭swap

  sed -ri 's/.*swap.*/#&/' /etc/fstab

• 确保 hostname 不同

  hostnamectl set-hostname master && bash
  hostnamectl set-hostname node1 && bash
  
  cat >> /etc/hosts <<EOF
  192.168.10.3 master
  192.168.10.5 node1
  EOF

修改内核参数

cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

sysctl --system

安装Docker

方法一（网络不好的情况）

• 配置yum源

  wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo

• 安装Docker

  yum -y install docker-ce docker-ce-cli

• 配置Docker

  cat > /etc/docker/daemon.json << EOF
  {
  "registry-mirrors": ["https://gqs7xcfd.mirror.aliyuncs.com","https://hub-mirror.c.163.com"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
  "max-size": "100m"
  },
  "storage-driver": "overlay2"
  }
  EOF

• 启动Docker服务

  systemctl daemon-reload && systemctl enable docker && systemctl start docker

方法二（网络还行的情况）

# 安装依赖
dnf -y install dnf-plugins-core
# 安装源
dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
# 安装Docker
dnf -y install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
# 启动
systemctl enable --now docker

如果无法拉取镜像，尝试修改DNS：

vim /etc/resolv.conf

# Generated by NetworkManager
search localdomain
nameserver 8.8.8.8
nameserver 223.5.5.5
nameserver 1.1.1.1 

# 重启生效
systemctl restart NetworkManager
或者
nmcli connection up ens160（你的网卡，tab出来）

一些前置条件

# 修改内核参数
modprobe br_netfilter
echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 1 > /proc/sys/net/ipv4/ip_forward

# containerd设置
systemctl enable containerd
注释掉/etc/containerd/config.toml中的disabled_plugins = ["cri"]
systemctl restart containerd

安装k8s

• 配置yum源

  cat > /etc/yum.repos.d/kubernetes.repo << EOF
  [kubernetes]
  name=Kubernetes
  baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
  enabled=1
  gpgcheck=1
  repo_gpgcheck=1
  gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
  EOF

• 安装

  yum install -y kubelet kubeadm kubectl

• 开机自启

  systemctl enable kubelet

初始化Master并将节点加入集群

初始化

# 执行master初始化
kubeadm init
# 显示详细信息（更多日志）
kubeadm init --v=5
# 成功示例
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.10.3:6443 --token h78x81.9zhf8o4i046fgh3m \
        --discovery-token-ca-cert-hash sha256:edc70987fe8c91d7130d7928b6421ef505e6f8b56908ec87c029b0550218f37a 

使用阿里云镜像源拉取并初始化

docker pull registry.aliyuncs.com/google_containers/kube-apiserver:v1.28.15
docker tag registry.aliyuncs.com/google_containers/kube-apiserver:v1.28.15 registry.k8s.io/kube-apiserver:v1.28.15

vim  /etc/containerd/config.toml 
[plugins."io.containerd.grpc.v1.cri".registry]
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
    [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
      endpoint = ["https://------.mirror.aliyuncs.com","https://registry-1.docker.io"]

[root@master ~]# kubeadm init
I1125 09:10:25.727093    5123 version.go:256] remote version is much newer: v1.31.3; falling back to: stable-1.28
[init] Using Kubernetes version: v1.28.15
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
W1125 09:19:12.478430    5123 checks.go:835] detected that the sandbox image "registry.k8s.io/pause:3.8" of the container runtime is inconsistent with that used by kubeadm. It is recommended that using "registry.k8s.io/pause:3.9" as the CRI sandbox image.

坐 和 放 宽！

产生这个问题的原因是：k8s的依赖库无法访问

解决方法：移民到朝鲜、俄罗斯、伊朗以外的任何国家

常见问题：

[WARNING Hostname]: hostname "master" could not be reached

# 主机名不可达
vim /etc/hosts
192.168.10.3 master

[ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables does not exist

# br_netfilter模块未加载（前面你可能配置了，但这个重启后好像会失效）
modprobe br_netfilter
echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 1 > /proc/sys/net/ipv4/ip_forward

[ERROR Mem]: the system RAM (433 MB) is less than the minimum 1700 MB

加内存

(更多待补充...)

加入集群

# 在初始化集群的最后输出一个命令，这个是节点用来加入集群的命令
kubeadm join 192.168.10.3:6443 --token h78x81.9zhf8o4i046fgh3m \
        --discovery-token-ca-cert-hash sha256:edc70987fe8c91d7130d7928b6421ef505e6f8b56908ec87c029b0550218f37a 

# 示例
[root@localhost ~]# kubeadm join 192.168.10.3:6443 --token h78x81.9zhf8o4i046fgh3m         --discovery-token-ca-cert-hash sha256:edc70987fe8c91d7130d7928b6421ef505e6f8b56908ec87c029b0550218f37a 
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

查看节点状态

# 查看节点
kubectl get nodes
[root@master ~]# kubectl get nodes
NAME                    STATUS     ROLES           AGE   VERSION
localhost.localdomain   NotReady   <none>          16m   v1.28.2
master                  NotReady   control-plane   17m   v1.28.2

# 不成功试试这个
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
# 还有这个
modprobe br_netfilter
echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 1 > /proc/sys/net/ipv4/ip_forward
# 如果显示
The connection to the server <host>:6443 was refused - did you specify the right host or port?
# 尝试
export KUBECONFIG=/etc/kubernetes/kubelet.conf
export KUBECONFIG=/etc/kubernetes/admin.conf
KUBECONFIG=$HOME/.kube/config
禁用交换分区
重启
更多待补充

一个简单(并不)的实操

要求

在服务器1、服务器2上安装containerd.io和kubernetes

• 服务器1作为master node
• 服务器2作为work node
• containerd的namespace为k8s.io

• 使用containerd.sock作为容器runtime-endpoint和image-endpoint

  master节点配置calico

# 安装Calico
kubectl create -f https://docs.projectcalico.org/manifests/tigera-operator.yaml
kubectl create -f https://docs.projectcalico.org/manifests/custom-resources.yaml

# 检查命名空间，如果没有calico-system新建
kubectl get namespaces
[root@master .kube]# kubectl get namespaces
NAME              STATUS   AGE
default           Active   30m
kube-node-lease   Active   30m
kube-public       Active   30m
kube-system       Active   30m
tigera-operator   Active   4m1s
# 没有就新建
kubectl create namespace calico-system
[root@master .kube]# kubectl create namespace calico-system
namespace/calico-system created