Istio入门与实战最新章节_毛广献著

4.4　创建Kubernetes集群的步骤

经过上面的基础步骤后，下面介绍如何使用Kubeadm创建Kubernetes集群的步骤，并使安装Flannel网格插件实现容器间通信。

1.集群架构说明

集群由三台虚拟机组成，主机名分别为lab1、lab2、lab3，操作系统为CentOS 7.4版本。安装Kubernetes版本为1.12.4版本，网络插件使用Flannel来实现跨节点通信，每台虚拟机有两张网卡，分别为eth0、eth1。eth0用于连接外网，安装下载软件，eth1用来集群内部通信。实验采用1个主节点两个工作节点，lab1作为master节点，lab2、lab3作为node工作节点，如下所示：

2.初始化Master节点

下面对lab1节点进行初始化。

1）配置Docker和Kubelet开机启动：

#设置Docker开机启动
$ sudo systemctl enable docker.service
#设置Kubelet开机启动
$ sudo systemctl enable kubelet.service

2）生成配置文件。

根据实际情况修改如下的配置文件，11.11.11.111为master节点的IP地址，默认情况下会使用gcr.io的镜像，由于无法拉取镜像，会导致集群创建失败，所以应配置imageRepository，使用国内阿里云的镜像，kube-proxy使用了ipvs模式。使用如下命令写入配置文件：

$ cat >kubeadm-master.config<<EOF
apiVersion: kubeadm.k8s.io/v1alpha2
kind: MasterConfiguration
kubernetesVersion: v1.12.4
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
api:
  advertiseAddress: 11.11.11.111
controllerManagerExtraArgs:
  node-monitor-grace-period: 10s
  pod-eviction-timeout: 10s
networking:
  podSubnet: 10.244.0.0/16
kubeProxy:
  config:
    mode: ipvs
EOF

3）提前拉取镜像。

如果执行失败可以多次执行，使用如下命令提前拉取镜像：

$ sudo kubeadm config images pull --config kubeadm-master.config
[config/images] Pulled registry.cn-hangzhou.aliyuncs.com/google_containers/
    kube-apiserver:v1.12.4
[config/images] Pulled registry.cn-hangzhou.aliyuncs.com/google_containers/
    kube-controller-manager:v1.12.4
[config/images] Pulled registry.cn-hangzhou.aliyuncs.com/google_containers/
    kube-scheduler:v1.12.4
[config/images] Pulled registry.cn-hangzhou.aliyuncs.com/google_containers/
    kube-proxy:v1.12.4
[config/images] Pulled registry.cn-hangzhou.aliyuncs.com/google_containers/
    pause:3.1
[config/images] Pulled registry.cn-hangzhou.aliyuncs.com/google_containers/
    etcd:3.2.24
[config/images] Pulled registry.cn-hangzhou.aliyuncs.com/google_containers/
    coredns:1.2.2

4）执行初始化。

初始化完成之后保留命令输出的join相关命令，用于后面步骤node节点加入集群中。使用如下命令执行初始化：

$ sudo kubeadm init --config kubeadm-master.config
[init] using Kubernetes version: v1.12.4
[preflight] running pre-flight checks
[preflight/images] Pulling images required for setting up a Kubernetes cluster
[preflight/images] This might take a minute or two, depending on the speed of 
  your internet connection
[preflight/images] You can also perform this action in beforehand using 
  'kubeadm config images pull'
[kubelet] Writing kubelet environment file with flags to file "/var/lib/
  kubelet/kubeadm-flags.env"
[kubelet] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[preflight] Activating the kubelet service
...
[bootstraptoken] creating the "cluster-info" ConfigMap in the "kube-public" 
  namespace
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
Your Kubernetes master has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of machines by running the following on each node
as root:
  kubeadm join 11.11.11.111:6443 --token xlivmy.jj73qkeobqoyfs0r --discovery-
    token-ca-cert-hash sha256:c6290480baaef64f3f1c6b6861de8685ca74d5a4a7c2703
    eace18a2e2117e87d

5）配置使用kubectl。

另外打开一个主机lab1终端会话，以vagrant用户执行如下操作：

$ rm -rf $HOME/.kube
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

使用kubectl获取集群中节点信息，测试kubectl配置是否正确：

$ kubectl get node
NAME         STATUS           ROLES          AGE         VERSION
lab1         NotReady         master         56s         v1.12.4

6）配置master节点接收负载。

由于实验环境虚拟机内存可能不足，导致后面的Istio实验无法成功，可以让master节点和其他node节点一样接收负载，分担其他node节点的压力，但也可能使master节点压力过大，导致master上kubernetes组件出现异常，实验无法成功。只有当不配置此步骤实验就不能继续进行时，才推荐执行此步骤，本书每台虚拟机2G内存的实验环境中并不需要此步骤。使用如下命令设置：

$ kubectl taint nodes lab1 node-role.kubernetes.io/master-
node/lab1 untainted

3.添加Node节点

1）配置Docker和Kubelet开机启动。

在lab2、lab3分别执行如下命令：

#设置Docker开机启动
$ sudo systemctl enable docker.service
#设置Kubelet开机启动
$ sudo systemctl enable kubelet.service

2）加入到集群。

在lab2、lab3分别执行如下命令，此处的命令为初始化master完成时，输出的join命令，注意替换为你在实验初始化时的输出命令：

$ sudo kubeadm join 11.11.11.111:6443 --token xlivmy.jj73qkeobqoyfs0r --discovery-token-ca-cert-hash sha256:c6290480baaef64f3f1c6b6861de8685ca74d5a4a7c2703eace18a2e2117e87d
[preflight] running pre-flight checks
[discovery] Trying to connect to API Server "11.11.11.111:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://11.11.11.111:6443"
[discovery] Requesting info from "https://11.11.11.111:6443" again to validate TLS against the pinned public key
[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "11.11.11.111:6443"
[discovery] Successfully established connection with API Server "11.11.11.111:6443"
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.12" ConfigMap in the kube-system namespace
[kubelet] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[preflight] Activating the kubelet service
[tlsbootstrap] Waiting for the kubelet to perform the TLS Bootstrap...
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "lab2" as an annotation
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the master to see this node join the cluster.

3）查看集群中的节点。

在lab1执行如下命令：

$ kubectl get node
NAME         STATUS           ROLES          AGE           VERSION
lab1         NotReady         master         2m38s         v1.12.4
lab2         NotReady         <none>         12s           v1.12.4
lab3         NotReady         <none>         5s            v1.12.4

4.部署Kubernetes网络插件

使用Kubeadm创建Kubernetes集群时，如果没有部署网络插件，所有的节点状态都会是NotReady。当网络插件部署完成后，状态会更新为Ready，并会启动CoreDNS，用于集群中的服务发现。

以下步骤在lab1上操作。

1）下载yaml文件：

$ curl -s https://raw.githubusercontent.com/coreos/flannel/v0.10.0/
    Documentation/kube-flannel.yml -o kube-flannel.yml

2）修改配置文件。修改kube-flannel.yml文件如下位置的Network参数为"10.244.0.0/16"，此参数与初始化master节点时的配置文件中的podSubnet值保持一致：

net-conf.json: |
  {
    "Network": "10.244.0.0/16",
    "Backend": {
      "Type": "vxlan"
    }
  }

修改kube-flannel.yml文件中的image字段：

image: registry.cn-shanghai.aliyuncs.com/gcr-k8s/flannel:v0.10.0-amd64

修改如下位置的tolerations配置，详见： https://github.com/coreos/flannel/issues/1044

#修改前
hostNetwork: true
nodeSelector:
  beta.kubernetes.io/arch: amd64
tolerations:
- key: node-role.kubernetes.io/master
  operator: Exists
  effect: NoSchedule
#修改后
hostNetwork: true
nodeSelector:
  beta.kubernetes.io/arch: amd64
tolerations:
- operator: Exists
  effect: NoSchedule

在如下的位置添加指定kube-flannel容器启动时的网卡名称，此网卡用于flannel内部通信（详见 https://github.com/kubernetes/kubernetes/issues/39701 ）：

#修改前
containers:
- name: kube-flannel
  image: registry.cn-shanghai.aliyuncs.com/gcr-k8s/flannel:v0.10.0-amd64
  command:
  - /opt/bin/flanneld
  args:
  - --ip-masq
  - --kube-subnet-mgr
  resources:
    requests:
      cpu: "100m"
      memory: "50Mi"
    limits:
      cpu: "100m"
      memory: "50Mi"
#修改后
containers:
- name: kube-flannel
  image: registry.cn-shanghai.aliyuncs.com/gcr-k8s/flannel:v0.10.0-amd64
  command:
  - /opt/bin/flanneld
  args:
  - --ip-masq
  - --kube-subnet-mgr
  - --iface=eth1
  resources:
    requests:
      cpu: "100m"
      memory: "50Mi"
    limits:
      cpu: "100m"
      memory: "50Mi"

3）部署Flannel：

$ kubectl apply -f kube-flannel.yml
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.extensions/kube-flannel-ds created

4）查看Flannel状态。

在lab1执行如下命令：

$ kubectl get pod -n kube-system -o wide
NAME          READY   STATUS    RESTARTS   AGE     IP             NODE   NOMINATED NODE
coredns-6c66ffc55b-5tpjm      1/1  Running  0  9m12s  10.244.1.3    lab2  <none>
coredns-6c66ffc55b-mztsx      1/1  Running  0  9m12s  10.244.1.2    lab2  <none>
etcd-lab1                     1/1  Running  0  8m50s  11.11.11.111  lab1  <none>
kube-apiserver-lab1           1/1  Running  0  8m44s  11.11.11.111  lab1  <none>
kube-controller-manager-lab1  1/1  Running  0  8m55s  11.11.11.111  lab1  <none>
kube-flannel-ds-bkz59         1/1  Running  0  82s    11.11.11.111  lab1  <none>
kube-flannel-ds-dk466         1/1  Running  0  82s    11.11.11.113  lab3  <none>
kube-flannel-ds-gmh5l         1/1  Running  0  82s    11.11.11.112  lab2  <none>
kube-proxy-82mtm              1/1  Running  0  7m7s   11.11.11.112  lab2  <none>
kube-proxy-vtv89              1/1  Running  0  7m     11.11.11.113  lab3  <none>
kube-proxy-xw66f              1/1  Running  0  9m13s  11.11.11.111  lab1  <none>
kube-scheduler-lab1           1/1  Running  0  8m46s  11.11.11.111  lab1  <none>
$ kubectl get svc -n kube-system
NAME       TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE
kube-dns   ClusterIP   10.96.0.10   <none>        53/UDP,53/TCP   9m56s

5）查看集群节点状态。

在lab1执行如下命令，部署完成Flannel之后，集群中的节点全部变为Ready状态：

$ kubectl get node
NAME         STATUS          ROLES          AGE         VERSION
lab1         Ready           master         42m         v1.12.4
lab2         Ready           <none>         40m         v1.12.4
lab3         Ready           <none>         39m         v1.12.4

4.4 创建Kubernetes集群的步骤

4.4　创建Kubernetes集群的步骤