kubernetes集群安装部署

Kubernetes / 作者：

Kubernetes介绍

Kubernetes（简称k8s）是Google在2014年6月开源的一个容器集群管理系统，使用Go语言开发，用于管理云平台中多个主机上的容器化的应用，Kubernetes的目标是让部署容器化的应用简单并且高效,Kubernetes提供了资源调度、部署管理、服务发现、扩容缩容、监控，维护等一整套功能。

k8s集群安装

Master和Node都需要操作

  • 关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
  • 关闭selinux
sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
setenforce 0
  • 修改桥接的ipv4流量传递到iptables的链
cat << EOF > /etc/sysctl.conf
net.ipv4.ip_forward=1
net.bridge.bridge-nf-call-iptables=1
net.ipv4.neigh.default.gc_thresh1=4096
net.ipv4.neigh.default.gc_thresh2=6144
net.ipv4.neigh.default.gc_thresh3=8192
vm.swappiness=0
EOF
sysctl -p
  • 关闭swap分区
#临时关闭
swapoff -a 
#永久关闭，注释掉/etc/fstab里面swap的挂载
  • 修改完以上配置，重启系统
  • 安装docker
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum install docker-ce-18.09.9 -y
systemctl start docker
  • 配置镜像加速
cat << EOF > /etc/docker/daemon.json
{
"registry-mirrors": [ "https://8wcr35gm.mirror.aliyuncs.com"]
}
EOF
  • 启动docker、配置开机启动
systemctl restart docker
systemctl enable docker
  • 配置kubernetes阿里云yum源
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
  • 下载kubernetes镜像
#server节点
#拉取镜像
docker pull registry.aliyuncs.com/google_containers/kube-apiserver-amd64:v1.10.0
docker pull registry.aliyuncs.com/google_containers/kube-scheduler-amd64:v1.10.0
docker pull registry.aliyuncs.com/google_containers/kube-controller-manager-amd64:v1.10.0
docker pull registry.aliyuncs.com/google_containers/kube-proxy-amd64:v1.10.0
docker pull registry.aliyuncs.com/google_containers/k8s-dns-kube-dns-amd64:1.14.8
docker pull registry.aliyuncs.com/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.8
docker pull registry.aliyuncs.com/google_containers/k8s-dns-sidecar-amd64:1.14.8
docker pull registry.aliyuncs.com/google_containers/etcd-amd64:3.1.12
docker pull registry.aliyuncs.com/google_containers/pause-amd64:3.1
docker pull dockerofwj/flannel

#修改镜像名称，与k8s原服务需求镜像保持一致
docker tag registry.aliyuncs.com/google_containers/kube-apiserver-amd64:v1.10.0 k8s.gcr.io/kube-apiserver-amd64:v1.10.0
docker tag registry.aliyuncs.com/google_containers/kube-scheduler-amd64:v1.10.0 k8s.gcr.io/kube-scheduler-amd64:v1.10.0
docker tag registry.aliyuncs.com/google_containers/kube-controller-manager-amd64:v1.10.0 k8s.gcr.io/kube-controller-manager-amd64:v1.10.0
docker tag registry.aliyuncs.com/google_containers/kube-proxy-amd64:v1.10.0 k8s.gcr.io/kube-proxy-amd64:v1.10.0
docker tag registry.aliyuncs.com/google_containers/k8s-dns-kube-dns-amd64:1.14.8 k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.8
docker tag registry.aliyuncs.com/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.8 k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.8
docker tag registry.aliyuncs.com/google_containers/k8s-dns-sidecar-amd64:1.14.8 k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.8
docker tag registry.aliyuncs.com/google_containers/etcd-amd64:3.1.12 k8s.gcr.io/etcd-amd64:3.1.12
docker tag registry.aliyuncs.com/google_containers/pause-amd64:3.1 k8s.gcr.io/pause-amd64:3.1
docker tag dockerofwj/flannel:latest quay.io/coreos/flannel:v0.10.0-amd64

#删除原镜像
docker rmi registry.aliyuncs.com/google_containers/kube-apiserver-amd64:v1.10.0
docker rmi registry.aliyuncs.com/google_containers/kube-scheduler-amd64:v1.10.0
docker rmi registry.aliyuncs.com/google_containers/kube-controller-manager-amd64:v1.10.0
docker rmi registry.aliyuncs.com/google_containers/kube-proxy-amd64:v1.10.0
docker rmi registry.aliyuncs.com/google_containers/k8s-dns-kube-dns-amd64:1.14.8
docker rmi registry.aliyuncs.com/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.8
docker rmi registry.aliyuncs.com/google_containers/k8s-dns-sidecar-amd64:1.14.8
docker rmi registry.aliyuncs.com/google_containers/etcd-amd64:3.1.12
docker rmi registry.aliyuncs.com/google_containers/pause-amd64:3.1
docker rmi dockerofwj/flannel:latest

#node节点
#拉取镜像
docker pull registry.aliyuncs.com/google_containers/kube-proxy-amd64:v1.10.0
docker pull registry.aliyuncs.com/google_containers/pause-amd64:3.1
docker pull dockerofwj/flannel

#tag
docker tag registry.aliyuncs.com/google_containers/pause-amd64:3.1 k8s.gcr.io/pause-amd64:3.1
docker tag registry.aliyuncs.com/google_containers/kube-proxy-amd64:v1.10.0 k8s.gcr.io/kube-proxy-amd64:v1.10.0
docker tag dockerofwj/flannel:latest quay.io/coreos/flannel:v0.10.0-amd64

#删除原镜像
docker rmi dockerofwj/flannel:latest
docker rmi registry.aliyuncs.com/google_containers/kube-proxy-amd64:v1.10.0
docker rmi registry.aliyuncs.com/google_containers/pause-amd64:3.1
  • 安装kubelet、kubeadm、kubectl
yum install -y kubelet kubeadm kubectl
systemctl enable kubelet

在Master节点操作

  • 初始化kubernetes master节点：
#apiserver-advertise指定Master Api组件监听的ip地址，与其他地址通信的地址
#image-repository指定仓库
#service-cidr指定service网络的ip地址段
#pod-network-cidr容器使用的网络段
kubeadm init --apiserver-advertise-address=192.168.1.171 --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.16.1 --service-cidr=172.16.0.0/16 --pod-network-cidr=10.244.0.0/16
#如果token忘记了，可以通过以下方式查看
kubeadm token list
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
#token 24小时后会过期，重新创建即可
kubeadm token create
  • 使用kubectl工具
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
  • 安装Pod网络插件 flannerl ，在Master节点操作
#先从国内获取flannel镜像
docker pull registry.cn-shenzhen.aliyuncs.com/chinabm_k8s/flannel
docker tag registry.cn-shenzhen.aliyuncs.com/chinabm_k8s/flannel:v0.10.0-amd64 quay.io/coreos/flannel:v0.10.0-amd64
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

#flannel的yml文件可能会有更新，我们还是用旧版，需要替换一下
sed -i 's/v0.11.0/v0.10.0/g' kube-flannel.yml
kubectl apply -f kube-flannel.yml

#查看部署状态
kubectl get pods -n kube-system
#flannel搞失败了重装方法：
kubectl delete -f kube-flannel.yml
备注：是 flannel 的镜像拉取错误，去 Docker hub 上搜索镜像代替一下
docker pull jmgao1983/flannel:v0.10.0-amd64

#打标记，标记的版本和yaml文件内需保持一致
docker tag jmgao1983/flannel:v0.10.0-amd64 quay.io/coreos/flannel:v0.10.0-amd64
kubectl get pod -n kube-system

#删除有问题的，会自动重建
kubectl delete pod kube-flannel-ds-amd64-z4mvc -n kube-system
  • 在Node节点操作：
#此处为上面master初始化后输出的内容，复制到节点操作即可
kubeadm join 192.168.1.171:6443 --token ltqdrr.iugj0d7sknf0yige --discovery-token-ca-cert-hash sha256:3b5de4eafde2bb496dfa26fb2e96bda678ed5265432009bf0aa7a0af647551c1

#如果想让管理节点也当计算节点使用在管理节点桥如下命令
kubectl taint nodes server node-role.kubernetes.io/master-
  • 安装dashboard
下载dashboard yaml：
wget https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml
修改为阿里源
vim kubernetes-dashboard.yaml
......
 containers:
      - name: kubernetes-dashboard
        #image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1
        image: registry.cn-hangzhou.aliyuncs.com/google_containers/kubernetes-dashboard-amd64:v1.10.1
        ports:
......
添加端口映射（web管理端口）：
vim kubernetes-dashboard.yaml 
# ------------------- Dashboard Service ------------------- #

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  type: NodePort       #增加type: NodePort
  ports:
    - port: 443
      targetPort: 8443
      nodePort: 32222   #增加nodePort: 32222
  selector:
    k8s-app: kubernetes-dashboard
------------------
安装dashboard
kubectl apply -f kubernetes-dashboard.yaml
kubectl create serviceaccount dashboard-admin -n kube-system
kubectl create clusterrolebinding dashborad-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin
获取web登录token
kubectl get secret -n kube-system |grep dashboard-admin
kubectl describe  secret -n dashboard-admin-token-rjknr -n kube-system | grep -A10 dashboard-admin
访问页面
https://ip:32222
备注：
使用chrome 版本 76.0.3809.100 会报证书问题，无法忽略
换firefox正常
Chrome浏览器无法访问K8S dashboard问题处理
常用命令
#映射外部端口 8080映射为外网18080
kubectl expose deployment tomcat001 --port=8080 --target-port=18080 --external-ip=192.168.1.171

启动mysql5.7

apiVersion: v1
kind: ReplicationController
metadata: 
  name: mysql
spec:
  replicas: 1
  selector:
    app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql
        image: centos/mysql-57-centos7
        ports: 
        - containerPort: 3306
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: "123456"

创建service

apiVersion: v1
kind: Service
metadata: 
  name: mysql1
spec:
  type: NodePort
  ports:
    - port: 3306
      nodePort: 30306
      targetPort: 3306
  selector: 
    app: mysql

添加阿里私有镜像仓库秘钥

kubectl create secret docker-registry alisecret --docker-server=registry.cn-shenzhen.aliyuncs.com --docker-username=USERNAME --docker-password=PASSWORD --docker-email=1139905088@qq.com

构建pod的时候需在containers并列级别加如下参数

imagePullSecrets:
- name: alisecret

发表评论