实验环境说明
基于mac m1芯片架构，虚拟机使用 vmware, 基于vagrant(保证节点网络通信正常，及可以访问外网,避免配置多网卡对小白不友好)进行部署k8s，gitlab,harbor等运维组件.
Vagrantfile 如下，box 自己去下，记得是要支持vmware 和 arm64芯片的
- k8s-01 master 节点
- k8s-02 node1节点
- k8s-03 node2节点
- k8s-04 公共服务节点如gitlab服务等
1# -*- mode: ruby -*-
2# vi: set ft=ruby :
3
4Vagrant.configure("2") do |config|
5
6 config.vm.define "k8s-01" do |master|
7 master.vm.box = "ubuntu2204"
8 master.vm.provider "vagrant-vmware-desktop" do |vb|
9 vb.memory = 4096
10 vb.cpus = 2
11 end
12 master.vm.network "private_network", ip: "192.168.56.80"
13 master.vm.synced_folder "/Users/edgar/code", "/home/www"
14 master.ssh.insert_key = false
15 master.vm.hostname = "master"
16 end
17
18 config.vm.define "k8s-02" do |node1|
19 node1.vm.box = "ubuntu2204"
20 node1.vm.provider "vagrant-vmware-desktop" do |vb|
21 vb.memory = 2048
22 vb.cpus = 2
23 end
24 node1.vm.network "private_network", ip: "192.168.56.81"
25 node1.vm.synced_folder "/Users/edgar/code", "/home/www"
26 node1.ssh.insert_key = false
27 node1.vm.hostname = "node1"
28 end
29
30 config.vm.define "k8s-03" do |node2|
31 node2.vm.box = "ubuntu2204"
32 node2.vm.provider "vagrant-vmware-desktop" do |vb|
33 vb.memory = 2048
34 vb.cpus = 2
35 end
36 node2.vm.network "private_network", ip: "192.168.56.82"
37 node2.vm.synced_folder "/Users/edgar/code", "/home/www"
38 node2.ssh.insert_key = false
39 node2.vm.hostname = "node2"
40 end
41
42 config.vm.define "k8s-04" do |node3|
43 node3.vm.box = "ubuntu2204"
44 node3.vm.provider "vagrant-vmware-desktop" do |vb|
45 vb.memory = 2048
46 vb.cpus = 2
47 end
48 node3.vm.network "private_network", ip: "192.168.56.83"
49 node3.vm.synced_folder "/Users/edgar/code", "/home/www"
50 node3.ssh.insert_key = false
51 node3.vm.hostname = "node3"
52 end
53end
gitlab、gitlab-runner安装
gitlab安装
注意镜像架构
1docker run --detach --privileged=true --hostname 192.168.56.83 --publish 8881:443 --publish 80:80 --publish 8883:22 --name gitlab --restart always --volume `pwd`/config:/etc/gitlab --volume `pwd`/logs:/var/log/gitlab --volume `pwd`/data:/var/opt/gitlab yrzr/gitlab-ce-arm64v8
gitlab-runner安装
二进制安装
1# Download the binary for your system
2sudo curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-arm64
3
4# Give it permission to execute
5sudo chmod +x /usr/local/bin/gitlab-runner
6
7# Create a GitLab Runner user
8sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
9
10# Install and run as a service
11sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
12# Runtime platform arch=arm64 os=linux pid=3745 revision=81ab07f6 version=16.10.0
13sudo gitlab-runner start
14
15# sudo gitlab-runner stop
共享 runner 创建
创建项目 group ： golang ，为 group 创建一个共享 runner，那么这个 group 下的所有项目都可以使用这个 runner。
当然你也可以为项目单独创建一个 runner。 那么这个 runner 只给此项目服务。
1sudo gitlab-runner register --url http://192.168.56.83/ --registration-token 4K6NqaPqjsJLy9ZEHrGh
harbor 镜像仓库准备
注释掉 http 段，端口改成 8080
1apt install docker-compose
2wget https://github.com/wise2c-devops/build-harbor-aarch64/releases/download/v2.10.1/harbor-offline-installer-aarch64-v2.10.1.tgz
3tar -zxvf harbor-offline-installer-aarch64-v2.10.1.tgz
4cd harbor
5cp harbor.yml.tmpl harbor.yml
6# 注释掉 https，然后改端口8080
7./install
创建一个公开的项目golang，先使用最简单的模式，先快速把案例跑通。再追求细节。
docker配置文件增加私有仓库支持 /etc/docker/daemon.json
1{
2 "exec-opts": ["native.cgroupdriver=systemd"],
3 "log-driver": "json-file",
4 "log-opts": { "max-size": "100m" },
5 "storage-driver": "overlay2",
6 "registry-mirrors": ["https://tdiin4cc.mirror.aliyuncs.com","http://hub-mirror.c.163.com"]
7 "insecure-registries": ["192.168.56.83:8080"]
8}
1systemctl daemon-reload
2systemctl restart docker
k8s集群安装
参考这个(mac arm芯片) https://blog.csdn.net/zhangzhaokun/article/details/131452979
https://www.crblog.cc/cloud-native/sre-k8s-install.html
这里只列出一些命令，还是要参考上面的连接。
1sudo apt-get install -y docker.io
2sudo apt-get update
3sudo apt-get install -y docker.io
4sudo service docker start
5sudo usermod -aG docker ${USER}
6sudo apt-get install net-tools
修改源
1vagrant@master:~$ cat /etc/apt/sources.list
2deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ jammy main restricted universe multiverse
3deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ jammy-updates main restricted universe multiverse
4deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ jammy-backports main restricted universe multiverse
5deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ jammy-security main restricted universe multiverse
6
7echo "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
8sudo apt-get update
9sudo apt install -y kubeadm=1.23.3-00 kubelet=1.23.3-00 kubectl=1.23.3-00
10kubeadm version
11sudo apt-mark hold kubeadm kubelet kubectl
12
13kubeadm config images list --kubernetes-version v1.23.3
在 master 节点上 安装 flannel 网络组件，其他节点不装。
https://github.com/chronolaw/k8s_study/blob/master/admin/flannel.yml
业务代码构建为容器镜像
go demo 代码如下：
1package main
2
3import (
4 "github.com/gin-gonic/gin"
5 "net/http"
6)
7
8func main() {
9 router := gin.Default()
10 router.GET("/", func(c *gin.Context) {
11 c.String(http.StatusOK, "Hello World")
12 })
13 router.Run(":8000")
14}
dockerfile
1FROM golang:1.21-alpine AS build
2ENV CGO_ENABLED=0 GOOS=linux GO111MODULE=on GOPROXY=https://goproxy.cn/,direct TZ=Asia/Shanghai
3RUN set -eux && sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories
4RUN apk update && apk add --no-cache tzdata git
5
6WORKDIR /app
7
8COPY ./go.mod ./
9COPY ./go.sum ./
10
11RUN go mod tidy
12
13COPY . ./
14
15RUN go build -o godemo main.go && go clean -cache
16
17FROM alpine:3.15.0
18RUN set -eux && sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories
19RUN apk update && apk add --no-cache tzdata
20ENV TZ "Asia/Shanghai"
21WORKDIR /app
22COPY --from=build /app/godemo /app/godemo
23RUN chmod +x /app/godemo
24
25EXPOSE 8000
26
27CMD ["/app/godemo"]
1docker run -d -p 8080:8000 --name godemo godemo
浏览器访问localhost:8080