目录
一、实验
1.环境
（1）主机
表1 主机
| 主机 | 架构 | 版本 | IP | 备注 |
| master | K8S master节点 | 1.29.0 | 192.168.204.8 | |
| node1 | K8S node节点 | 1.29.0 | 192.168.204.9 | |
| node2 | K8S node节点 | 1.29.0 | 192.168.204.10 | 已部署Kuboard |
（2）master节点查看集群
1）查看node
kubectl get node
2)查看node详细信息
kubectl get node -o wide
（3）查看pod
[root@master ~]# kubectl get pod -A
(4) 访问Kuboard
http://192.168.204.10:30080/kuboard/cluster查看节点
（5）查看docker版本
docker -v
master节点
node1节点
node2节点
2.Linux 部署docker compose
(1) 查阅
https://github.com/docker/compose/releases最新版为v2.27.0
(2) 下载
https://github.com/docker/compose/releases/download/v2.27.0/docker-compose-linux-x86_64(3) 查看
[root@master ~]# ls | grep docker
（4）移动并更名为docker-compose
[root@master ~]# mv docker-compose-linux-x86_64 /usr/bin/docker-compose
（5）为二进制文件添加可执行权限
[root@master ~]# chmod +x /usr/bin/docker-compose
（6）查看docker-compse版本
[root@master ~]# docker-compose version
3.证书秘钥配置
（1）确认openssl
[root@master harbor]# rpm -q openssl
(2)创建文件夹用于存放生成的证书秘钥文件
[root@master ~]# mkdir -p /harbor/certs/
(3) 生成私钥
[root@master ~]# openssl genrsa -out ca.key 4096
（4）证书秘钥与IP绑定
openssl req -x509 -new -nodes -sha512 -days 3650 \
-subj "/C=CN/ST=Nanjing/L=Nanjing/O=example/OU=Personal/CN=192.168.204.8" \
-key ca.key \
-out ca.crt
（5）移动证书位置并查看
[root@master ~]# mv ca* /harbor/certs
[root@master ~]# ls /harbor/certs/
4.K8S 1.29版本 部署Harbor
(1) 查阅
https://github.com/goharbor/harbor/releases最新版为v2.10.2
(2) 下载
https://github.com/goharbor/harbor/releases/download/v2.10.2/harbor-offline-installer-v2.10.2.tgz（3）查看
[root@master ~]# ls | grep harbor
（4）解压
[root@master ~]# tar xf harbor-offline-installer-v2.10.2.tgz
（6）进入目录并查看
[root@master ~]# cd harbor/
[root@master harbor]# ls
（7） 修改名称
[root@master harbor]# mv harbor.yml.tmpl harbor.yml
（8）监听端口
8800与8443没有被占用
[root@master harbor]# netstat -antlp | grep 8800
[root@master harbor]# netstat -antlp | grep 8443
（9）修改配置文件
[root@master harbor]# vim harbor.yml
hostname域名修改为要安装harbor的服务器的ip
http端口修改为8800
https端口修改为8443
certificate: 证书路径修改为/harbor/certs/ca.crt
private_key: 秘钥路径修改为/harbor/certs/ca.key
harbor_admin_password: 默认Harbor密码，不修改，就用默认的Harbor12345
data_value: 修改为/data/harbor
修改前：
修改后：
（10）Docker Hub查看镜像
docker pull goharbor/prepare:v2.10.2
(11)拉取镜像
node2
[root@node2 ~]# docker pull goharbor/prepare:v2.10.2
(12) 导出镜像
[root@node2 ~]# docker save -o prepare.tar goharbor/prepare:v2.10.2
(13）复制Docker镜像到master节点
[root@node2 ~]# scp prepare.tar root@master:~
(14）master节点导入Docker镜像
[root@master ~]# docker load -i prepare.tar
（15）执行预备脚本
[root@master harbor]# ./prepare
(16) 执行安装脚本
[root@master harbor]# ./install.sh
[Step 0]: checking if docker is installed ...
Note: docker version: 24.0.7
[Step 1]: checking docker-compose is installed ...
Note: Docker Compose version v2.21.0
[Step 2]: loading Harbor images ...
a8840e4ae316: Loading layer [==================================================>] 21.63MB/21.63MB
515f0ee642d6: Loading layer [==================================================>] 173.8MB/173.8MB
7e1a15fa7f7c: Loading layer [==================================================>] 25.5MB/25.5MB
4cd39a5aa67d: Loading layer [==================================================>] 18.27MB/18.27MB
3daae6a42996: Loading layer [==================================================>] 5.12kB/5.12kB
e23a6fbc3acf: Loading layer [==================================================>] 6.144kB/6.144kB
87a67eef40a6: Loading layer [==================================================>] 3.072kB/3.072kB
5439373f0cf5: Loading layer [==================================================>] 2.048kB/2.048kB
44c50cb14a57: Loading layer [==================================================>] 2.56kB/2.56kB
e343431b8ac0: Loading layer [==================================================>] 7.68kB/7.68kB
Loaded image: goharbor/harbor-db:v2.10.2
a7437080fdff: Loading layer [==================================================>] 17.16MB/17.16MB
89c54c70338a: Loading layer [==================================================>] 3.584kB/3.584kB
927d190fe457: Loading layer [==================================================>] 2.56kB/2.56kB
dab52f006605: Loading layer [==================================================>] 44.92MB/44.92MB
abc8dcfeeb98: Loading layer [==================================================>] 45.71MB/45.71MB
Loaded image: goharbor/harbor-jobservice:v2.10.2
9343d1011e8e: Loading layer [==================================================>] 21.63MB/21.63MB
d8aa785c45d8: Loading layer [==================================================>] 110.6MB/110.6MB
af30438adf19: Loading layer [==================================================>] 3.072kB/3.072kB
046baf104eef: Loading layer [==================================================>] 59.9kB/59.9kB
b4bf035573f1: Loading layer [==================================================>] 61.95kB/61.95kB
Loaded image: goharbor/redis-photon:v2.10.2
1a53aa673223: Loading layer [==================================================>] 14.15MB/14.15MB
e3ef818740f3: Loading layer [==================================================>] 4.096kB/4.096kB
f51244ac7c03: Loading layer [==================================================>] 3.072kB/3.072kB
971632584e63: Loading layer [==================================================>] 17.4MB/17.4MB
0cfb29c07835: Loading layer [==================================================>] 18.19MB/18.19MB
Loaded image: goharbor/registry-photon:v2.10.2
9fb8de59ccbd: Loading layer [==================================================>] 130.9MB/130.9MB
b0a99212add7: Loading layer [==================================================>] 3.584kB/3.584kB
c63a00cfdba3: Loading layer [==================================================>] 3.072kB/3.072kB
ea1dbece5b3a: Loading layer [==================================================>] 2.56kB/2.56kB
3c735fb3805e: Loading layer [==================================================>] 3.072kB/3.072kB
1b257c0b8709: Loading layer [==================================================>] 3.584kB/3.584kB
6bb174e7db78: Loading layer [==================================================>] 20.48kB/20.48kB
Loaded image: goharbor/harbor-log:v2.10.2
ea9d3f5fbb0c: Loading layer [==================================================>] 121.2MB/121.2MB
b88bdbd268ea: Loading layer [==================================================>] 6.532MB/6.532MB
a6a0154f827a: Loading layer [==================================================>] 246.8kB/246.8kB
664d5baa31ef: Loading layer [==================================================>] 1.477MB/1.477MB
Loaded image: goharbor/harbor-portal:v2.10.2
fe6dfc816265: Loading layer [==================================================>] 17.16MB/17.16MB
c8d1d3bbdf04: Loading layer [==================================================>] 3.584kB/3.584kB
a95b761bfdbe: Loading layer [==================================================>] 2.56kB/2.56kB
c3c092020a00: Loading layer [==================================================>] 58.96MB/58.96MB
75c8aba5c8aa: Loading layer [==================================================>] 5.632kB/5.632kB
84bdcacb7f37: Loading layer [==================================================>] 123.4kB/123.4kB
c80d23d5cead: Loading layer [==================================================>] 80.38kB/80.38kB
fb9a83422043: Loading layer [==================================================>] 59.95MB/59.95MB
17416c18f4d3: Loading layer [==================================================>] 2.56kB/2.56kB
Loaded image: goharbor/harbor-core:v2.10.2
6a65c62b44a3: Loading layer [==================================================>] 14.15MB/14.15MB
50d4a017d046: Loading layer [==================================================>] 4.096kB/4.096kB
b00285e03979: Loading layer [==================================================>] 17.4MB/17.4MB
bd20c4f45bc4: Loading layer [==================================================>] 3.072kB/3.072kB
cdef8cae2c49: Loading layer [==================================================>] 32.77MB/32.77MB
8c8e4274f70f: Loading layer [==================================================>] 50.95MB/50.95MB
Loaded image: goharbor/harbor-registryctl:v2.10.2
acbecb34a764: Loading layer [==================================================>] 17.16MB/17.16MB
478163dbc994: Loading layer [==================================================>] 27.56MB/27.56MB
4ec3465523a2: Loading layer [==================================================>] 4.608kB/4.608kB
07fb78bc4c52: Loading layer [==================================================>] 28.35MB/28.35MB
Loaded image: goharbor/harbor-exporter:v2.10.2
113051e862a0: Loading layer [==================================================>] 121.2MB/121.2MB
Loaded image: goharbor/nginx-photon:v2.10.2
2716f555462f: Loading layer [==================================================>] 14.67MB/14.67MB
d4471d1781eb: Loading layer [==================================================>] 4.096kB/4.096kB
2b9cfa166f1f: Loading layer [==================================================>] 3.072kB/3.072kB
4943bbb50494: Loading layer [==================================================>] 209.9MB/209.9MB
1f70ad7459fb: Loading layer [==================================================>] 17.45MB/17.45MB
c6de541f3b98: Loading layer [==================================================>] 228.1MB/228.1MB
Loaded image: goharbor/trivy-adapter-photon:v2.10.2
Loaded image: goharbor/prepare:v2.10.2
[Step 3]: preparing environment ...
[Step 4]: preparing harbor configs ...
prepare base dir is set to /root/harbor
Clearing the configuration file: /config/portal/nginx.conf
Clearing the configuration file: /config/log/logrotate.conf
Clearing the configuration file: /config/log/rsyslog_docker.conf
Clearing the configuration file: /config/nginx/nginx.conf
Clearing the configuration file: /config/core/env
Clearing the configuration file: /config/core/app.conf
Clearing the configuration file: /config/registry/passwd
Clearing the configuration file: /config/registry/config.yml
Clearing the configuration file: /config/registryctl/env
Clearing the configuration file: /config/registryctl/config.yml
Clearing the configuration file: /config/db/env
Clearing the configuration file: /config/jobservice/env
Clearing the configuration file: /config/jobservice/config.yml
Generated configuration file: /config/portal/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
loaded secret from file: /data/secret/keys/secretkey
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
Note: stopping existing Harbor instance ...
[Step 5]: starting Harbor ...
[+] Running 10/10
✔ Network harbor_harbor Created 0.1s
✔ Container harbor-log Started 0.0s
✔ Container registry Started 0.1s
✔ Container registryctl Started 0.1s
✔ Container harbor-portal Started 0.1s
✔ Container harbor-db Started 0.1s
✔ Container redis Started 0.1s
✔ Container harbor-core Started 0.0s
✔ Container nginx Started 0.0s
✔ Container harbor-jobservice Started 0.0s
✔ ----Harbor has been installed and started successfully.----
(17) 查看harbor相关容器
[root@master harbor]# docker ps | grep goharbor
（18）docker-compose命令显示其管理的harbor容器
[root@master harbor]# docker-compose -f docker-compose.yml ps
（19）访问harbor
https://192.168.204.8:8443/选高级
继续访问
弹出界面
（20）登录
账号：admin
密码：Harbor12345
成功进入系统
5. K8S 1.29版本 使用Harbor
(1) 创建私有项目
(2) 查看
目前项目为空
(3) 创建公开项目
（4）再次查看
目前项目为空
（5）Docker配置Harbor（所有节点）
1）修改
vim /etc/docker/daemon.json
……
"insecure-registries": ["https://192.168.204.8:8443"]
……
2）重新加载配置
systemctl daemon-reload
3）重启服务
systemctl restart docker
systemctl restart cri-docker
修改前：
修改后：
这里以node1节点为例
[root@node1 ~]# systemctl status cri-docker
（6）docker 登录harbor并推送镜像到公开项目
node1节点
[root@node1 ~]# docker login 192.168.204.8:8443 -u admin -p Harbor12345
拉取镜像
打标签
[root@node1 ~]# docker tag alpine:latest 192.168.204.8:8443/devops02/alpine:20240502
推送镜像到公开项目
[root@node1 ~]# docker push 192.168.204.8:8443/devops02/alpine:20240502
查看
删除本地镜像
[root@node1 ~]# docker rmi -f 192.168.204.8:8443/devops02/alpine:20240502
拉取harbor仓库的镜像
[root@node1 ~]# docker pull 192.168.204.8:8443/devops02/alpine:20240502
（7）docker 登录harbor并推送镜像到私有项目
打标签
[root@node1 ~]# docker tag alpine:latest 192.168.204.8:8443/devops01/alpine:20240502
推送镜像到私有项目
[root@node1 ~]# docker push 192.168.204.8:8443/devops01/alpine:20240502
查看
删除本地镜像
[root@node1 ~]# docker rmi -f 192.168.204.8:8443/devops01/alpine:20240502
拉取harbor仓库的镜像
[root@node1 ~]# docker pull 192.168.204.8:8443/devops01/alpine:20240502
(8) 登出账号
[root@node1 ~]# docker logout 192.168.204.8
（9）其他方式的harbor部署
可以参考本人博客：
Docker容器与虚拟化技术：Harbor私有仓库部署与迁移-CSDN博客
云原生Kubernetes：K8S集群使用带凭证的harbor仓库-CSDN博客
二、问题
1.docker 登录harbor失败
（1）报错
Error response from daemon: Get "https://192.168.204.8:8443/v2/": tls: failed to verify certificate: x509: certificate relies on legacy Common Name field, use SANs instead
（2）原因分析
未重启docker。
（3）解决方法
重启docker:
[root@node1 ~]# systemctl restart docker
成功：
