一、實驗環(huán)境準(zhǔn)備
- 準(zhǔn)備VMware安裝三臺CentOS 8的虛擬機，啟動且相互ping通網(wǎng)絡(luò)，ip分別為192.168.50.12.208、192.168.50.12.127、192.168.50.126.后續(xù)各個host簡稱208、126、127；
- 以208為主安裝Docker Machine對127和126進行Docker的安裝和部署，需要利用ssh-copy-id 實現(xiàn)在127和126的免密碼登錄，免密登錄參考之前的文章；
- (可選)在127 126上，國內(nèi)為yum添加阿里云鏡像避免安裝下載docker的失敗。
二、Docker Machine
在208上安裝Docker Machine，不同平臺有不同的安裝命令參考官方文檔
[woods@localhost ~]$ base=https://github.com/docker/machine/releases/download/v0.16.0 &&
curl -L $base/docker-machine-$(uname -s)-$(uname -m) >/tmp/docker-machine &&
sudo mv /tmp/docker-machine /usr/local/bin/docker-machine &&
chmod +x /usr/local/bin/docker-machine
驗證是否安裝完成
[woods@localhost /]$ docker-machine -v
docker-machine version 0.16.0, build 702c267f
三、使用Docker Machine安裝Docker
在208 host上安裝Docker Machine，不同平臺有不同的安裝命令參考官方文檔，這里是普通的Linux系統(tǒng)所以--drive選項用generic 而--generic-ip-address是指定安裝host的ip地址
[woods@localhost ~]$ docker-machine create --driver generic --generic-ip-address=192.168.50.127 host2
Running pre-create checks...
Creating machine...
(host2) No SSH key specified. Assuming an existing key at the default location.
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with centos...
發(fā)現(xiàn)卡住了（也可能顯示install docker 失敗等信息），此時用docker-machine ls發(fā)現(xiàn)host2已經(jīng)存在，先用docker-machine rm host2刪除了，再執(zhí)行docker-machine -D create安裝，-D是顯示日志。然后
1.從日志里面Cannot download Packages/docker-ce-cli-19.03.9-3.el7.x86_64.rpm: All mirrors were tried下載失敗，備好梯子，docker-machine rm host2后從新執(zhí)行docker-machine -D create安裝
……
if ! type docker; then curl -sSL https://get.docker.com | sh -; fi
SSH cmd err, output: exit status 1: bash: 第 0 行:type: docker: 未找到
# Executing docker install script, commit: 26ff363bcf3b3f5a00498ac43694bf1c7d9ce16c
+ sh -c 'yum install -y -q yum-utils'
+ sh -c 'yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo'
添加倉庫自：https://download.docker.com/linux/centos/docker-ce.repo
+ '[' stable '!=' stable ']'
+ sh -c 'yum makecache'
CentOS-8 - AppStream 8.1 kB/s | 4.3 kB 00:00
CentOS-8 - Base 2.2 kB/s | 3.9 kB 00:01
CentOS-8 - Extras 540 B/s | 1.5 kB 00:02
Docker CE Stable - x86_64 1.1 kB/s | 3.5 kB 00:03
元數(shù)據(jù)緩存已建立。
+ '[' -n '' ']'
+ sh -c 'yum install -y -q docker-ce'
錯誤：下載軟件包出錯
Cannot download Packages/docker-ce-cli-19.03.9-3.el7.x86_64.rpm: All mirrors were tried
Error creating machine: Error running provisioning: error installing docker:
notifying bugsnag: [Error creating machine: Error running provisioning: error installing docker: ]
……
- 從日志里面發(fā)現(xiàn)
package docker-ce-3:19.03.9-3.el7.x86_64 requires containerd.io >= 1.2.2-3, but none of the providers can be installed或者錯誤：事務(wù)檢查錯誤： file /usr/share/man/man1/docker-attach.1.gz from install of docker-ce-cli-1:19.03.8-3.el7.x86_64 conflicts with file from package podman-manpages-1.4.2-5.module_el8.1.0+237+63e26edc.noarch，用ssh登陸host2參考文章修復(fù)問題再安裝
……
(host2) Calling .GetSSHHostname
(host2) Calling .GetSSHPort
(host2) Calling .GetSSHKeyPath
(host2) Calling .GetSSHUsername
Using SSH client type: external
&{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@192.168.50.127 -p 22] /usr/bin/ssh <nil>}
About to run SSH command:
if ! type docker; then curl -sSL https://get.docker.com | sh -; fi
SSH cmd err, output: exit status 1: bash: 第 0 行:type: docker: 未找到
# Executing docker install script, commit: 26ff363bcf3b3f5a00498ac43694bf1c7d9ce16c
+ sh -c 'yum install -y -q yum-utils'
+ sh -c 'yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo'
添加倉庫自：https://download.docker.com/linux/centos/docker-ce.repo
+ '[' stable '!=' stable ']'
+ sh -c 'yum makecache'
CentOS-8 - AppStream 2.6 kB/s | 4.3 kB 00:01
CentOS-8 - Base 1.9 kB/s | 3.9 kB 00:02
CentOS-8 - Extras 2.6 kB/s | 1.5 kB 00:00
Docker CE Stable - x86_64 1.6 kB/s | 3.5 kB 00:02
元數(shù)據(jù)緩存已建立。
+ '[' -n '' ']'
+ sh -c 'yum install -y -q docker-ce'
錯誤：
問題: package docker-ce-3:19.03.9-3.el7.x86_64 requires containerd.io >= 1.2.2-3, but none of the providers can be installed
- cannot install the best candidate for the job
- package containerd.io-1.2.10-3.2.el7.x86_64 is excluded
- package containerd.io-1.2.13-3.1.el7.x86_64 is excluded
- package containerd.io-1.2.13-3.2.el7.x86_64 is excluded
- package containerd.io-1.2.2-3.3.el7.x86_64 is excluded
- package containerd.io-1.2.2-3.el7.x86_64 is excluded
- package containerd.io-1.2.4-3.1.el7.x86_64 is excluded
- package containerd.io-1.2.5-3.1.el7.x86_64 is excluded
- package containerd.io-1.2.6-3.3.el7.x86_64 is excluded
Error creating machine: Error running provisioning: error installing docker:
notifying bugsnag: [Error creating machine: Error running provisioning: error installing docker: ]
- TLS認(rèn)證失敗,
Error checking and/or regenerating the certs，但此時host上已經(jīng)成功安裝docker及配置好遠程連接，結(jié)果排查很久發(fā)現(xiàn)是host上的端口沒有放通導(dǎo)致，參考文章放通后正常
……
SSH cmd err, output: <nil>: Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:5355 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN
tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN
tcp6 0 0 :::5355 :::* LISTEN
tcp6 0 0 :::111 :::* LISTEN
tcp6 0 0 :::22 :::* LISTEN
tcp6 0 0 ::1:631 :::* LISTEN
tcp6 0 0 :::2376 :::* LISTEN
Checking connection to Docker...
(node3) Calling .GetURL
Reading CA certificate from /home/woods/.docker/machine/certs/ca.pem
Reading client certificate from /home/woods/.docker/machine/certs/cert.pem
Reading client key from /home/woods/.docker/machine/certs/key.pem
Error creating machine: Error checking the host: Error checking and/or regenerating the certs: There was an error validating certificates for host "192.168.137.3:2376": dial tcp 192.168.137.3:2376: connect: no route to host
You can attempt to regenerate them using 'docker-machine regenerate-certs [name]'.
Be advised that this will trigger a Docker daemon restart which might stop running containers.
notifying bugsnag: [Error creating machine: Error checking the host: Error checking and/or regenerating the certs: There was an error validating certificates for host "192.168.137.3:2376": dial tcp 192.168.137.3:2376: connect: no route to host
You can attempt to regenerate them using 'docker-machine regenerate-certs [name]'.
Be advised that this will trigger a Docker daemon restart which might stop running containers.
]
host上的端口放通前后，docker-machine ls 或出現(xiàn)connect: no route to host
#放通前
[woods@localhost node3]$ docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
node1 - generic Running tcp://192.168.137.2:2376 Unknown Unable to query docker version: Get https://192.168.137.2:2376/v1.15/version: dial tcp 192.168.137.2:2376: connect: no route to host
node3 - generic Running tcp://192.168.137.3:2376 v19.03.9
#放通后
[woods@localhost node3]$ docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
node1 - generic Running tcp://192.168.137.2:2376 v19.03.9
node3 - generic Running tcp://192.168.137.3:2376 v19.03.9
四、題外話
1. 安裝后host上的docker配置在/etc/systemd/system/docker.service.d/10-machine.conf
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: dis>
Drop-In: /etc/systemd/system/docker.service.d
└─10-machine.conf
Active: active (running) since Thu 2020-05-21 12:10:31 EDT; 6min ago
Docs: https://docs.docker.com
Main PID: 11243 (dockerd)
Tasks: 8
Memory: 45.8M
CGroup: /system.slice/docker.service
└─11243 /usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.so>
$ cat 10-machine.conf
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver overlay2 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=generic
Environment=
- -H tcp://0.0.0.0:2376 使 docker daemon 接受遠程連接。
- --tls* 對遠程連接啟用安全認(rèn)證和加密。
2. machine上對host的配置在~/.docker/machine/machines/目錄下其中node1是hostname，里面是證書及配置文件
[woods@localhost /]$ ls ~/.docker/machine/machines/
node1 node3
[woods@localhost machines]$ cd node3/
[woods@localhost node3]$ ls
ca.pem cert.pem config.json id_rsa id_rsa.pub key.pem server-key.pem server.pem
[woods@localhost node3]$ cat config.json
{
"ConfigVersion": 3,
"Driver": {
"IPAddress": "192.168.137.3",
"MachineName": "node3",
"SSHUser": "root",
"SSHPort": 22,
"SSHKeyPath": "/home/woods/.docker/machine/machines/node3/id_rsa",
"StorePath": "/home/woods/.docker/machine",
"SwarmMaster": false,
"SwarmHost": "",
"SwarmDiscovery": "",
"EnginePort": 2376,
"SSHKey": "/home/woods/.ssh/id_rsa"
},
"DriverName": "generic",
"HostOptions": {
"Driver": "",
"Memory": 0,
"Disk": 0,
"EngineOptions": {
"ArbitraryFlags": [],
"Dns": null,
"GraphDir": "",
"Env": [],
"Ipv6": false,
"InsecureRegistry": [],
"Labels": [],
"LogLevel": "",
"StorageDriver": "",
"SelinuxEnabled": false,
"TlsVerify": true,
"RegistryMirror": [],
"InstallURL": "https://get.docker.com"
},
"SwarmOptions": {
"IsSwarm": false,
"Address": "",
"Discovery": "",
"Agent": false,
"Master": false,
"Host": "tcp://0.0.0.0:3376",
"Image": "swarm:latest",
"Strategy": "spread",
"Heartbeat": 0,
"Overcommit": 0,
"ArbitraryFlags": [],
"ArbitraryJoinFlags": [],
"Env": null,
"IsExperimental": false
},
"AuthOptions": {
"CertDir": "/home/woods/.docker/machine/certs",
"CaCertPath": "/home/woods/.docker/machine/certs/ca.pem",
"CaPrivateKeyPath": "/home/woods/.docker/machine/certs/ca-key.pem",
"CaCertRemotePath": "",
"ServerCertPath": "/home/woods/.docker/machine/machines/node3/server.pem",
"ServerKeyPath": "/home/woods/.docker/machine/machines/node3/server-key.pem",
"ClientKeyPath": "/home/woods/.docker/machine/certs/key.pem",
"ServerCertRemotePath": "",
"ServerKeyRemotePath": "",
"ClientCertPath": "/home/woods/.docker/machine/certs/cert.pem",
"ServerCertSANs": [],
"StorePath": "/home/woods/.docker/machine/machines/node3"
}
},
"Name": "node3"
}
