1、環(huán)境
IP 服務 主機名
192.168.3.204 Swarm管理節(jié)點 controler
192.168.3.202 Swarm的node節(jié)點 node2
192.168.3.201 Swarm的node節(jié)點 node1
2、配置docker yum源(三臺機器)
cd /etc/yum.repos.d/
touch docker.repo
vim docker.repo
添加內容如下:*
[dockerrepo]
name=Docker Repository
baseurl=https://yum.dockerproject.org/repo/main/centos/$releasever/
enabled=1
gpgcheck=1
gpgkey=https://yum.dockerproject.org/gpg
:wq保存退出
3、配置hosts表(三臺機器)
vim /etc/hosts
添加內容如下:
192.168.3.204 controller
192.168.3.201 node1
192.168.3.202 node2
4、配置manager節(jié)點(192.168.3.204)
4.1、刪除舊版本的環(huán)境
yum remove docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-selinux docker-engine-selinux docker-engine
4.2、查看可用的docker版本
[root@controller lib]# yum list docker-engine.x86_64 --showduplicates |sort -r
Loading mirror speeds from cached hostfile
Loaded plugins: fastestmirror, langpacks
docker-engine.x86_64 1.9.1-1.el7.centos dockerrepo
docker-engine.x86_64 1.9.0-1.el7.centos dockerrepo
docker-engine.x86_64 1.8.3-1.el7.centos dockerrepo
docker-engine.x86_64 1.8.2-1.el7.centos dockerrepo
docker-engine.x86_64 1.8.1-1.el7.centos dockerrepo
docker-engine.x86_64 1.8.0-1.el7.centos dockerrepo
docker-engine.x86_64 1.7.1-1.el7.centos dockerrepo
docker-engine.x86_64 17.05.0.ce-1.el7.centos dockerrepo
docker-engine.x86_64 17.04.0.ce-1.el7.centos dockerrepo
docker-engine.x86_64 17.03.1.ce-1.el7.centos dockerrepo
docker-engine.x86_64 17.03.0.ce-1.el7.centos dockerrepo
docker-engine.x86_64 1.7.0-1.el7.centos dockerrepo
docker-engine.x86_64 1.13.1-1.el7.centos dockerrepo
docker-engine.x86_64 1.13.0-1.el7.centos dockerrepo
docker-engine.x86_64 1.12.6-1.el7.centos dockerrepo
docker-engine.x86_64 1.12.5-1.el7.centos dockerrepo
docker-engine.x86_64 1.12.4-1.el7.centos dockerrepo
docker-engine.x86_64 1.12.3-1.el7.centos dockerrepo
docker-engine.x86_64 1.12.2-1.el7.centos dockerrepo
docker-engine.x86_64 1.12.1-1.el7.centos dockerrepo
docker-engine.x86_64 1.12.0-1.el7.centos dockerrepo
docker-engine.x86_64 1.11.2-1.el7.centos dockerrepo
docker-engine.x86_64 1.11.1-1.el7.centos dockerrepo
docker-engine.x86_64 1.11.0-1.el7.centos dockerrepo
docker-engine.x86_64 1.10.3-1.el7.centos dockerrepo
docker-engine.x86_64 1.10.2-1.el7.centos dockerrepo
docker-engine.x86_64 1.10.1-1.el7.centos dockerrepo
docker-engine.x86_64 1.10.0-1.el7.centos dockerrepo
Available Packages
4.3、安裝穩(wěn)定的版本1.13.1
[root@controller lib]# yum -y install docker-engine-1.13.1-1.el7.centos
啟動docker服務并設置開機自啟動
systemctl start docker
systemctl enable docker
4.4、下載swarm鏡像
[root@controller ~]# docker pull swarm
Using default tag: latest
latest: Pulling from library/swarm
Digest: sha256:406022f04a3d0c5ce4dbdb60422f24052c20ab7e6d41ebe5723aa649c3833975
Status: Downloaded newer image for swarm:latest
[root@controller ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
swarm latest ff454b4a0e84 7 weeks ago 12.7 MB
4.5、創(chuàng)建swarm
[root@controller ~]# docker swarm init --advertise-addr 192.168.3.204
Swarm initialized: current node (r344b4bz1ug3bnw6olsxzqtis) is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join \
--token SWMTKN-1-1kvrsn7j74ie7fqxzxb0wndk76gjnwdyc6jnhe4771h55edgzw-20spfbubw7ckh1w0w3plvsak3 \
192.168.3.204:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
注:上面的token值要保存,其他節(jié)點均通過它加入集群
4.6、查看集群信息
[root@controller ~]# docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 1
Server Version: 1.13.1
Storage Driver: overlay
Backing Filesystem: xfs
Supports d_type: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: active
NodeID: r344b4bz1ug3bnw6olsxzqtis
Is Manager: true
ClusterID: p5r0a72jdoq9cztshyf0bvfye
Managers: 1
Nodes: 1
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 3
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Node Address: 192.168.3.204
Manager Addresses:
192.168.3.204:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1
runc version: 9df8b306d01f59d3a8029be411de015b7304dd8f
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 3.10.0-327.18.2.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.551 GiB
Name: controller
ID: 5E6V:BHYM:3L74:EYS3:ZUID:INQP:DINI:U7I6:QKJZ:UCYY:QZJF:GXGN
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
4.7、查看集群節(jié)點信息
[root@controller ~]# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
r344b4bz1ug3bnw6olsxzqtis * controller Ready Active Leader
5、配置節(jié)點(192.168.3.202,192.168.3.201)
5.1、安裝docker
yum -y install docker-engine-1.13.1-1.el7.centos
啟動docker
systemctl start docker
開機自啟動
systemctl enable docker
5.2、加入到集群
docker swarm join --token SWMTKN-1-1kvrsn7j74ie7fqxzxb0wndk76gjnwdyc6jnhe4771h55edgzw-20spfbubw7ckh1w0w3plvsak3 192.168.3.204:2377
5.3、查看docker信息(兩個節(jié)點一樣執(zhí)行,我這里只貼出在node2執(zhí)行的信息)
[root@node2 ~]# docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 1.13.1
Storage Driver: overlay
Backing Filesystem: xfs
Supports d_type: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: active
NodeID: pdqau3dpxtgnr6gg78zbt7nnz
Is Manager: false
Node Address: 192.168.3.202
Manager Addresses:
192.168.3.204:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1
runc version: 9df8b306d01f59d3a8029be411de015b7304dd8f
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 3.10.0-229.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.797 GiB
Name: node2
ID: Q7VU:TMEP:NCO6:YVXC:O6U2:2DON:5RUL:VGJ4:D2TA:345M:I4Q6:UHN5
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: bridge-nf-call-ip6tables is disabled
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
5.4、在集群節(jié)點(192.168.3.204)查看節(jié)點信息
[root@controller ~]# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
hjba4ers0ahzbdvggajc3b73s node1 Ready Active
pdqau3dpxtgnr6gg78zbt7nnz node2 Ready Active
r344b4bz1ug3bnw6olsxzqtis * controller Ready Active Leader
6、在swarm中部署nginx服務
6.1、創(chuàng)建覆蓋網(wǎng)絡(保障不同主機容器互通)
[root@controller ~]# docker network create -d overlay nginx
692ud2yyqimkuq6rufpnwnt05
[root@controller ~]# docker network ls
NETWORK ID NAME DRIVER SCOPE
27940e4bc00a bridge bridge local
cd675706a24e docker_gwbridge bridge local
ec66ee0e74b1 host host local
l8d4j4w5o52u ingress overlay swarm
692ud2yyqimk nginx overlay swarm
3a44f24d7c1f none null local
6.2、創(chuàng)建服務
創(chuàng)建:
[root@controller ~]# docker service create --replicas 1 --network nginx --name MyNginx -p 80:80 nginx
8sk44dipvooy4sb2e7e4l8wb1
查看服務
[root@controller ~]# docker service ls
ID NAME MODE REPLICAS IMAGE
8sk44dipvooy MyNginx replicated 1/1 nginx:latest
查看摘要信息:
[root@controller ~]# docker service inspect --pretty MyNginx
ID: 8sk44dipvooy4sb2e7e4l8wb1
Name: MyNginx
Service Mode: Replicated
Replicas: 1
Placement:
UpdateConfig:
Parallelism: 1
On failure: pause
Max failure ratio: 0
ContainerSpec:
Image: nginx:latest@sha256:d85914d547a6c92faa39ce7058bd7529baacab7e0cd4255442b04577c4d1f424
Resources:
Networks: nginx
Endpoint Mode: vip
Ports:
PublishedPort 80
Protocol = tcp
TargetPort = 80
6.3、查看服務的運行情況
[root@controller ~]# docker service ps MyNginx
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
khvp4r03de4f MyNginx.1 nginx:latest node2 Running Running about a minute ago
6.4、訪問服務
瀏覽器或者curl訪問服務http://192.168.3.204
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a >nginx.org</a>.<br/>
Commercial support is available at
<a >nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
7、Service的復制功能
7.1、增加
設置5個副本
[root@controller ~]# docker service scale MyNginx=5
MyNginx scaled to 5
[root@controller ~]# docker service ps MyNginx
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
khvp4r03de4f MyNginx.1 nginx:latest node2 Running Running 12 minutes ago
wecu7jye2mg3 MyNginx.2 nginx:latest controller Running Preparing 5 seconds ago
5a6rp8c6arvt MyNginx.3 nginx:latest node2 Running Running 4 seconds ago
68lu0fpg75on MyNginx.4 nginx:latest node1 Running Preparing 5 seconds ago
todza2i5lnnt MyNginx.5 nginx:latest controller Running Preparing 5 seconds ago
7.2、減少
變更為1個副本
[root@controller ~]# docker service scale MyNginx=1
MyNginx scaled to 1
[root@controller ~]# docker service ls
ID NAME MODE REPLICAS IMAGE
8sk44dipvooy MyNginx replicated 1/1 nginx:latest
[root@controller ~]# docker service ps MyNginx
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
exoh860irad5 MyNginx.1 nginx:latest node1 Running Running 4 minutes ago
8、使用volume
8.1、查看volume
[root@controller ~]# docker volume ls
DRIVER VOLUME NAME
8.2、創(chuàng)建volume
[root@controller ~]# docker volume create --name nginx
nginx
[root@controller ~]# docker volume ls
DRIVER VOLUME NAME
local nginx
8.3、創(chuàng)建服務并指定掛載
src是volume名稱
dst是掛載點(也就是容器內的路徑,如果不存在會自動創(chuàng)建)
注:若指定的是本機的路徑,需要把type改成bind, src是本機的路徑
[root@controller ~]# docker service create --replicas 2 --mount type=volume,src=nginx,dst=/farmer --name ng nginx
r7bld5xg9cy6285z6207wlly2
[root@controller ~]# docker service ls
ID NAME MODE REPLICAS IMAGE
9et4o10fn0w2 MyNginx replicated 2/2 nginx:latest
r7bld5xg9cy6 ng replicated 2/2 nginx:latest
[root@controller ~]# docker service ps ng
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
190gm96z6brc ng.1 nginx:latest node2 Running Running 25 seconds ago
093pql11ttdm ng.2 nginx:latest node1 Running Running 25 seconds ago
9、創(chuàng)建nginx服務和負載均衡的測試
9.1、創(chuàng)建volume名稱為modules
docker volume create --name modules
注:因為/etc/nginx/mudules是軟連接到/usr/lib/nginx/modules,一旦將/etc/nginx掛載到本地,這個目錄將會失效,所以需要在modules也掛到本地
lrwxrwxrwx 1 root root 22 Jul 24 21:02 modules -> /usr/lib/nginx/modules
9.2、創(chuàng)建軟連接(所有節(jié)點,192.168.3.204,192.168.3.202,192.168.3.201)
mkdir /usr/lib/nginx
ln -s /var/lib/docker/volumes/modules/_data/modules/ /usr/lib/nginx/modules
9.3、創(chuàng)建test服務
[root@controller ~]# docker service create --replicas 2 --mount type=volume,src=nginx,dst=/etc/nginx/ --mount type=volume,dst=/usr/lib/nginx/,src=modules --name test -p 8880:80 nginx
9mt9wuabsluykwfh2j7ez9gut
[root@controller ~]# docker service ps test
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
hwp14enb23gz test.1 nginx:latest node1 Running Running 6 seconds ago
t6mpq6sbongg test.2 nginx:latest controller Running Running 6 seconds ago
9.4、配置HTML服務(三個節(jié)點,192.168.3.201,192.168.3.202,192.168.3.204)
cd /var/lib/docker/volumes/nginx/_data
mkdir html
vim conf.d/default.conf
修改如下:
location / {
root /etc/nginx/html;
index index.html index.htm;
}
注:這里用/etc/nginx/html路徑是因為在docker里面路徑是這個
我把創(chuàng)建的新的html當做nginx的root目錄
9.4.1、管理節(jié)點
cd /var/lib/docker/volumes/nginx/_data/html
touch index.html
添加內容:
<h1>Controller</h1>
9.4.1、node1節(jié)點
cd /var/lib/docker/volumes/nginx/_data/html
touch index.html
添加內容:
<h1>Node1</h1>
9.4.2、node2節(jié)點
cd /var/lib/docker/volumes/nginx/_data/html
touch index.html
添加內容:
<h1>Node2</h1>
9.5、重啟nginx服務
當前服務在node1和controller節(jié)點運行,故而只在這兩個節(jié)點執(zhí)行(機器環(huán)境不同,所在的節(jié)點不同,請看清自己的節(jié)點)
9.5.1、node1節(jié)點
[root@node1 ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
33b4e08c57d0 nginx@sha256:d85914d547a6c92faa39ce7058bd7529baacab7e0cd4255442b04577c4d1f424 "nginx -g 'daemon ..." 4 hours ago Up 4 hours 80/tcp test.1.ov76pdo715cc1f24i5vfgu6si
docker exec -ti 33b4e08c57d0 /bin/bash
/etc/init.d/nginx reload
9.5.2、controller節(jié)點
[root@controller _data]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e54913d64ce9 nginx@sha256:d85914d547a6c92faa39ce7058bd7529baacab7e0cd4255442b04577c4d1f424 "nginx -g 'daemon ..." 3 days ago Up 3 days 80/tcp test.2.t6mpq6sbongg5rq8rd4lofp6u
[root@controller _data]# docker exec -ti e54913d64ce9 /bin/bash
root@e54913d64ce9:/# /etc/init.d/nginx reload
9.6、測試服務(服務是否正常,是否有負載)
注:測試過程中有時候多curl幾次才會負載到另外的一個節(jié)點
[root@mysql76 ~]# curl http://192.168.3.204:8880/
<h1>Node1</h1>
[root@mysql76 ~]# curl http://192.168.3.204:8880/
<h1>Controller</h1>
9.7、測試集群
在節(jié)點1執(zhí)行
docker stop 33b4e08c57d0
查看集群服務
[root@controller _data]# docker service ps test
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
ov76pdo715cc test.1 nginx:latest node2 Running Running 1 hours ago
qoyuk652tky2 \_ test.1 nginx:latest node1 Shutdown Complete 1 hours ago
t6mpq6sbongg test.2 nginx:latest controller Running Running 3 days ago
測試集群服務(服務是否正常,是否有負載)
注:測試過程中有時候多curl幾次才會負載到另外的一個節(jié)點
[root@mysql76 ~]# curl http://192.168.3.204:8880/
<h1>Node2</h1>
[root@mysql76 ~]# curl http://192.168.3.204:8880/
<h1>Controller</h1>