MongoDB 集群構(gòu)建：分片+副本+選舉

?? 環(huán)境準(zhǔn)備

三臺(tái)服務(wù)器，建立 Docker Swarm 集群，一個(gè) Manager(node1)，兩個(gè) Worker(node2 + node3)。

  • docker 版本：19.03.0-beta1
  • mongo 版本：4.0.3

?? MongoDB 集群架構(gòu)設(shè)計(jì)

架構(gòu)圖

高清圖地址: https://www.processon.com/view/link/5a3c7386e4b0bf89b8530376

?? 搭建集群

1、【Manager】創(chuàng)建集群網(wǎng)絡(luò)

docker network create -d overlay --attachable mongo

--attachable 允許其他容器加入此網(wǎng)絡(luò)

2、創(chuàng)建 9 個(gè) Data 服務(wù)，3 個(gè) Config 服務(wù)，1 個(gè) Global 模式的 Mongos 服務(wù)

2.1、【所有機(jī)器】創(chuàng)建相關(guān)文件夾

mkdir -p  ~/mongo/config ~/mongo/shard1 ~/mongo/shard2 ~/mongo/shard3

2.2、【Manager】創(chuàng)建 stack.yml

version: '3.7'
services:
  mongors1n1:
    image: mongo
    command: mongod --shardsvr --replSet shard1 --dbpath /data/db --bind_ip_all --port 27017
    networks:
      - mongo
    environment:
      - TZ=Asia/Shanghai
    volumes:
      - ~/mongo/shard1:/data/db
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname==node1
  mongors2n1:
    image: mongo
    command: mongod --shardsvr --replSet shard2 --dbpath /data/db --bind_ip_all --port 27017
    networks:
      - mongo
    environment:
      - TZ=Asia/Shanghai
    volumes:
      - ~/mongo/shard2:/data/db
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname==node1
  mongors3n1:
    image: mongo
    command: mongod --shardsvr --replSet shard3 --dbpath /data/db --bind_ip_all --port 27017
    networks:
      - mongo
    environment:
      - TZ=Asia/Shanghai
    volumes:
      - ~/mongo/shard3:/data/db
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname==node1
  mongors1n2:
    image: mongo
    command: mongod --shardsvr --replSet shard1 --dbpath /data/db --bind_ip_all --port 27017
    networks:
      - mongo
    environment:
      - TZ=Asia/Shanghai
    volumes:
      - ~/mongo/shard1:/data/db
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname==node2
  mongors2n2:
    image: mongo
    command: mongod --shardsvr --replSet shard2 --dbpath /data/db --bind_ip_all --port 27017
    networks:
      - mongo
    environment:
      - TZ=Asia/Shanghai
    volumes:
      - ~/mongo/shard2:/data/db
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname==node2
  mongors3n2:
    image: mongo
    command: mongod --shardsvr --replSet shard3 --dbpath /data/db --bind_ip_all --port 27017
    networks:
      - mongo
    environment:
      - TZ=Asia/Shanghai
    volumes:
      - ~/mongo/shard3:/data/db
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname==node2
  mongors1n3:
    image: mongo
    command: mongod --shardsvr --replSet shard1 --dbpath /data/db --bind_ip_all --port 27017
    networks:
      - mongo
    environment:
      - TZ=Asia/Shanghai
    volumes:
      - ~/mongo/shard1:/data/db
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname==node3
  mongors2n3:
    image: mongo
    command: mongod --shardsvr --replSet shard2 --dbpath /data/db --bind_ip_all --port 27017
    networks:
      - mongo
    environment:
      - TZ=Asia/Shanghai
    volumes:
      - ~/mongo/shard2:/data/db
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname==node3
  mongors3n3:
    image: mongo
    command: mongod --shardsvr --replSet shard3 --dbpath /data/db --bind_ip_all --port 27017
    networks:
      - mongo
    environment:
      - TZ=Asia/Shanghai
    volumes:
      - ~/mongo/shard3:/data/db
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname==node3
  cfg1:
    image: mongo
    command: mongod --configsvr --replSet cfgrs --dbpath /data/db --bind_ip_all --port 27017
    networks:
      - mongo
    environment:
      - TZ=Asia/Shanghai
    volumes:
      - ~/mongo/config:/data/db
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname==node1
  cfg2:
    image: mongo
    command: mongod --configsvr --replSet cfgrs --dbpath /data/db --bind_ip_all --port 27017
    networks:
      - mongo
    environment:
      - TZ=Asia/Shanghai
    volumes:
      - ~/mongo/config:/data/db
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname==node2
  cfg3:
    image: mongo
    command: mongod --configsvr --replSet cfgrs --dbpath /data/db --bind_ip_all --port 27017
    networks:
      - mongo
    environment:
      - TZ=Asia/Shanghai
    volumes:
      - ~/mongo/config:/data/db
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname==node3
  mongos:
    image: mongo
    command: mongos --configdb cfgrs/cfg1:27017,cfg2:27017,cfg3:27017 --bind_ip_all --port 27017
    networks:
      - mongo
    ports:
      - 27017:27017
    environment:
      - TZ=Asia/Shanghai
    deploy:
      mode: global
networks:
  mongo:
    external: true

2.3、啟動(dòng)服務(wù)，在 Manager 上執(zhí)行

docker stack deploy -c stack.yml mongo

2.4、【Manager】查看服務(wù)的啟動(dòng)情況

docker service ls

正常情況下，會(huì)出現(xiàn)如下結(jié)果：

[docker@manager ~]# docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE                                         PORTS
z1l5zlghlfbi        mongo_cfg1          replicated          1/1                 registry.docker-cn.com/library/mongo:latest
lg9vbods29th        mongo_cfg2          replicated          1/1                 registry.docker-cn.com/library/mongo:latest
i6d6zwxsq0ss        mongo_cfg3          replicated          1/1                 registry.docker-cn.com/library/mongo:latest
o0lfdavd8kpj        mongo_mongors1n1    replicated          1/1                 registry.docker-cn.com/library/mongo:latest
n85yeyod7mlu        mongo_mongors1n2    replicated          1/1                 registry.docker-cn.com/library/mongo:latest
cwurdqng9tdk        mongo_mongors1n3    replicated          1/1                 registry.docker-cn.com/library/mongo:latest
vu6al5kys28u        mongo_mongors2n1    replicated          1/1                 registry.docker-cn.com/library/mongo:latest
xrjiep0vrf0w        mongo_mongors2n2    replicated          1/1                 registry.docker-cn.com/library/mongo:latest
qqzifwcejjyk        mongo_mongors2n3    replicated          1/1                 registry.docker-cn.com/library/mongo:latest
tddgw8hygv1b        mongo_mongors3n1    replicated          1/1                 registry.docker-cn.com/library/mongo:latest
qrb6fjty03mw        mongo_mongors3n2    replicated          1/1                 registry.docker-cn.com/library/mongo:latest
m8ikdzjssmhn        mongo_mongors3n3    replicated          1/1                 registry.docker-cn.com/library/mongo:latest
mnnlm49b7kyb        mongo_mongos        global              3/3                 registry.docker-cn.com/library/mongo:latest   *:27017->27017/tcp

3、初始化集群

3.1 【node1】初始化 Mongo 配置集群

docker exec -it $(docker ps | grep "cfg1" | awk '{ print $1 }') bash -c "echo 'rs.initiate({_id: \"cfgrs\",configsvr: true, members: [{ _id : 0, host : \"cfg1\" },{ _id : 1, host : \"cfg2\" }, { _id : 2, host : \"cfg3\" }]})' | mongo"

3.2 【node1】初始化三個(gè) Mongo 數(shù)據(jù)集群

docker exec -it $(docker ps | grep "mongors1n1" | awk '{ print $1 }') bash -c "echo 'rs.initiate({_id : \"shard1\", members: [{ _id : 0, host : \"mongors1n1\" },{ _id : 1, host : \"mongors1n2\" },{ _id : 2, host : \"mongors1n3\"}]})' | mongo"

docker exec -it $(docker ps | grep "mongors2n1" | awk '{ print $1 }') bash -c "echo 'rs.initiate({_id : \"shard2\", members: [{ _id : 0, host : \"mongors2n1\" },{ _id : 1, host : \"mongors2n2\" },{ _id : 2, host : \"mongors2n3\" }]})' | mongo"

docker exec -it $(docker ps | grep "mongors3n1" | awk '{ print $1 }') bash -c "echo 'rs.initiate({_id : \"shard3\", members: [{ _id : 0, host : \"mongors3n1\" },{ _id : 1, host : \"mongors3n2\" },{ _id : 2, host : \"mongors3n3\" }]})' | mongo"

3.3 【node1】將三個(gè)數(shù)據(jù)集群當(dāng)做分片加入 mongos

docker exec -it $(docker ps | grep "mongos" | awk '{ print $1 }') bash -c "echo 'sh.addShard(\"shard1/mongors1n1:27017,mongors1n2:27017,mongors1n3:27017\")' | mongo "

docker exec -it $(docker ps | grep "mongos" | awk '{ print $1 }') bash -c "echo 'sh.addShard(\"shard2/mongors2n1:27017,mongors2n2:27017,mongors2n3:27017\")' | mongo "

docker exec -it $(docker ps | grep "mongos" | awk '{ print $1 }') bash -c "echo 'sh.addShard(\"shard3/mongors3n1:27017,mongors3n2:27017,mongors3n3:27017\")' | mongo "

4、連接集群

4.1 內(nèi)部：在 mongo 網(wǎng)絡(luò)下的容器，通過(guò) mongos:27017 連接

4.2 外部：通過(guò) IP:27017 連接，IP 可以為三臺(tái)服務(wù)的中的一個(gè)的 IP

5、測(cè)試數(shù)據(jù)

# 【node1】連接集群
docker exec -it $(docker ps | grep "mongos" | awk '{ print $1 }') mongo mongos:27017

# 【node1】創(chuàng)建數(shù)據(jù)庫(kù)
use test_db

# 【node1】允許數(shù)據(jù)庫(kù)分片
sh.enableSharding("test_db", "hashed")

# 【node1】創(chuàng)建文檔
db.createCollection("user")

# 【node1】分片設(shè)置
sh.shardCollection( "test_db.user", {_id:"hashed"} )

# 【node1】添加數(shù)據(jù)
for(var i = 0; i < 1000; i++) { db.user.insert({_id:i, name:"test" + i}) }

# 【node1】查詢數(shù)據(jù)
db.user.find().pretty()

# 【node1】查詢分片狀態(tài)
db.user.getShardDistribution()

https://docs.mongodb.com/manual/core/ranged-sharding
https://docs.mongodb.com/manual/core/hashed-sharding/

?? 相關(guān)問(wèn)題

此處記錄相關(guān)問(wèn)題，歡迎提問(wèn)！

微服務(wù) WebSocket 集群項(xiàng)目 AnyIM 實(shí)戰(zhàn)

最后編輯于 ：
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡(jiǎn)書系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

友情鏈接更多精彩內(nèi)容