MongoDB學(xué)習(xí)筆記（二）

分片集群（Sharded Cluster）

  • 分片（sharding）是一種跨多臺(tái)機(jī)器分布數(shù)據(jù)的方法， MongoDB使用分片來支持具有非常大的數(shù)據(jù)集和高吞吐量操作的部署。將數(shù)據(jù)分散到不同的機(jī)器上，不需要功能強(qiáng)大的大型計(jì)算機(jī)就可以儲(chǔ)存更多的數(shù)據(jù)，處理更多的負(fù)載。
  • 具有大型數(shù)據(jù)集或高吞吐量應(yīng)用程序的數(shù)據(jù)庫系統(tǒng)會(huì)挑戰(zhàn)單個(gè)服務(wù)器的容量，例如：高查詢率會(huì)耗盡服務(wù)器的CPU容量。工作集大小大于系統(tǒng)的RAM會(huì)強(qiáng)調(diào)磁盤驅(qū)動(dòng)器的I/O容量。
  • 有兩種解決系統(tǒng)增長(zhǎng)的方法：垂直擴(kuò)展和水平擴(kuò)展。
    • 垂直擴(kuò)展意味著增加單個(gè)服務(wù)器的容量，例如使用更強(qiáng)大的CPU，添加更多的RAM或增加存儲(chǔ)空間量。因?yàn)榛谠频奶峁┥袒诳捎玫挠布渲镁哂杏残陨舷蓿源怪笨s放有實(shí)際的最大值。
    • 水平擴(kuò)展意味著劃分系統(tǒng)數(shù)據(jù)集并加載多個(gè)服務(wù)器，添加其它服務(wù)器以根據(jù)需要增加容量。雖然單個(gè)機(jī)器的總體速度或容量可能不高，但每臺(tái)機(jī)器處理整個(gè)工作負(fù)載的子集，可能提供比單個(gè)高速大容量服務(wù)器更高的效率。擴(kuò)展部署容量只需根據(jù)需添加額外的服務(wù)器，這可能比單個(gè)機(jī)器的高端硬件的總體成本更低。權(quán)衡是基礎(chǔ)架構(gòu)和部署維護(hù)的復(fù)雜性增加。
  • MongoDB支持通過分片進(jìn)行水平擴(kuò)展。
  • MongoDB分片群集包含以下組件：
    • 分片（存儲(chǔ)）：每個(gè)分片包含分片數(shù)據(jù)的子集，每個(gè)分片都可以部署為副本集。
    • mongos （路由）：mongos充當(dāng)查詢路由器，在客戶端應(yīng)用程序和分片集群之間提供接口。
    • config servers （"調(diào)度"的配置）：配置服務(wù)器存儲(chǔ)群集的元數(shù)據(jù)。 從MongoDB 3.4開始，必須將配置服務(wù)器部署為副本集（CSRS）。
分片集群中組件的交互
  • MongoDB在集合級(jí)別對(duì)數(shù)據(jù)進(jìn)行分片，將集合數(shù)據(jù)分布在集群中的分片上。
  • 分片集群架構(gòu)目標(biāo)：兩個(gè)分片節(jié)點(diǎn)副本集（3+3）+一個(gè)配置節(jié)點(diǎn)副本集（3）+兩個(gè)路由節(jié)點(diǎn)（2），共11個(gè)服務(wù)節(jié)點(diǎn)。

分片（存儲(chǔ)）節(jié)點(diǎn)副本集的創(chuàng)建

第一套副本集

  • 準(zhǔn)備存放數(shù)據(jù)、日志和配置文件的目錄：
mkdir -p /opt/mongodb/sharded_cluster/myshardrs01_27018/logs \ &
mkdir -p /opt/mongodb/sharded_cluster/myshardrs01_27018/data/db \ &
mkdir -p /opt/mongodb/sharded_cluster/myshardrs01_27018/conf \&
mkdir -p /opt/mongodb/sharded_cluster/myshardrs01_27118/logs \ &
mkdir -p /opt/mongodb/sharded_cluster/myshardrs01_27118/data/db \ &
mkdir -p /opt/mongodb/sharded_cluster/myshardrs01_27118/conf \&
mkdir -p /opt/mongodb/sharded_cluster/myshardrs01_27218/logs \ &
mkdir -p /opt/mongodb/sharded_cluster/myshardrs01_27218/data/db \&
mkdir -p /opt/mongodb/sharded_cluster/myshardrs01_27218/conf \&
  • 新建一個(gè)配置文件：vim /opt/mongodb/sharded_cluster/myshardrs01_27018/conf/mongod.conf 
 systemLog:
    destination: file
    path: /opt/mongodb/sharded_cluster/myshardrs01_27018/logs/mongod.log
    logAppend: true
 storage:
    dbPath: /opt/mongodb/sharded_cluster/myshardrs01_27018/data/db
    journal:
        enabled: true
 processManagement:
    fork: true
    pidFilePath: /opt/mongodb/sharded_cluster/myshardrs01_27018/logs/mongod.pid
 net:
    bindIp: localhost,192.168.0.128
    port: 27018
 replication:
    replSetName: myshardrs01
 sharding:
    clusterRole: shardsvr
  • 對(duì)sharding.clusterRole值的說明：
值 描述
configsvr Start this instance as a config server. The instance starts on port 27019 by default.
shardsvr Start this instance as a shard. The instance starts on port 27018 by default.
  • 注意：設(shè)置sharding.clusterRole需要mongod實(shí)例運(yùn)行復(fù)制，要將實(shí)例部署為副本集成員，并設(shè)置replSetName副本集的名稱。
  • 新建一個(gè)配置文件：vim /opt/mongodb/sharded_cluster/myshardrs01_27118/conf/mongod.conf 
 systemLog:
    destination: file
    path: /opt/mongodb/sharded_cluster/myshardrs01_27118/logs/mongod.log
    logAppend: true
 storage:
    dbPath: /opt/mongodb/sharded_cluster/myshardrs01_27118/data/db
    journal:
        enabled: true
 processManagement:
    fork: true
    pidFilePath: /opt/mongodb/sharded_cluster/myshardrs01_27118/logs/mongod.pid
 net:
    bindIp: localhost,192.168.0.128
    port: 27118
 replication:
    replSetName: myshardrs01
 sharding:
    clusterRole: shardsvr
  • 新建一個(gè)配置文件：vim /opt/mongodb/sharded_cluster/myshardrs01_27218/conf/mongod.conf 
 systemLog:
    destination: file
    path: /opt/mongodb/sharded_cluster/myshardrs01_27218/logs/mongod.log
    logAppend: true
 storage:
    dbPath: /opt/mongodb/sharded_cluster/myshardrs01_27218/data/db
    journal:
        enabled: true
 processManagement:
    fork: true
    pidFilePath: /opt/mongodb/sharded_cluster/myshardrs01_27218/logs/mongod.pid
 net:
    bindIp: localhost,192.168.0.128
    port: 27218
 replication:
    replSetName: myshardrs01
 sharding:
    clusterRole: shardsvr
  • 啟動(dòng)第一套副本集：一主一副本一仲裁。
[root@dev mongodb]# /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27018/conf/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 24008
child process started successfully, parent exiting
[root@dev mongodb]# /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27118/conf/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 24054
child process started successfully, parent exiting
[root@dev mongodb]# /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27218/conf/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 24097
child process started successfully, parent exiting
[root@dev mongodb]# ps -ef |grep mongod
root     24008     1  1 10:17 ?        00:00:01 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27018/conf/mongod.conf
root     24054     1  1 10:17 ?        00:00:01 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27118/conf/mongod.conf
root     24097     1  1 10:18 ?        00:00:00 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27218/conf/mongod.conf
root     24151 23883  0 10:19 pts/0    00:00:00 grep --color=auto mongod
  • 初始化副本集和創(chuàng)建主節(jié)點(diǎn)：使用客戶端命令連接任意一個(gè)節(jié)點(diǎn)，但盡量要連接主節(jié)點(diǎn)，并添加副本節(jié)點(diǎn)和仲裁節(jié)點(diǎn)：
[root@dev mongodb]# /opt/mongodb/bin/mongo --port 27018
> rs.initiate()
{
    "info2" : "no configuration specified. Using a default configuration for the set",
    "me" : "公網(wǎng)ip:27018",
    "ok" : 1,
    "$clusterTime" : {
        "clusterTime" : Timestamp(1610419832, 1),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    },
    "operationTime" : Timestamp(1610419832, 1)
}
myshardrs01:SECONDARY> 
myshardrs01:PRIMARY> rs.add("公網(wǎng)ip:27118")
{
    "ok" : 1,
    "$clusterTime" : {
        "clusterTime" : Timestamp(1610420477, 1),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    },
    "operationTime" : Timestamp(1610420477, 1)
}
myshardrs01:PRIMARY> rs.addArb("公網(wǎng)ip:27218")
{
    "ok" : 1,
    "$clusterTime" : {
        "clusterTime" : Timestamp(1610420529, 1),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    },
    "operationTime" : Timestamp(1610420529, 1)
}

第二套副本集

  • 準(zhǔn)備存放數(shù)據(jù)、日志和配置文件的目錄：
mkdir -p /opt/mongodb/sharded_cluster/myshardrs02_27318/logs \ &
mkdir -p /opt/mongodb/sharded_cluster/myshardrs02_27318/data/db \ &
mkdir -p /opt/mongodb/sharded_cluster/myshardrs02_27318/conf \&
mkdir -p /opt/mongodb/sharded_cluster/myshardrs02_27418/logs \ &
mkdir -p /opt/mongodb/sharded_cluster/myshardrs02_27418/data/db \ &
mkdir -p /opt/mongodb/sharded_cluster/myshardrs02_27418/conf \&
mkdir -p /opt/mongodb/sharded_cluster/myshardrs02_27518/logs \ &
mkdir -p /opt/mongodb/sharded_cluster/myshardrs02_27518/data/db \&
mkdir -p /opt/mongodb/sharded_cluster/myshardrs02_27518/conf \&
  • 新建一個(gè)配置文件：vim /opt/mongodb/sharded_cluster/myshardrs02_27318/conf/mongod.conf 
 systemLog:
    destination: file
    path: /opt/mongodb/sharded_cluster/myshardrs02_27318/logs/mongod.log
    logAppend: true
 storage:
    dbPath: /opt/mongodb/sharded_cluster/myshardrs02_27318/data/db
    journal:
        enabled: true
 processManagement:
    fork: true
    pidFilePath: /opt/mongodb/sharded_cluster/myshardrs02_27318/logs/mongod.pid
 net:
    bindIp: localhost,192.168.0.128
    port: 27318
 replication:
    replSetName: myshardrs02
 sharding:
    clusterRole: shardsvr
  • 新建一個(gè)配置文件：vim /opt/mongodb/sharded_cluster/myshardrs02_27418/conf/mongod.conf 
 systemLog:
    destination: file
    path: /opt/mongodb/sharded_cluster/myshardrs02_27418/logs/mongod.log
    logAppend: true
 storage:
    dbPath: /opt/mongodb/sharded_cluster/myshardrs02_27418/data/db
    journal:
        enabled: true
 processManagement:
    fork: true
    pidFilePath: /opt/mongodb/sharded_cluster/myshardrs02_27418/logs/mongod.pid
 net:
    bindIp: localhost,192.168.0.128
    port: 27418
 replication:
    replSetName: myshardrs02
 sharding:
    clusterRole: shardsvr
  • 新建一個(gè)配置文件：vim /opt/mongodb/sharded_cluster/myshardrs02_27518/conf/mongod.conf 
 systemLog:
    destination: file
    path: /opt/mongodb/sharded_cluster/myshardrs02_27518/logs/mongod.log
    logAppend: true
 storage:
    dbPath: /opt/mongodb/sharded_cluster/myshardrs02_27518/data/db
    journal:
        enabled: true
 processManagement:
    fork: true
    pidFilePath: /opt/mongodb/sharded_cluster/myshardrs02_27518/logs/mongod.pid
 net:
    bindIp: localhost,192.168.0.128
    port: 27518
 replication:
    replSetName: myshardrs02
 sharding:
    clusterRole: shardsvr
  • 啟動(dòng)第二套副本集：一主一副本一仲裁。
[root@dev mongodb]# /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs02_27318/conf/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 24173
child process started successfully, parent exiting
[root@dev mongodb]# /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs02_27418/conf/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 24216
child process started successfully, parent exiting
[root@dev mongodb]# /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs02_27518/conf/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 24259
child process started successfully, parent exiting
[root@dev mongodb]# ps -ef | grep mongod
root     24008     1  0 10:17 ?        00:00:05 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27018/conf/mongod.conf
root     24054     1  0 10:17 ?        00:00:05 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27118/conf/mongod.conf
root     24097     1  0 10:18 ?        00:00:05 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27218/conf/mongod.conf
root     24173     1  1 10:30 ?        00:00:01 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs02_27318/conf/mongod.conf
root     24216     1  1 10:30 ?        00:00:01 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs02_27418/conf/mongod.conf
root     24259     1  1 10:30 ?        00:00:00 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs02_27518/conf/mongod.conf
root     24301 23883  0 10:31 pts/0    00:00:00 grep --color=auto mongod
  • 初始化副本集和創(chuàng)建主節(jié)點(diǎn)：使用客戶端命令連接任意一個(gè)節(jié)點(diǎn)，但盡量要連接主節(jié)點(diǎn)，并添加副本節(jié)點(diǎn)和仲裁節(jié)點(diǎn)：
[root@dev mongodb]# /opt/mongodb/bin/mongo --port 27318
> rs.initiate()
{
    "info2" : "no configuration specified. Using a default configuration for the set",
    "me" : "公網(wǎng)ip:27318",
    "ok" : 1,
    "$clusterTime" : {
        "clusterTime" : Timestamp(1610420740, 1),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    },
    "operationTime" : Timestamp(1610420740, 1)
}
myshardrs02:SECONDARY> 
myshardrs02:PRIMARY> rs.add("公網(wǎng)ip:27418")
{
    "ok" : 1,
    "$clusterTime" : {
        "clusterTime" : Timestamp(1610420841, 1),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    },
    "operationTime" : Timestamp(1610420841, 1)
}
myshardrs02:PRIMARY> rs.addArb("公網(wǎng)ip:27518")
{
    "ok" : 1,
    "$clusterTime" : {
        "clusterTime" : Timestamp(1610420911, 1),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    },
    "operationTime" : Timestamp(1610420911, 1)
}

配置節(jié)點(diǎn)副本集

  • 準(zhǔn)備存放數(shù)據(jù)、日志和配置文件的目錄：
mkdir -p /opt/mongodb/sharded_cluster/myconfigrs_27019/logs \ &
mkdir -p /opt/mongodb/sharded_cluster/myconfigrs_27019/data/db \ &
mkdir -p /opt/mongodb/sharded_cluster/myconfigrs_27019/conf \&
mkdir -p /opt/mongodb/sharded_cluster/myconfigrs_27119/logs \ &
mkdir -p /opt/mongodb/sharded_cluster/myconfigrs_27119/data/db \ &
mkdir -p /opt/mongodb/sharded_cluster/myconfigrs_27119/conf \&
mkdir -p /opt/mongodb/sharded_cluster/myconfigrs_27219/logs \ &
mkdir -p /opt/mongodb/sharded_cluster/myconfigrs_27219/data/db \&
mkdir -p /opt/mongodb/sharded_cluster/myconfigrs_27219/conf \&
  • 新建一個(gè)配置文件：vim /opt/mongodb/sharded_cluster/myconfigrs_27019/conf/mongod.conf 
 systemLog:
    destination: file
    path: /opt/mongodb/sharded_cluster/myconfigrs_27019/logs/mongod.log
    logAppend: true
 storage:
    dbPath: /opt/mongodb/sharded_cluster/myconfigrs_27019/data/db
    journal:
        enabled: true
 processManagement:
    fork: true
    pidFilePath: /opt/mongodb/sharded_cluster/myconfigrs_27019/logs/mongod.pid
 net:
    bindIp: localhost,192.168.0.128
    port: 27019
 replication:
    replSetName: myconfigrs
 sharding:
    clusterRole: configsvr
  • 新建一個(gè)配置文件：vim /opt/mongodb/sharded_cluster/myconfigrs_27119/conf/mongod.conf 
 systemLog:
    destination: file
    path: /opt/mongodb/sharded_cluster/myconfigrs_27119/logs/mongod.log
    logAppend: true
 storage:
    dbPath: /opt/mongodb/sharded_cluster/myconfigrs_27119/data/db
    journal:
        enabled: true
 processManagement:
    fork: true
    pidFilePath: /opt/mongodb/sharded_cluster/myconfigrs_27119/logs/mongod.pid
 net:
    bindIp: localhost,192.168.0.128
    port: 27119
 replication:
    replSetName: myconfigrs
 sharding:
    clusterRole: configsvr
  • 新建一個(gè)配置文件：vim /opt/mongodb/sharded_cluster/myconfigrs_27219/conf/mongod.conf 
 systemLog:
    destination: file
    path: /opt/mongodb/sharded_cluster/myconfigrs_27219/logs/mongod.log
    logAppend: true
 storage:
    dbPath: /opt/mongodb/sharded_cluster/myconfigrs_27219/data/db
    journal:
        enabled: true
 processManagement:
    fork: true
    pidFilePath: /opt/mongodb/sharded_cluster/myconfigrs_27219/logs/mongod.pid
 net:
    bindIp: localhost,192.168.0.128
    port: 27219
 replication:
    replSetName: myconfigrs
 sharding:
    clusterRole: configsvr
  • 啟動(dòng)副本集：一主兩副本。
[root@dev mongodb]# /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myconfigrs_27019/conf/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 24329
child process started successfully, parent exiting
[root@dev mongodb]# /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myconfigrs_27119/conf/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 24380
child process started successfully, parent exiting
[root@dev mongodb]# /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myconfigrs_27219/conf/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 24431
child process started successfully, parent exiting
[root@dev mongodb]# ps -ef | grep mongod
root     24008     1  0 10:17 ?        00:00:09 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27018/conf/mongod.conf
root     24054     1  0 10:17 ?        00:00:09 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27118/conf/mongod.conf
root     24097     1  0 10:18 ?        00:00:09 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27218/conf/mongod.conf
root     24173     1  0 10:30 ?        00:00:05 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs02_27318/conf/mongod.conf
root     24216     1  0 10:30 ?        00:00:05 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs02_27418/conf/mongod.conf
root     24259     1  0 10:30 ?        00:00:05 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs02_27518/conf/mongod.conf
root     24329     1  1 10:44 ?        00:00:00 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myconfigrs_27019/conf/mongod.conf
root     24380     1  1 10:45 ?        00:00:00 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myconfigrs_27119/conf/mongod.conf
root     24431     1  2 10:45 ?        00:00:00 /opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myconfigrs_27219/conf/mongod.conf
root     24483 23883  0 10:45 pts/0    00:00:00 grep --color=auto mongod
  • 初始化副本集和創(chuàng)建主節(jié)點(diǎn)：使用客戶端命令連接任意一個(gè)節(jié)點(diǎn)，但盡量要連接主節(jié)點(diǎn)，并添加兩個(gè)副本節(jié)點(diǎn)：
[root@dev mongodb]# /opt/mongodb/bin/mongo --port 27019
> rs.initiate()
{
    "info2" : "no configuration specified. Using a default configuration for the set",
    "me" : "公網(wǎng)ip:27019",
    "ok" : 1,
    "$gleStats" : {
        "lastOpTime" : Timestamp(1610421160, 1),
        "electionId" : ObjectId("000000000000000000000000")
    },
    "lastCommittedOpTime" : Timestamp(0, 0),
    "$clusterTime" : {
        "clusterTime" : Timestamp(1610421160, 1),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    },
    "operationTime" : Timestamp(1610421160, 1)
}
myconfigrs:SECONDARY> 
myconfigrs:PRIMARY> rs.add("公網(wǎng)ip:27119")
{
    "ok" : 1,
    "$gleStats" : {
        "lastOpTime" : {
            "ts" : Timestamp(1610421330, 1),
            "t" : NumberLong(1)
        },
        "electionId" : ObjectId("7fffffff0000000000000001")
    },
    "lastCommittedOpTime" : Timestamp(1610421330, 1),
    "$clusterTime" : {
        "clusterTime" : Timestamp(1610421332, 1),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    },
    "operationTime" : Timestamp(1610421330, 1)
}
myconfigrs:PRIMARY> rs.add("公網(wǎng)ip:27219")
{
    "ok" : 1,
    "$gleStats" : {
        "lastOpTime" : {
            "ts" : Timestamp(1610421343, 2),
            "t" : NumberLong(1)
        },
        "electionId" : ObjectId("7fffffff0000000000000001")
    },
    "lastCommittedOpTime" : Timestamp(1610421345, 1),
    "$clusterTime" : {
        "clusterTime" : Timestamp(1610421345, 1),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    },
    "operationTime" : Timestamp(1610421343, 2)
}

路由節(jié)點(diǎn)的創(chuàng)建和操作

第一個(gè)路由節(jié)點(diǎn)的創(chuàng)建和連接

  • 準(zhǔn)備存放數(shù)據(jù)、日志和配置文件的目錄：
mkdir -p /opt/mongodb/sharded_cluster/mymongos_27017/logs \ &
mkdir -p /opt/mongodb/sharded_cluster/mymongos_27017/conf
  • 新建一個(gè)配置文件：vim /opt/mongodb/sharded_cluster/mymongos_27017/conf/mongos.conf 
 systemLog:
    destination: file
    path: /opt/mongodb/sharded_cluster/mymongos_27017/logs/mongod.log
    logAppend: true
 processManagement:
    fork: true
    pidFilePath: /opt/mongodb/sharded_cluster/mymongos_27017/logs/mongod.pid
 net:
    bindIp: localhost,192.168.0.128
    port: 27017
 sharding:
    configDB: myconfigrs/公網(wǎng)ip:27019,公網(wǎng)ip:27119,公網(wǎng)ip:27219
  • 啟動(dòng)mongos服務(wù)，若啟動(dòng)失敗，則查看 logs目錄下的日志。
[root@dev mongodb]# /opt/mongodb/bin/mongos -f /opt/mongodb/sharded_cluster/mymongos_27017/conf/mongos.conf
about to fork child process, waiting until server is ready for connections.
forked process: 25188
child process started successfully, parent exiting
  • 登錄mongos客戶端，若寫數(shù)據(jù)則會(huì)報(bào)錯(cuò)，原因：通過路由節(jié)點(diǎn)操作，現(xiàn)只是連接了配置節(jié)點(diǎn)，還沒有連接分片數(shù)據(jù)節(jié)點(diǎn)，需要添加第一套分片副本集和第二套分片副本集。
  • 在路由節(jié)點(diǎn)上進(jìn)行分片配置，添加分片的語法：sh.addShard("IP:Port") 
[root@dev mongodb]# /opt/mongodb/bin/mongo --port 27017
mongos> show dbs
admin   0.000GB
config  0.000GB
mongos> use aadb
switched to db aadb
mongos> db.aa.insert({aa:"aa"})
WriteCommandError({
    "ok" : 0,
    "errmsg" : "unable to initialize targeter for write op for collection aadb.aa :: caused by :: Database aadb could not be created :: caused by :: No shards found",
    "code" : 70,
    "codeName" : "ShardNotFound",
    "operationTime" : Timestamp(1610430671, 2),
    "$clusterTime" : {
        "clusterTime" : Timestamp(1610430671, 2),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    }
})
mongos> sh.addShard("myshardrs01/公網(wǎng)ip:27018,公網(wǎng)ip:27118,公網(wǎng)ip:27218")
{
    "shardAdded" : "myshardrs01",
    "ok" : 1,
    "operationTime" : Timestamp(1610431326, 5),
    "$clusterTime" : {
        "clusterTime" : Timestamp(1610431326, 5),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    }
}
mongos> sh.addShard("myshardrs02/公網(wǎng)ip:27318,公網(wǎng)ip:27418,公網(wǎng)ip:27518")
{
    "shardAdded" : "myshardrs02",
    "ok" : 1,
    "operationTime" : Timestamp(1610431577, 3),
    "$clusterTime" : {
        "clusterTime" : Timestamp(1610431577, 3),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    }
}
mongos> sh.status()
--- Sharding Status --- 
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("5ffd13a86c140163a85aa531")
  }
  shards:
        {  "_id" : "myshardrs01",  "host" : "myshardrs01/公網(wǎng)ip:27018,公網(wǎng)ip:27118",  "state" : 1 }
        {  "_id" : "myshardrs02",  "host" : "myshardrs02/公網(wǎng)ip:27318,公網(wǎng)ip:27418",  "state" : 1 }
  active mongoses:
        "4.4.3" : 1
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours: 
                39 : Success
  databases:
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
                config.system.sessions
                        shard key: { "_id" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                myshardrs01 985
                                myshardrs02 39
                        too many chunks to print, use verbose if you want to force print
  • 提示：若添加分片失敗，則需要先手動(dòng)移除分片、檢查添加分片信息的正確性，再次添加分片。注意：若只剩下最后一個(gè) shard，是無法被移除的；移除時(shí)會(huì)自動(dòng)轉(zhuǎn)移分片數(shù)據(jù)，需要一個(gè)時(shí)間過程。完成后，再次執(zhí)行刪除分片命令才能真正刪除。
mongos> use admin
mongos> db.runCommand( { removeShard: "myshardrs02" } )
  • 開啟分片功能：①在mongos服務(wù)器上的articledb數(shù)據(jù)庫配置sharding：sh.enableSharding("庫名")；②對(duì)集合進(jìn)行分片：sh.shardCollection(namespace, key, unique)，如：sh.shardCollection("庫名.集合名", {"key":1})
參數(shù) 類型 描述
namespace string 要分片共享的目標(biāo)集合的命名空間，格式：<database>.<collection>
key document 用作分片鍵的索引規(guī)范文檔。shard鍵決定MongoDB如何在shard之間分發(fā)文檔。除非集合為空，否則索引必須在shardCollection命令之前存在。若集合為空，則MongoDB在對(duì)集合進(jìn)行分片之前創(chuàng)建索引，前提是支持片鍵的索引不存在。簡(jiǎn)單來說，其由包含字段和該字段的索引遍歷方向的文檔組成。
unique boolean 當(dāng)值為true的情況下，片鍵字段上會(huì)限制為唯一索引。哈希策略片鍵不支持唯一索引。默認(rèn)是false。
  • 對(duì)集合進(jìn)行分片時(shí)，需要選擇一個(gè)片鍵（Shard Key），shard key是每條記錄都必須包含的，且建立了索引的單個(gè)字段或復(fù)合字段，MongoDB按照片鍵將數(shù)據(jù)劃分到不同的數(shù)據(jù)塊中，并將數(shù)據(jù)塊均衡地分布到所有分片中。為了按照片鍵劃分?jǐn)?shù)據(jù)塊，MongoDB使用基于哈希的分片方式（隨機(jī)平均分配）或者基于范圍的分片方式（數(shù)值大小分配） 。
  • 分片規(guī)則一：哈希策略。對(duì)于基于哈希的分片，MongoDB計(jì)算一個(gè)字段的哈希值，并用這個(gè)哈希值來創(chuàng)建數(shù)據(jù)塊。在使用基于哈希分片的系統(tǒng)中，擁有"相近"片鍵的文檔很可能不會(huì)存儲(chǔ)在同一個(gè)數(shù)據(jù)塊中，因此數(shù)據(jù)的分離性更好一些。如：使用nickname作為片鍵，根據(jù)其值的哈希值進(jìn)行數(shù)據(jù)分片：sh.shardCollection("articledb.comment",{"nickname":"hashed"})。
  • 分片規(guī)則二：范圍策略。對(duì)于基于范圍的分片，MongoDB按照片鍵的范圍把數(shù)據(jù)分成不同部分。假設(shè)有一個(gè)數(shù)字的片鍵：想象一個(gè)從負(fù)無窮到正無窮的直線，每一個(gè)片鍵的值都在直線上畫了一個(gè)點(diǎn)。MongoDB把這條直線劃分為更短的不重疊的片段，并稱之為數(shù)據(jù)塊，每個(gè)數(shù)據(jù)塊包含了片鍵在一定范圍內(nèi)的數(shù)據(jù)。在使用片鍵做范圍劃分的系統(tǒng)中，擁有"相近"片鍵的文檔很可能存儲(chǔ)在同一個(gè)數(shù)據(jù)塊中，因此也會(huì)存儲(chǔ)在同一個(gè)分片中。如：使用作者年齡字段作為片鍵，按照點(diǎn)贊數(shù)的值進(jìn)行分片：sh.shardCollection("articledb.author",{"age":1})
  • 注意：①一個(gè)集合只能指定一個(gè)片鍵。②一旦對(duì)一個(gè)集合分片，分片鍵和分片值就不可改變。 如：不能給集合選擇不同的分片鍵、不能更新分片鍵的值。③根據(jù)age索引進(jìn)行分配數(shù)據(jù)。
mongos> sh.enableSharding("articledb")
{
    "ok" : 1,
    "operationTime" : Timestamp(1610432042, 8),
    "$clusterTime" : {
        "clusterTime" : Timestamp(1610432042, 8),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    }
}
mongos> sh.shardCollection("articledb.comment",{"nickname":"hashed"})
{
    "collectionsharded" : "articledb.comment",
    "collectionUUID" : UUID("f3676c7e-25be-4364-8216-5a8c0338512f"),
    "ok" : 1,
    "operationTime" : Timestamp(1610433360, 5),
    "$clusterTime" : {
        "clusterTime" : Timestamp(1610433360, 5),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    }
}
mongos> sh.shardCollection("articledb.author",{"age":1})
{
    "collectionsharded" : "articledb.author",
    "collectionUUID" : UUID("e993611c-f30c-4871-a488-dffbd56d5413"),
    "ok" : 1,
    "operationTime" : Timestamp(1610433415, 12),
    "$clusterTime" : {
        "clusterTime" : Timestamp(1610433415, 12),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    }
}
mongos> sh.status()
--- Sharding Status --- 
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("5ffd13a86c140163a85aa531")
  }
  shards:
        {  "_id" : "myshardrs01",  "host" : "myshardrs01/公網(wǎng)ip:27018,公網(wǎng)ip:27118",  "state" : 1 }
        {  "_id" : "myshardrs02",  "host" : "myshardrs02/公網(wǎng)ip:27318,公網(wǎng)ip:27418",  "state" : 1 }
  active mongoses:
        "4.4.3" : 1
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours: 
                512 : Success
  databases:
        {  "_id" : "articledb",  "primary" : "myshardrs02",  "partitioned" : true,  "version" : {  "uuid" : UUID("03cc6699-146c-46f3-b266-5f43b8d65a39"),  "lastMod" : 1 } }
                articledb.author
                        shard key: { "age" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                myshardrs02 1
                        { "age" : { "$minKey" : 1 } } -->> { "age" : { "$maxKey" : 1 } } on : myshardrs02 Timestamp(1, 0) 
                articledb.comment
                        shard key: { "nickname" : "hashed" }
                        unique: false
                        balancing: true
                        chunks:
                                myshardrs01 2
                                myshardrs02 2
                        { "nickname" : { "$minKey" : 1 } } -->> { "nickname" : NumberLong("-4611686018427387902") } on : myshardrs01 Timestamp(1, 0) 
                        { "nickname" : NumberLong("-4611686018427387902") } -->> { "nickname" : NumberLong(0) } on : myshardrs01 Timestamp(1, 1) 
                        { "nickname" : NumberLong(0) } -->> { "nickname" : NumberLong("4611686018427387902") } on : myshardrs02 Timestamp(1, 2) 
                        { "nickname" : NumberLong("4611686018427387902") } -->> { "nickname" : { "$maxKey" : 1 } } on : myshardrs02 Timestamp(1, 3) 
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
                config.system.sessions
                        shard key: { "_id" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                myshardrs01 512
                                myshardrs02 512
                        too many chunks to print, use verbose if you want to force print
  • 基于范圍的分片方式與基于哈希的分片方式性能對(duì)比：
    • 前者提供了更高效的范圍查詢，給定一個(gè)片鍵的范圍，分發(fā)路由可以很簡(jiǎn)單地確定哪個(gè)數(shù)據(jù)塊存儲(chǔ)了請(qǐng)求需要的數(shù)據(jù)，并將請(qǐng)求轉(zhuǎn)發(fā)到相應(yīng)的分片中。不過基于范圍的分片會(huì)導(dǎo)致數(shù)據(jù)在不同分片上的不均衡，有時(shí)帶來的消極作用會(huì)大于查詢性能的積極
      作用。比如：若片鍵所在的字段是線性增長(zhǎng)的，則一定時(shí)間內(nèi)所有請(qǐng)求都會(huì)落到某個(gè)固定的數(shù)據(jù)塊中，最終導(dǎo)致分布在同一個(gè)分片中。在這種情況下，一小部分分片承載了集群大部分的數(shù)據(jù)，系統(tǒng)并不能很好地進(jìn)行擴(kuò)展。
    • 與此相比，后者以范圍查詢性能的損失為代價(jià)，保證了集群中數(shù)據(jù)的均衡。哈希值的隨機(jī)性使數(shù)據(jù)隨機(jī)分布在每個(gè)數(shù)據(jù)塊中，因此也隨機(jī)分布在不同分片中。但是正因?yàn)殡S機(jī)性，一個(gè)范圍查詢很難確定應(yīng)該請(qǐng)求哪些分片。通常為了返回需要的結(jié)果，需要請(qǐng)求所有分片。
    • 如無特殊情況，一般推薦使用Hash Sharding。使用_id作為片鍵是一個(gè)不錯(cuò)的選擇，因?yàn)樗緛砭陀械?，所以可以使用?shù)據(jù)文檔_id的哈希作為片鍵。這個(gè)方案使得讀和寫都能夠平均分布，且它能夠保證每個(gè)文檔都有不同的片鍵，所以數(shù)據(jù)塊能夠分得很精細(xì)。理想化的 shard key 可以讓 documents 均勻地在集群中分布：
  • 顯示集群的詳細(xì)信息：
mongos> db.printShardingStatus()
  • 查看均衡器是否工作（需要重新均衡時(shí)系統(tǒng)才會(huì)自動(dòng)啟動(dòng)，不用管它）：
mongos> sh.isBalancerRunning()
false
  • 查看當(dāng)前 Balancer狀態(tài)：
mongos> sh.getBalancerState()
true

分片后插入數(shù)據(jù)測(cè)試

  • 測(cè)試一（哈希規(guī)則）：登錄mongs后，測(cè)試向comment集合插入1000條數(shù)據(jù)。提示：for插入語句是js的語法，因?yàn)閙ongo的shell是一個(gè)JavaScript的shell。注意：從路由上插入的數(shù)據(jù)必須包含片鍵，否則無法插入！
mongos> use articledb
switched to db articledb
mongos> for(var i=1;i<=1000;i++){db.comment.insert({_id:i+"",nickname:"BoBo"+i})}
WriteResult({ "nInserted" : 1 })
mongos> db.comment.count()
1000
  • 分別登陸兩個(gè)片的主節(jié)點(diǎn)，統(tǒng)計(jì)文檔數(shù)量：
  • 第一個(gè)分片副本集：/opt/mongodb/bin/mongo --port 27018 
myshardrs01:PRIMARY> use articledb
switched to db articledb
myshardrs01:PRIMARY> db.comment.count()
507
  • 第二個(gè)分片副本集：/opt/mongodb/bin/mongo --port 27318 
myshardrs02:PRIMARY> use articledb
switched to db articledb
myshardrs02:PRIMARY> db.comment.count()
493
  • 從結(jié)果可以看到，1000條數(shù)據(jù)近似均勻地分布到了2個(gè)shard上，其是根據(jù)片鍵的哈希值分配的。這種分配方式非常易于水平擴(kuò)展：一旦數(shù)據(jù)存儲(chǔ)需要更大空間，可以直接再增加分片即可，同時(shí)提升了性能。
  • 使用db.comment.stats()查看單個(gè)集合的完整情況，mongos執(zhí)行該命令可以查看該集合的數(shù)據(jù)分片的情況。
  • 使用sh.status()查看本庫內(nèi)所有集合的分片信息。
  • 測(cè)試二（范圍規(guī)則）：登錄mongs后，向comment集合插入12057條數(shù)據(jù)做測(cè)試：
mongos> use config
switched to db config
mongos> db.settings.save({ _id:"chunksize", value: 1 })
WriteResult({ "nMatched" : 0, "nUpserted" : 1, "nModified" : 0, "_id" : "chunksize" })
mongos> for(var i=1;i<=20000;i++){db.author.save({"name":"BoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBoBo"+i,"age":NumberInt(i%120)})}
mongos> db.author.count()
12057
  • 分別登陸兩個(gè)片的主節(jié)點(diǎn)，統(tǒng)計(jì)文檔數(shù)量：
  • 第一個(gè)分片副本集：/opt/mongodb/bin/mongo --port 27018 
myshardrs01:PRIMARY> use articledb
switched to db articledb
myshardrs01:PRIMARY> db.author.count()
99
  • 第二個(gè)分片副本集：/opt/mongodb/bin/mongo --port 27318 
myshardrs02:PRIMARY> use articledb
switched to db articledb
myshardrs02:PRIMARY> db.author.count()
11958
  • 若查看狀態(tài)發(fā)現(xiàn)沒有分片，則可能是以下原因造成的：①系統(tǒng)繁忙，正在分片中；②數(shù)據(jù)塊（chunk）沒有填滿，默認(rèn)的數(shù)據(jù)塊尺寸（chunksize）是64M，填滿后才會(huì)考慮向其它片的數(shù)據(jù)塊填充數(shù)據(jù)。因此，為了方便測(cè)試，可以將其改為1M，操作如下：
use config
db.settings.save( { _id:"chunksize", value: 1 } )

第二個(gè)路由節(jié)點(diǎn)的創(chuàng)建和連接

  • 準(zhǔn)備存放數(shù)據(jù)、日志和配置文件的目錄：
mkdir -p /opt/mongodb/sharded_cluster/mymongos_27117/logs \ &
mkdir -p /opt/mongodb/sharded_cluster/mymongos_27117/conf
  • 新建一個(gè)配置文件：vim /opt/mongodb/sharded_cluster/mymongos_27117/conf/mongos.conf 
 systemLog:
    destination: file
    path: /opt/mongodb/sharded_cluster/mymongos_27117/logs/mongod.log
    logAppend: true
 processManagement:
    fork: true
    pidFilePath: /opt/mongodb/sharded_cluster/mymongos_27117/logs/mongod.pid
 net:
    bindIp: localhost,192.168.0.128
    port: 27117
 sharding:
    configDB: myconfigrs/公網(wǎng)ip:27019,公網(wǎng)ip:27119,公網(wǎng)ip:27219
  • 啟動(dòng)mongos服務(wù)，若啟動(dòng)失敗，則查看 logs目錄下的日志。
[root@dev mongodb]# /opt/mongodb/bin/mongos -f /opt/mongodb/sharded_cluster/mymongos_27117/conf/mongos.conf
about to fork child process, waiting until server is ready for connections.
forked process: 27783
child process started successfully, parent exiting
使用Compass 連接分片集群
  • 若在搭建分片時(shí)有操作失敗或配置有問題，需要重新來過，可以進(jìn)行如下操作：
  • 第一步：查詢出所有測(cè)試服務(wù)節(jié)點(diǎn)的進(jìn)程：ps -ef | grep mongo，根據(jù)列舉的進(jìn)程編號(hào)，依次中斷進(jìn)程：kill -2 進(jìn)程編號(hào)。
  • 第二步：清除所有節(jié)點(diǎn)的數(shù)據(jù)：
rm -rf /opt/mongodb/sharded_cluster/myconfigrs_27019/data/db/*.* \ &
rm -rf /opt/mongodb/sharded_cluster/myconfigrs_27119/data/db/*.* \ &
rm -rf /opt/mongodb/sharded_cluster/myconfigrs_27219/data/db/*.* \ &
rm -rf /opt/mongodb/sharded_cluster/myshardrs01_27018/data/db/*.* \ &
rm -rf /opt/mongodb/sharded_cluster/myshardrs01_27118/data/db/*.* \ &
rm -rf /opt/mongodb/sharded_cluster/myshardrs01_27218/data/db/*.* \ &
rm -rf /opt/mongodb/sharded_cluster/myshardrs02_27318/data/db/*.* \ &
rm -rf /opt/mongodb/sharded_cluster/myshardrs02_27418/data/db/*.* \ &
rm -rf /opt/mongodb/sharded_cluster/myshardrs02_27518/data/db/*.* \ &
rm -rf /opt/mongodb/sharded_cluster/mymongos_27017/data/db/*.* \ &
rm -rf /opt/mongodb/sharded_cluster/mymongos_27117/data/db/*.*
  • 第三步：查看或修改有問題的配置。
  • 第四步：依次啟動(dòng)所有節(jié)點(diǎn)，不包括路由節(jié)點(diǎn)：
/opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27018/conf/mongod.conf
/opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27118/conf/mongod.conf
/opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27218/conf/mongod.conf
/opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27318/conf/mongod.conf
/opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27418/conf/mongod.conf
/opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myshardrs01_27518/conf/mongod.conf
/opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myconfigrs_27019/conf/mongod.conf
/opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myconfigrs_27119/conf/mongod.conf
/opt/mongodb/bin/mongod -f /opt/mongodb/sharded_cluster/myconfigrs_27219/conf/mongod.conf
  • 第五步：對(duì)兩個(gè)數(shù)據(jù)分片副本集和一個(gè)配置副本集進(jìn)行初始化和相關(guān)配置。
  • 第六步：檢查路由mongos的配置，并啟動(dòng)mongos服務(wù)：
/opt/mongodb/bin/mongos -f /opt/mongodb/sharded_cluster/mymongos_27017/conf/mongos.conf
/opt/mongodb/bin/mongos -f /opt/mongodb/sharded_cluster/mymongos_27117/conf/mongos.conf
  • 第七步：mongo登錄mongos，在其上進(jìn)行相關(guān)操作。
最后編輯于 ：
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡(jiǎn)書系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容