1.環(huán)境及設(shè)置：

架構(gòu)圖

ceph-node1 192.168.0.232 這個(gè)節(jié)點(diǎn)同時(shí)用作ceph-deploy
ceph-node2 192.168.0.250
ceph-node3 192.168.0.220
操作系統(tǒng)： centos7.4
內(nèi)核：3.10.0-693.21.1.el7.x86_64
關(guān)閉防火墻，selinux

安裝ceph-deploy：

cat >/etc/yum.repos.d/ceph.repo
[ceph-noarch]
name=Ceph noarch packages
baseurl=https://download.ceph.com/rpm/el7/noarch
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://download.ceph.com/keys/release.asc

更新操作系統(tǒng)并安裝ceph-deploy，其實(shí)兩臺(tái)也需要update，不需要安裝ceph-deploy

sudo yum update
sudo yum install ceph-deploy

在所有節(jié)點(diǎn)創(chuàng)建ceph用戶

sudo useradd -d /home/ceph-node -m ceph-node
sudo passwd ceph-node
echo "ceph-node ALL = (root) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/ceph-node
sudo chmod 0440 /etc/sudoers.d/ceph-node

在ceph-deploy節(jié)點(diǎn)創(chuàng)建密鑰并分發(fā)到各節(jié)點(diǎn)

ssh-keygen
ssh-copy-id ceph-node@ceph-node2
ssh-copy-id ceph-node@ceph-node1

修改ssh配置文件默認(rèn)ssh連接是用ceph-node用戶

Host ceph-node1
   Hostname ceph-node1
   User ceph-node
Host ceph-node2
   Hostname ceph-node2
   User ceph-node
Host ceph-node3
   Hostname ceph-node3
   User ceph-node

每個(gè)節(jié)點(diǎn)設(shè)置時(shí)間同步

yum -y install ntpdate
crontab -e
*/5 * * * * /usr/sbin/ntpdate ntp1.aliyun.com 

2.安裝ceph

切換到ceph-deploy節(jié)點(diǎn)

mkdir /root/my-cluster
cd /root/my-cluster

如果安裝過(guò)程有錯(cuò)誤或者失敗可以用以下命令從頭再來(lái)

ceph-deploy purge {ceph-node} [{ceph-node}]
ceph-deploy purgedata {ceph-node} [{ceph-node}]
ceph-deploy forgetkeys
rm ceph.*

創(chuàng)建集群

ceph-deploy new ceph-node1 ceph-node2 ceph-node3
vim ceph.conf #加入公共網(wǎng)絡(luò)網(wǎng)段
public network = 192.168.0.0/24

安裝ceph

ceph-deploy install --release luminous ceph-node1 ceph-node2 ceph-node3

部署mon生成密鑰文件

ceph-deploy mon create-initial

發(fā)送配置文件

ceph-deploy admin ceph-node1 ceph-node2 ceph-node3

創(chuàng)建mgr節(jié)點(diǎn)，mgr需要高可用所以創(chuàng)建三個(gè)節(jié)點(diǎn)

ceph-deploy mgr create ceph-node1
ceph-deploy mgr create ceph-node2
ceph-deploy mgr create ceph-node3

創(chuàng)建osd

ceph-deploy osd create --data /dev/sdb ceph-node1
ceph-deploy osd create --data /dev/sdc ceph-node1
ceph-deploy osd create --data /dev/sdd ceph-node1
ceph-deploy osd create --data /dev/sdb ceph-node2
ceph-deploy osd create --data /dev/sdc ceph-node2
ceph-deploy osd create --data /dev/sdd ceph-node2
ceph-deploy osd create --data /dev/sdb ceph-node3
ceph-deploy osd create --data /dev/sdc ceph-node3
ceph-deploy osd create --data /dev/sdd ceph-node3

使用ceph -s查看集群是否健康，osd是否加入節(jié)點(diǎn)

狀態(tài)圖

添加額外mon節(jié)點(diǎn)，mon節(jié)點(diǎn)也需要高可用

ceph-deploy mon add ceph-node2 ceph-node3

至此ceph集群安裝完成

3.安裝常見錯(cuò)誤

• 錯(cuò)誤：

[ceph_deploy][ERROR ] RuntimeError: connecting to host: n1 resulted in errors: IOError cannot send (already closed?)
解決方法：
編輯visudo注釋下面的
Defaults    requiretty

• 錯(cuò)誤：

[n2][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
解決方法：
在ceph.conf添加
public network = 192.168.122.0/24

• 錯(cuò)誤：

ceph-deploy安裝的時(shí)候到y(tǒng)um安裝那報(bào)錯(cuò)
解決：可能是源有問(wèn)題執(zhí)行yum -y remove centos-release-ceph