Fsl's Markdown site

mon是ceph的最重要的组件，也是在搭建一个ceph集群的第一个组件，所以我们需要引导一个mon起来再进行后面的步骤，mon的引导需要以下几个东西：

Unique Identifier（fsid）：整个集群的唯一标识符。
Cluster Name：集群名，不包含空字符。
Monitor Name：顾名思义，一般使用主机名。
Monitor Map：fsid, the cluster name (or uses the default), and at least one host name and its IP address.
Monitor Keyring：mon通信时需要携带的secret
Administrator Keyring：后期使用ceph的cli工具的秘钥，ceph.admin

在安装的过程可能会出现某些命令不存在，需要手动去安装

登录到指定的机器
确保有一个文件夹存放ceph配置文件，在安装工具的时候会自动创建/etc/ceph
创建一个配置文件（/etc/ceph/ceph.conf）,并添加一行配置
shell
```
echo "[global]" >>/etc/ceph/ceph.conf
```

生成uuid（fsid）

shell

[root@ceph-0 /etc/ceph]# uuidgen 
8a212a0e-0fb5-4f4a-a1ee-53d7b3783a64

添加uuid到配置文件

shell

[root@ceph-0 /etc/ceph]# vim /etc/ceph/ceph.conf 
[root@ceph-0 /etc/ceph]# cat /etc/ceph/ceph.conf
[global]
fsid = 8a212a0e-0fb5-4f4a-a1ee-53d7b3783a64

添加初始mon到配置文件

shell

[root@ceph-0 /etc/ceph]# cat ceph.conf 
[global]
fsid = 8a212a0e-0fb5-4f4a-a1ee-53d7b3783a64
mon_initial_members = ceph-0

添加初始mon的ip地址到配置文件

shell

[root@ceph-0 /etc/ceph]# cat ceph.conf 
[global]
fsid = 8a212a0e-0fb5-4f4a-a1ee-53d7b3783a64
mon_initial_members = ceph-0
mon_host = 10.0.0.80

为你的集群创建一个keyring并且生成一个monitor keyring

shell

[root@ceph-0 /etc/ceph]# ceph-authtool --create-keyring /tmp/ceph.mon.keyring --gen-key -n mon. --cap mon 'allow *'
creating /tmp/ceph.mon.keyring
[root@ceph-0 /etc/ceph]# cat /tmp/ceph.mon.keyring 
[mon.]
	key = AQBSoRdmI/UwDxAAQVIXNuH6lI9dWyg8dioSAA==
	caps mon = "allow *"

生成Administrator Keyring，生成 client.admin 用户并将该用户添加到密钥中。

shell

[root@ceph-0 /etc/ceph]# ceph-authtool --create-keyring /etc/ceph/ceph.client.admin.keyring --gen-key -n client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *'
creating /etc/ceph/ceph.client.admin.keyring
[root@ceph-0 /etc/ceph]# cat /etc/ceph/ceph.client.admin.keyring
[client.admin]
	key = AQD0oRdmKcK1MBAAA8Mk/V84ZkafH4Ttmde/0Q==
	caps mds = "allow *"
	caps mgr = "allow *"
	caps mon = "allow *"
	caps osd = "allow *"

生成bootstrap-osd密匙环，生成client.bootstrap-osd。引导osd用户，并将该用户添加到keyring中。

shell

[root@ceph-0 /etc/ceph]# mkdir /var/lib/ceph/bootstrap-osd/
[root@ceph-0 /etc/ceph]# ceph-authtool --create-keyring /var/lib/ceph/bootstrap-osd/ceph.keyring --gen-key -n client.bootstrap-osd --cap mon 'profile bootstrap-osd' --cap mgr 'allow r'
[root@ceph-0 /etc/ceph]# cat /var/lib/ceph/bootstrap-osd/ceph.keyring
[client.bootstrap-osd]
	key = AQDXohdmcYkaFBAAvXvVGIpsFbjuYaNv4gEKgw==
	caps mgr = "allow r"
	caps mon = "profile bootstrap-osd"

将生成的密钥添加到ceph.mon.keyring中

shell

[root@ceph-0 /etc/ceph]# sudo ceph-authtool /tmp/ceph.mon.keyring --import-keyring /etc/ceph/ceph.client.admin.keyring
importing contents of /etc/ceph/ceph.client.admin.keyring into /tmp/ceph.mon.keyring
[root@ceph-0 /etc/ceph]# sudo ceph-authtool /tmp/ceph.mon.keyring --import-keyring /var/lib/ceph/bootstrap-osd/ceph.keyring
importing contents of /var/lib/ceph/bootstrap-osd/ceph.keyring into /tmp/ceph.mon.keyring
[root@ceph-0 /etc/ceph]# cat /tmp/ceph.mon.keyring
[mon.]
	key = AQBSoRdmI/UwDxAAQVIXNuH6lI9dWyg8dioSAA==
	caps mon = "allow *"
[client.admin]
	key = AQD0oRdmKcK1MBAAA8Mk/V84ZkafH4Ttmde/0Q==
	caps mds = "allow *"
	caps mgr = "allow *"
	caps mon = "allow *"
	caps osd = "allow *"
[client.bootstrap-osd]
	key = AQDXohdmcYkaFBAAvXvVGIpsFbjuYaNv4gEKgw==
	caps mgr = "allow r"
	caps mon = "profile bootstrap-osd"

更改ceph.mon.keyring的所有者

shell

sudo chown ceph:ceph /tmp/ceph.mon.keyring

使用主机名，主机IP地址和FSID生成monitor map。将其保存为 /tmp/monmap：

shell

[root@ceph-0 /etc/ceph]# monmaptool --create --add ceph-0 10.0.0.80 --fsid 8a212a0e-0fb5-4f4a-a1ee-53d7b3783a64 /tmp/monmap
monmaptool: monmap file /tmp/monmap
monmaptool: set fsid to 8a212a0e-0fb5-4f4a-a1ee-53d7b3783a64
monmaptool: writing epoch 0 to /tmp/monmap (1 monitors)

在mon上创建一个默认数据目录(或多个目录)。
shell
```
sudo -u ceph mkdir /var/lib/ceph/mon/ceph-ceph-0 -p
```

读取秘钥信息，生成monfs，为启动做准备

shell

sudo -u ceph ceph-mon --mkfs -i ceph-0 --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring

编写默认配置

shell

[root@ceph-0 /etc/ceph]# cat ceph.conf 
[global]
fsid = 8a212a0e-0fb5-4f4a-a1ee-53d7b3783a64
mon_initial_members = ceph-0
mon_host = 10.0.0.80
public_network = 10.0.0.0/24
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
osd_pool_default_size = 3
osd_pool_default_min_size = 2
osd_pool_default_pg_num = 333
osd_crush_chooseleaf_type = 1

使用systemd启动mon
shell
```
systemctl start ceph-mon@ceph-0
```
确保防火墙对应端口开启（3300和6789）

验证集群健康状态

shell

[root@ceph-0 /etc/ceph]# ceph -s
  cluster:
    id:     8a212a0e-0fb5-4f4a-a1ee-53d7b3783a64
    health: HEALTH_WARN
            1 monitors have not enabled msgr2
 
  services:
    mon: 1 daemons, quorum ceph-0 (age 61s)
    mgr: no daemons active
    osd: 0 osds: 0 up, 0 in
 
  data:
    pools:   0 pools, 0 pgs
    objects: 0 objects, 0 B
    usage:   0 B used, 0 B / 0 B avail
    pgs:

1 monitors have not enabled msgr2需要开启msgr2功能使用 ceph mon enable-msgr2

mgr: no daemons active,需要安装ceph-mgr，然后

[root@ceph-0 /etc/ceph]# ceph auth get-or-create mgr.ceph-0 mon 'allow profile mgr' osd 'allow *' mds 'allow *'
[mgr.ceph-0]
	key = AQCgrRdmeoNSLxAABAdYNXtL2id+5hljbcY83A==
#生成对应的秘钥，将生成的结果放到对应的目录中，目录为 /var/lib/ceph/mgr/{cluster-name}/{node-name}
[root@ceph-0 /var/lib/ceph]# sudo -u ceph mkdir /var/lib/ceph/mgr/ceph-ceph-0/
[root@ceph-0 /var/lib/ceph]# sudo -u ceph vim /var/lib/ceph/mgr/ceph-ceph-0/keyring 
[root@ceph-0 /var/lib/ceph]# cat /var/lib/ceph/mgr/ceph-ceph-0/keyring
[mgr.ceph-0]
	key = AQCgrRdmeoNSLxAABAdYNXtL2id+5hljbcY83A==
[root@ceph-0 /var/lib/ceph]# ceph-mgr -i ceph-0
[root@ceph-0 /var/lib/ceph]# ceph -s
  cluster:
    id:     8a212a0e-0fb5-4f4a-a1ee-53d7b3783a64
    health: HEALTH_OK
 
  services:
    mon: 1 daemons, quorum ceph-0 (age 21m)
    mgr: ceph-0(active, since 1.59473s)
    osd: 0 osds: 0 up, 0 in
 
  data:
    pools:   0 pools, 0 pgs
    objects: 0 objects, 0 B
    usage:   0 B used, 0 B / 0 B avail
    pgs: