Setting up a Ceph File System

MDS nodes

ceph-deploy mds create ceph1 ceph2 ceph3

Note. MDS are inactive until you create FS pools.

FS pools

Create the data and metadata pools and define the PG,

ceph osd pool create cephfs_data 256
ceph osd pool create cephfs_metadata 256

Enable Ceph/FS on the two pools,

ceph fs new cephfs cephfs_metadata cephfs_data
ceph fs ls

Note. MDS are now showing up as active/standby,

ceph -s
ceph mds stat

Mounting a Ceph/FS

as ceph@admin

Install a Ceph on the clients (if not already done),

ceph-deploy install client1 client2 client3

as root@client1,2,3

Prepare the key file,

cd /etc/ceph/
cat ceph.client.admin.keyring
vi ceph.key


chmod 400 ceph.key

Mount each client against the monitors, here the nodes themselfs,

#nmap -p 6789 ceph1
#nmap -p 6789 ceph2
#nmap -p 6789 ceph3

mkdir /datafs/
touch /datafs/NOT_MOUNTED

#mount.ceph ceph1,ceph2,ceph3:/ /datafs -o name=admin,secretfile=/etc/ceph/ceph.key,noatime

vi /etc/fstab

ceph1,ceph2,ceph3:/ /datafs ceph name=admin,secretfile=/etc/ceph/ceph.key,noatime,_netdev

mount /datafs/

cd /datafs/
ls -alhF
echo ok > test

as root@admin


dsh -e -g cepha date
dsh -e -g ceph grep ^ceph /etc/fstab
dsh -e -g ceph "mount | grep datafs"
dsh -e -g ceph "df -h | grep datafs"