GFS2 FROM SCRATCH

tested on Slackware Linux current (Aug 2020)

Corosync setup

totem {
    version: 2
    secauth: off
    cluster_name: pro5s
}

nodelist {
    node {
        ring0_addr: x.x.x.x
        nodeid: 1
    }
    node {
        ring0_addr: x.x.x.x
        nodeid: 2
    }
}

quorum {
    provider: corosync_votequorum
    two_node: 1
}

logging {
    to_syslog: yes
}

replicate the conf

scp /etc/corosync/corosync.conf pro5s2:/etc/corosync/

/etc/init.d/corosync start
netstat -lntup | grep corosync

DRBD setup

see drbd

Requirements

zcat /proc/config.gz | grep GFS2

CONFIG_GFS2_FS=m
CONFIG_GFS2_FS_LOCKING_DLM=y
CONFIG_DLM=m

slackpkg install libtool

Build

git clone https://pagure.io/gfs2-utils.git
cd gfs2-utils/
./autogen.sh
./configure --disable-nls
make > ../make.gfs2-utils.log && echo BUILT
make install
cd ../

git clone https://pagure.io/dlm.git
cd dlm/
vi dlm_controld/Makefile

comment this out (don’t just switch USE_SD_NOTIFY to no, that will prevent udev to work properly afterwards)

#ifeq ($(USE_SD_NOTIFY),yes)
#       BIN_CFLAGS += $(shell $(PKG_CONFIG) --cflags libsystemd) \
#                     -DUSE_SD_NOTIFY
#       BIN_LDFLAGS += $(shell $(PKG_CONFIG) --libs libsystemd)
#endif

and proceed

make clean
make > ../dlm.log && echo BUILT
make install
cd ../

DLM

mkdir /etc/dlm/
vi /etc/dlm/dlm.conf

enable_fencing=0
log_debug=1
debug_logfile=1

ls -lF /var/log/dlm_controld/dlm_controld.log

Format once

only once from some node

drbdadm status
drbdadm primary res-data

mkfs.gfs2 -j2 -t pro5s:data /dev/drbd0
fsck.gfs2 -fy /dev/drbd0

Ready to go

on all nodes

vi /etc/rc.d/rc.local

echo DRBD
/sbin/modprobe drbd
/sbin/modprobe drbd_transport_tcp
/etc/init.d/drbd start
/usr/local/sbin/drbdadm primary res-data
echo

echo GFS2/DLM
/etc/init.d/corosync start
/sbin/modprobe dlm
/sbin/modprobe gfs2
/bin/mount -t configfs none /sys/kernel/config
/usr/sbin/dlm_controld
sleep 2
/bin/mount -t gfs2 -o noatime,nodiratime,noquota /dev/drbd0 /data
echo

vi /etc/rc.d/rc.local_shutdown

echo GFS2/DLM DOWN
/bin/umount -t gfs2 -a
/usr/bin/pkill dlm_controld
/sbin/rmmod gfs2 dlm
/etc/init.d/corosync stop
echo

echo DRBD DOWN
/etc/init.d/drbd stop
/sbin/rmmod drbd_transport_tcp
/sbin/rmmod drbd
echo

Additional notes

more mount options for GFS2

rw,suid,dev,exec,nouser,async,_netdev   0 0
_netdev,noauto
acl

Note fs_passno HAS TO BE ZERO.

Troubleshooting

lsmod | grep gfs2
ls -lF /usr/lib/udev/rules.d/51-dlm.rules
ls -lF /dev/misc/dlm-control
pgrep -a dlm

drbdadm status
corosync-quorumtool

dlm_controld -D --enable_quorum_fencing 0

Resources

GFS2 https://en.wikipedia.org/wiki/GFS2

GFS Project Page https://sourceware.org/cluster/gfs/

dlm https://pagure.io/dlm

gfs2/dlm usage in cluster4 http://people.redhat.com/teigland/cluster4-gfs2-dlm.txt

https://linux.die.net/man/8/gfs2_mount

https://www.thegeekdiary.com/how-to-create-a-gfs2-formatted-cluster-file-system/

https://www.admin-magazin.de/Das-Heft/2012/01/HA-Workshop-Teil-7-GFS-mit-DRBD-und-Pacemaker/(offset)/6

https://xahteiwi.eu/resources/hints-and-kinks/gfs2-pacemaker-debianubuntu/

configfs - Userspace-driven kernel object configuration. https://www.kernel.org/doc/Documentation/filesystems/configfs/configfs.txt

DRAFTS

CRMSH

crm ra info ocf:pacemaker:controld
crm configure property stonith-enabled=false
crm configure show

crm
configure primitive gfs ocf:heartbeat:Filesystem params device="/dev/drbd0" directory="/data" fstype="gfs2"
...

configure primitive p_dlm_controld ocf:pacemaker:controld \
  params daemon="dlm_controld.pcmk" \
  op start interval="0" timeout="90" \
  op stop interval="0" timeout="100" \
  op monitor interval="10"

PCS

pcs cluster status

pcs resource create clusterfs Filesystem device="/dev/drbd0" directory="/data" fstype="gfs2" "options=noatime" op monitor interval=10s on-fail=fence clone interleave=true

HOME | GUIDES | BENCHMARKS | html