GFS2 FROM SCRATCH

tested on Slackware Linux current (Aug 2020)

Corosync setup

totem {
    version: 2
    secauth: off
    cluster_name: pro5s
}

nodelist {
    node {
        ring0_addr: x.x.x.x
        nodeid: 1
    }
    node {
        ring0_addr: x.x.x.x
        nodeid: 2
    }
}

quorum {
    provider: corosync_votequorum
    two_node: 1
}

logging {
    to_syslog: yes
}

replicate the conf

scp /etc/corosync/corosync.conf pro5s2:/etc/corosync/

/etc/init.d/corosync start
netstat -lntup | grep corosync

DRBD setup

See drbd. Using v8 here as:

Requirements

zcat /proc/config.gz | grep GFS2

CONFIG_GFS2_FS=m
CONFIG_GFS2_FS_LOCKING_DLM=y
CONFIG_DLM=m

slackpkg install libtool

Build

#git clone https://pagure.io/gfs2-utils.git
#cd gfs2-utils/
wget https://pagure.io/gfs2-utils/archive/3.3.0/gfs2-utils-3.3.0.tar.gz
cd gfs2-utils-3.3.0/
./autogen.sh
./configure --disable-nls
make > ../make.gfs2-utils.log && echo BUILT
make install
cd ../

#git clone https://pagure.io/dlm.git
#cd dlm/
wget https://pagure.io/dlm/archive/dlm-4.0.9/dlm-dlm-4.0.9.tar.gz
cd dlm-dlm-4.0.9/
vi dlm_controld/Makefile

comment this out (don’t just switch USE_SD_NOTIFY to no, that will prevent udev to work properly afterwards)

#ifeq ($(USE_SD_NOTIFY),yes)
#       BIN_CFLAGS += $(shell $(PKG_CONFIG) --cflags libsystemd) \
#                     -DUSE_SD_NOTIFY
#       BIN_LDFLAGS += $(shell $(PKG_CONFIG) --libs libsystemd)
#endif

and proceed

make clean
make > ../dlm.log && echo BUILT
make install
cd ../

DLM

there is no quorum on a two-node cluster

mkdir /etc/dlm/
vi /etc/dlm/dlm.conf

enable_quorum = 0
enable_fencing = 0
log_debug = 1
debug_logfile = 1

ls -lF /var/log/dlm_controld/dlm_controld.log

Format once

only once from some node

drbdadm status
drbdadm primary res-data

time mkfs.gfs2 -j2 -t pro5s:data /dev/drbd0
fsck.gfs2 -fy /dev/drbd0

Ready to go

on all nodes

vi /etc/rc.d/rc.local

echo DRBD
/sbin/modprobe drbd
/sbin/modprobe drbd_transport_tcp
/etc/init.d/drbd start
/usr/local/sbin/drbdadm primary res-data
echo

echo GFS2/DLM
/etc/init.d/corosync start
/sbin/modprobe dlm
/sbin/modprobe gfs2
/bin/mount -t configfs none /sys/kernel/config
/usr/sbin/dlm_controld
sleep 2
/bin/mount -t gfs2 -o noatime,nodiratime,noquota /dev/drbd0 /data
echo

vi /etc/rc.d/rc.local_shutdown

echo GFS2/DLM DOWN
/bin/umount -R /data/
lsof | grep data1
/bin/umount -R /data1/
lsof | grep data2
/bin/umount -R /data2/
#/bin/umount -t gfs2 -a
/usr/bin/pkill dlm_controld
/sbin/rmmod gfs2 dlm
/etc/init.d/corosync stop
echo

echo DRBD DOWN
#/etc/init.d/drbd stop
/usr/local/sbin/drbdadm down all
#v9 /sbin/rmmod drbd_transport_tcp
/sbin/rmmod drbd
echo

STATUS

echo DRBD
drbdadm status
echo

echo DLM
pgrep -a dlm_controld
dlm_tool ls
dlm_tool status
echo

echo COROSYNC
/etc/init.d/corosync status
echo

echo GFS2
mount | grep gfs2
echo

Additional notes

more mount options for GFS2

rw,suid,dev,exec,nouser,async,_netdev   0 0
_netdev,noauto
acl

Note fs_passno HAS TO BE ZERO.

Troubleshooting

lsmod | grep gfs2
ls -lF /usr/lib/udev/rules.d/51-dlm.rules
ls -lF /dev/misc/dlm-control
pgrep -a dlm

drbdadm status
corosync-quorumtool

dlm_controld -D --enable_quorum_fencing 0

you are getting

umount: /data2: target is busy.

==>

fuser -vm /data2
fuser -vmMki /data2
lsof | grep data2

Resources

GFS2 https://en.wikipedia.org/wiki/GFS2

GFS Project Page https://sourceware.org/cluster/gfs/

dlm https://pagure.io/dlm

gfs2/dlm usage in cluster4 http://people.redhat.com/teigland/cluster4-gfs2-dlm.txt

history

The OpenGFS Project (2004…) http://opengfs.sourceforge.net/

tutorials

How to use gfs2/dlm directly http://people.redhat.com/teigland/cluster4-dlm.txt

How to Create a GFS2 Formatted Cluster File System https://www.thegeekdiary.com/how-to-create-a-gfs2-formatted-cluster-file-system/

DE - Linux-HA: GFS mit DRBD und Pacemaker » ADMIN-Magazin https://www.admin-magazin.de/Das-Heft/2012/01/HA-Workshop-Teil-7-GFS-mit-DRBD-und-Pacemaker/(offset)/6

GFS2 in Pacemaker (Debian/Ubuntu) https://xahteiwi.eu/resources/hints-and-kinks/gfs2-pacemaker-debianubuntu/

Clustering with Pacemaker, DRBD and GFS2 on Bare-Metal servers in SoftLayer https://icicimov.github.io/blog/high-availability/Clustering-with-Pacemaker-DRBD-and-GFS2-on-Bare-Metal-servers-in-SoftLayer/

troubles

mount.gfs2 - GFS2 mount options https://linux.die.net/man/8/gfs2_mount

configfs - Userspace-driven kernel object configuration. https://www.kernel.org/doc/Documentation/filesystems/configfs/configfs.txt

target is busy https://stackoverflow.com/questions/7878707/how-to-unmount-a-busy-device

troubles - redhat kernel version

Re: [Linux-cluster] sparse-file clone breaks on GFS2 https://www.mail-archive.com/linux-cluster@redhat.com/msg13797.html

Re: [Linux-cluster] sparse-file clone breaks on GFS2 https://www.mail-archive.com/linux-cluster@redhat.com/msg13798.html

where to find the kernel source

Where does the RHEL 7 source code live? https://lwn.net/Articles/603865/

Can we git clone the redhat kernel source code and see the changes made by them? https://stackoverflow.com/questions/41314978/can-we-git-clone-the-redhat-kernel-source-code-and-see-the-changes-made-by-them

I Need the Kernel Source https://wiki.centos.org/HowTos/I_need_the_Kernel_Source

I Need to Build a Custom Kernel https://wiki.centos.org/HowTos/Custom_Kernel

Sources https://wiki.centos.org/Sources

4. CentOSPlus kernel differences https://wiki.centos.org/AdditionalResources/Repositories/CentOSPlus#CentOSPlus_kernel_differences

I Need the Kernel Source https://wiki.centos.org/HowTos/I_need_the_Kernel_Source

source repos

Welcome to CentOS Git server https://git.centos.org/

(source have moved) http://ftp.redhat.com/pub/redhat/linux/enterprise/7Server/en/os/README https://git.centos.org/project/rpms https://access.redhat.com/site/downloads/

kernelim / linux https://github.com/kernelim/linux/tree/centos8

use the source, luke

Rebuild a Source RPM https://wiki.centos.org/HowTos/RebuildSRPM

c8s/f/SPECS/kernel.spec https://git.centos.org/rpms/kernel/raw/c8s/f/SPECS/kernel.spec

RPM http://centos.mirrors.proxad.net/8.2.2004/BaseOS/x86_64/os/Packages/ http://mirror.nsc.liu.se/centos-store/8.2.2004/BaseOS/ http://vault.centos.org/8.2.2004/BaseOS/Source/SPackages/

extract RPM https://www.cyberciti.biz/tips/how-to-extract-an-rpm-package-without-installing-it.html https://en.wikipedia.org/wiki/RPM_Package_Manager


HOME | GUIDES | LECTURES | LAB | SMTP HEALTH | HTML5 | CONTACT
Copyright © 2024 Pierre-Philipp Braun