setting up gfs2

requirements

make sure you’ve enable those at kernerl build-time

WARNING AS MODULE SO THE LOCKING_DLM OPTION BECOMES AVAILABLE

CONFIG_GFS2_FS=m
CONFIG_GFS2_FS_LOCKING_DLM=y
CONFIG_DLM=m

and check

zcat /proc/config.gz | grep CONFIG_GFS2_FS=
zcat /proc/config.gz | grep CONFIG_GFS2_FS_LOCKING_DLM=
zcat /proc/config.gz | grep CONFIG_DLM=

install

debian

apt install corosync dlm-controld gfs2-utils

slackware

    sbopkg -i libqb
    sbopkg -i corosync
    groupadd -g 226 haclient
    useradd -u 226 -g 226 -c "Pacemaker" -d / -s /bin/false hacluster
    sbopkg -i pacemaker
    sbopkg -i dlm

as for gfs2 utils, there’s no sbo package yet – see gfs2-source

setup

corosync

assuming two nodes

there is no quorum on a two-node cluster

cd /etc/corosync/

# debian
mv -i corosync.conf corosync.conf.dist

# slackware
ls -lhF corosync.conf.example*

vi corosync.conf

_assuming two nodes_

totem {
    version: 2
    secauth: off
    cluster_name: gfs
}

nodelist {
    node {
        ring0_addr: x.x.x.x
        nodeid: 1
    }
    node {
        ring0_addr: x.x.x.x
        nodeid: 2
    }
}

quorum {
    provider: corosync_votequorum
    two_node: 1
}

logging {
    to_syslog: yes
}

dlm

assuming two nodes

same here, no quorum

mkdir /etc/dlm/
vi /etc/dlm/dlm.conf

enable_quorum = 0
enable_fencing = 0
log_debug = 1
debug_logfile = 1

prepare resource

both nodes

lvcreate --virtualsize 10G --thin -n gfs2 thin/pool
vi /etc/drbd.d/gfs2.res

resource gfs2 {
        device /dev/drbd1;
        meta-disk internal;
        on risc1 {
                node-id   1;
                address   10.3.3.2:7001;
                disk      /dev/mapper/thin-gfs2;
        }
        on risc2 {
                node-id   2;
                address   10.3.3.3:7001;
                disk      /dev/mapper/thin-gfs2;
        }
        connection-mesh {
                hosts risc1 risc2;
        }
}

drbdadm create-md gfs2
drbdadm up gfs2

single node

drbdadm status gfs2 #disk:Inconsistent #peer-disk:Inconsistent drbdadm new-current-uuid –clear-bitmap gfs2 drbdadm status gfs2 #disk:UpToDate #peer-disk:UpToDate ```

ready to go

only once from some node

mkfs.gfs2 -j2 -t gfs:data /dev/drbd1

note gfs here matches corosync cluster_name – while the argument is an id for the file-system

mkdir /data_gfs2/
touch /data_gfs2/GFS2_NOT_MOUNTED
vi /etc/fstab

/dev/drbd1 /data_gfs2   gfs2    rw,noatime,nodiratime,_netdev,noauto,noquota,discard    0 0

note fs_passno HAS TO BE ZERO

more options

suid
dev
exec
nouser
async
acl
noauto

operations

debian

/etc/init.d/corosync restart
/etc/init.d/corosync status
netstat -lntup | grep corosync

/etc/init.d/dlm restart
/etc/init.d/dlm status
ls -lF /var/log/dlm_controld/dlm_controld.log

systemctl enable corosync
systemctl enable dlm

slackware

vi /etc/rc.d/rc.local

echo DRBD
/sbin/modprobe drbd
/sbin/modprobe drbd_transport_tcp
/etc/init.d/drbd start
/usr/local/sbin/drbdadm primary res-data
echo

echo GFS2/DLM
/etc/init.d/corosync start
/sbin/modprobe dlm
/sbin/modprobe gfs2
/bin/mount -t configfs none /sys/kernel/config
/usr/sbin/dlm_controld
sleep 2
/bin/mount -t gfs2 -o noatime,nodiratime,noquota /dev/drbd0 /data
echo

vi /etc/rc.d/rc.local_shutdown

echo GFS2/DLM DOWN
/bin/umount -R /data/
lsof | grep data1
/bin/umount -R /data1/
lsof | grep data2
/bin/umount -R /data2/
#/bin/umount -t gfs2 -a
/usr/bin/pkill dlm_controld
/sbin/rmmod gfs2 dlm
/etc/init.d/corosync stop
echo

echo DRBD DOWN
#/etc/init.d/drbd stop
/usr/local/sbin/drbdadm down all
#v9 /sbin/rmmod drbd_transport_tcp
/sbin/rmmod drbd
echo

STATUS

echo DRBD
drbdadm status
echo

echo DLM
pgrep -a dlm_controld
dlm_tool ls
dlm_tool status
echo

echo COROSYNC
/etc/init.d/corosync status
echo

echo GFS2
mount | grep gfs2
echo

troubleshooting

lsmod | grep gfs2
ls -lF /usr/lib/udev/rules.d/51-dlm.rules
ls -lF /dev/misc/dlm-control
pgrep -a dlm

drbdadm status
corosync-quorumtool

dlm_controld -D --enable_quorum_fencing 0

you are getting

umount: /data2: target is busy.

==>

fuser -vm /data2
fuser -vmMki /data2
lsof | grep data2

troubleshooting

mount: /data_gfs2: mount(2) system call failed: Transport endpoint is not connected.

==> are corosync and dlm daemon started?

resources

GFS2 https://en.wikipedia.org/wiki/GFS2

GFS Project Page https://sourceware.org/cluster/gfs/

dlm https://pagure.io/dlm

gfs2/dlm usage in cluster4 http://people.redhat.com/teigland/cluster4-gfs2-dlm.txt

history

The OpenGFS Project (2004…) http://opengfs.sourceforge.net/

tutorials

How to use gfs2/dlm directly http://people.redhat.com/teigland/cluster4-dlm.txt

How to Create a GFS2 Formatted Cluster File System https://www.thegeekdiary.com/how-to-create-a-gfs2-formatted-cluster-file-system/

DE - Linux-HA: GFS mit DRBD und Pacemaker » ADMIN-Magazin https://www.admin-magazin.de/Das-Heft/2012/01/HA-Workshop-Teil-7-GFS-mit-DRBD-und-Pacemaker/(offset)/6

GFS2 in Pacemaker (Debian/Ubuntu) https://xahteiwi.eu/resources/hints-and-kinks/gfs2-pacemaker-debianubuntu/

Clustering with Pacemaker, DRBD and GFS2 on Bare-Metal servers in SoftLayer https://icicimov.github.io/blog/high-availability/Clustering-with-Pacemaker-DRBD-and-GFS2-on-Bare-Metal-servers-in-SoftLayer/

troubles

mount.gfs2 - GFS2 mount options https://linux.die.net/man/8/gfs2_mount

configfs - Userspace-driven kernel object configuration. https://www.kernel.org/doc/Documentation/filesystems/configfs/configfs.txt

target is busy https://stackoverflow.com/questions/7878707/how-to-unmount-a-busy-device

troubles - redhat kernel version

Re: [Linux-cluster] sparse-file clone breaks on GFS2 https://www.mail-archive.com/linux-cluster@redhat.com/msg13797.html

Re: [Linux-cluster] sparse-file clone breaks on GFS2 https://www.mail-archive.com/linux-cluster@redhat.com/msg13798.html

where to find the kernel source

Where does the RHEL 7 source code live? https://lwn.net/Articles/603865/

Can we git clone the redhat kernel source code and see the changes made by them? https://stackoverflow.com/questions/41314978/can-we-git-clone-the-redhat-kernel-source-code-and-see-the-changes-made-by-them

I Need the Kernel Source https://wiki.centos.org/HowTos/I_need_the_Kernel_Source

I Need to Build a Custom Kernel https://wiki.centos.org/HowTos/Custom_Kernel

Sources https://wiki.centos.org/Sources

4. CentOSPlus kernel differences https://wiki.centos.org/AdditionalResources/Repositories/CentOSPlus#CentOSPlus_kernel_differences

I Need the Kernel Source https://wiki.centos.org/HowTos/I_need_the_Kernel_Source

source repos

Welcome to CentOS Git server https://git.centos.org/

(source have moved) http://ftp.redhat.com/pub/redhat/linux/enterprise/7Server/en/os/README https://git.centos.org/project/rpms https://access.redhat.com/site/downloads/

kernelim / linux https://github.com/kernelim/linux/tree/centos8

use the source, luke

Rebuild a Source RPM https://wiki.centos.org/HowTos/RebuildSRPM

c8s/f/SPECS/kernel.spec https://git.centos.org/rpms/kernel/raw/c8s/f/SPECS/kernel.spec

RPM http://centos.mirrors.proxad.net/8.2.2004/BaseOS/x86_64/os/Packages/ http://mirror.nsc.liu.se/centos-store/8.2.2004/BaseOS/ http://vault.centos.org/8.2.2004/BaseOS/Source/SPackages/

extract RPM https://www.cyberciti.biz/tips/how-to-extract-an-rpm-package-without-installing-it.html https://en.wikipedia.org/wiki/RPM_Package_Manager


HOME | GUIDES | LECTURES | LAB | SMTP HEALTH | HTML5 | CONTACT
Copyright © 2024 Pierre-Philipp Braun