w/o drbd-manage nor linstor
tested on Slackware 14.2 and current
this guide is mainly for DRBD v8 (included in the Linux kernel). The drbd.conf configuration for the v9 does not change much, though, so you could also build and use it, if you wish.
eventually build drbd from scratch on ubuntu, slackware or centos
not sure NTP is that mandatory for this type of cluster, but let’s do it anyway
make sure the nodes communicate with hostname and also IP (validate fingerprints), not sure they need to talk incl to themselves but let’s do it too
three nodes, on which we are going to use 10G partitions.
storage1 -- storage2 -- storage3 sda2 sda2 sda2 sda3 sda3 sda3 sda4 sda4 sda4
and those will be setup as such,
storage1:sda2 + storage2:sda3 --> drbd1 storage2:sda2 + storage3:sda3 --> drbd2 storage3:sda2 + storage1:sda3 --> drbd3
all nodes
cfdisk /dev/sda
then create sda2,3 e.g. 10G each.
slackpkg install parted #official fdisk -l /dev/sda partprobe ls -lF /dev/sda2 /dev/sda3 /dev/sda4
on node1
vi /etc/drbd.conf
global {
usage-count yes;
udev-always-use-vnr;
}
common {
net {
protocol C;
fencing resource-only;
allow-two-primaries yes;
}
disk {
read-balancing when-congested-remote;
}
}
resource r1 {
device /dev/drbd1;
meta-disk internal;
on storage1 {
disk /dev/sda2;
address 10.1.1.221:7701;
}
on storage2 {
disk /dev/sda3;
address 10.1.1.222:7701;
}
}
resource r2 {
device /dev/drbd2;
meta-disk internal;
on storage2 {
disk /dev/sda2;
address 10.1.1.222:7702;
}
on storage3 {
disk /dev/sda3;
address 10.1.1.223:7702;
}
}
resource r3 {
device /dev/drbd3;
meta-disk internal;
on storage3 {
disk /dev/sda2;
address 10.1.1.223:7703;
}
on storage1 {
disk /dev/sda3;
address 10.1.1.221:7703;
}
}
scp /etc/drbd.conf storage2:
scp /etc/drbd.conf storage3:
--force) & ACTIVE/PASSIVEall nodes
lsmod | egrep 'drbd|lru_cache' rmmod lru_cache rmmod drbd_transport_tcp rmmod drbd modprobe drbd modprobe drbd_transport_tcp drbdadm down all drbdadm create-md all drbdadm up all
on node1
state is currently inconsistent to start with, this is why we need to force synchronization
ls -lF /dev/drbd1 ls -lF /dev/drbd3 drbdadm primary --force r1 drbdadm secondary r3 drbdadm status
on node2
ls -lF /dev/drbd1 ls -lF /dev/drbd2 drbdadm primary --force r2 drbdadm secondary r1 drbdadm status
on node3
ls -lF /dev/drbd2 ls -lF /dev/drbd3 drbdadm status drbdadm primary --force r3 drbdadm secondary r2
on any node
watch the volumes synchronize live
watch cat /proc/drbd
once the volume is synchronized, see the difference in size
fdisk -l /dev/sda2 /dev/drbd1
for a 10GB partition as sda2, it takes exactly and persistantly 364544 bytes difference
vi /etc/rc.d/rc.local /sbin/modprobe drbd #v9 /sbin/modprobe drbd_transport_tcp #/etc/init.d/drbd start /usr/local/sbin/drbdadm up all echo vi /etc/rc.d/rc.local_shutdown #/etc/init.d/drbd stop /usr/local/sbin/drbdadm down all #v9 /sbin/rmmod drbd_transport_tcp /sbin/rmmod drbd echo
check what protocol a resource is on on v8
cat /proc/drbd
or on v9 (as long as debug is enabled)
cat /sys/kernel/debug/drbd/resources/<resource>/connections/<connection>/<volume>/proc_drbd
see operations
all nodes
drbdadm down all rm -rf /etc/drbd.d/ rm -f /etc/drbd.conf
what about net {fencing resource-only;}? Does it do anything if I do not set handlers {fence-peer somecommand;}?
drbd-utils/scripts/stonith_admin-fence-peer.sh https://github.com/LINBIT/drbd-utils/blob/master/scripts/stonith_admin-fence-peer.sh
[DRBD-user] Trying to Understanding crm-fence-peer.sh https://lists.linbit.com/pipermail/drbd-user/2019-January/024759.html
LINBIT DRBD kernel module https://github.com/LINBIT/drbd
DRBD userspace utilities (for 9.0, 8.4, 8.3) https://github.com/LINBIT/drbd-utils
“read-balancing” with 8.4.1+ https://www.linbit.com/en/read-balancing/
DRBD 9.0 Manual Pages https://docs.linbit.com/man/v9/
drbd.conf - DRBD Configuration Files https://docs.linbit.com/man/v9/drbd-conf-5/
DRBD 8.4 https://github.com/LINBIT/drbd-8.4
drbd.conf - Configuration file for DRBD’s devices https://docs.linbit.com/man/v84/drbd-conf-5/
How to Install DRBD on CentOS Linux https://linuxhandbook.com/install-drbd-linux/
How to Setup DRBD 9 on Ubuntu 16 https://www.globo.tech/learning-center/setup-drbd-9-ubuntu-16/
LINSTOR SDS server https://github.com/LINBIT/linstor-server
CLI management tool for DRBD. Like top, but for DRBD resources. https://github.com/LINBIT/drbdtop
[DRBD-user] drbd-dkms fails to build under proxmox 6 https://lists.linbit.com/pipermail/drbd-user/2019-August/025208.html
[DRBD-user] Problems compiling kernel module https://lists.linbit.com/pipermail/drbd-user/2016-June/022391.html