XEN // Bootstrap FreeBSD/ZFS from GNU/Linux

freebsd full | freebsd cloud | freebsd ext2fs | freebsd zfs

tested on a Slackware current dom0 (Sep 2020)

RequirementsRequirements

for the guest, we need a kernel with ZFS built-in because we don’t have a loader here and we cannot load deliver the zfs and opensolaris modules.

we then bootstrap the guest system from the XEN host/dom0, similarly to what was done with NetBSD and Slackware. therefore dom0 also needs to be ZFS-capable already

grab latest base tarball

mkdir -p /data/kernels/freebsd130/
cd /data/kernels/freebsd130/
wget https://download.freebsd.org/releases/amd64/amd64/13.0-RELEASE/base.txz
tar xaf kernel.txz

for userland it’s better to use bsdtar

# ubuntu
apt install libarchive-tools

# slackware
slackpkg search libarchive

Guest vdiskGuest vdisk

raw sparse file virtual disk

mkdir -p /root/guests/freebsd-zfs/lala/
cd /root/guests/freebsd-zfs/

dd if=/dev/zero of=freebsd-zfs.zfs bs=1G count=0 seek=10

File-systemFile-system

the kernel wants a GPT partition (GUID a504)

cgdisk freebsd-zfs.zfs

Part. #     Size    Partition Type      Partition Name
----------------------------------------------------------------
        1007.0 KiB  free space
   1    10.0 GiB    FreeBSD ZFS        zfs0

losetup -a
losetup --find --partscan --show freebsd-zfs.zfs

#cannot work - kernel won't find it
#zpool create tank `pwd`/freebsd-zfs.zfs

ls -lF /dev/loop*
zpool create tank /dev/loop0p1
zpool list
zpool status

zfs get compression
zfs set compression=lz4 tank
zfs create -o mountpoint=`pwd`/lala/ tank/root
zfs list
zfs mount

UserlandUserland

bsdtar -xJ --numeric-owner -f /data/kernels/freebsd130/base.txz -C lala/
ls -lhF lala/

no need for a valid file-system table unless you have swap, but still it’s good to have one, even empty, to avoid complains at system startup

vi lala/etc/fstab # new file

# Device        Mountpoint      FStype  Options  Dump    Pass#
#/dev/xbd0p2        none        swap    sw        0       0

cp -pi lala/etc/ttys lala/etc/ttys.dist
vi lala/etc/ttys

#ttyv0  "/usr/libexec/getty Pc"  xterm   onifexists secure
xc0     "/usr/libexec/getty Pc"  xterm   onifexists secure

zfs unmount `pwd`/lala/
zfs unmount /tank/
zfs mount
rmdir lala/

zpool export tank
zpool list

xen guest tuning

fix the serial console

mv -i lala/etc/ttys lala/etc/ttys.dist
vi lala/etc/ttys

xc0     "/usr/libexec/getty Pc"  xterm   onifexists secure

network

vi lala/etc/rc.conf

hostname="freebsd.localdomain"

# esp. required for the cloud image
growfs_enable="YES"

ifconfig_xn0="inet CIDR up"
defaultrouter="GW-IP"
sshd_enable=yes

mv lala/etc/ssh/sshd_config lala/etc/ssh/sshd_config.dist
grep -vE '^#|^$' lala/etc/ssh/sshd_config.dist > lala/etc/ssh/sshd_config
echo permitrootlogin prohibit-password >> lala/etc/ssh/sshd_config

mkdir lala/root/.ssh/
cat ~/.ssh/*pub
vi lala/root/.ssh/authorized_keys

(paste your pub key)

chmod 600 lala/root/.ssh/authorized_keys
chmod 700 lala/root/.ssh/
chmod 700 lala/root/

mv -i lala/etc/hosts lala/etc/hosts.dist
vi lala/etc/hosts

127.0.0.1          localhost.localdomain localhost
::1          localhost.localdomain localhost

10.1.0.9        freebsd.localdomain freebsd

vi lala/etc/resolv.conf

nameserver 10.1.255.19
nameserver 10.1.255.20

environment

ls -lF lala/etc/localtime
ln -s ../usr/share/zoneinfo/Europe/Moscow lala/etc/localtime

losetup -a
losetup -d /dev/loop0

Guest skeletonGuest skeleton

dom0

ls -lhF /data/kernels/kernel-zfs
vi freebsd-zfs

# booting from custom ZFS kernel
kernel = "/data/kernels/kernel-zfs"
extra = "vfs.root.mountfrom=zfs:tank/root"
#extra = "vfs.root.mountfrom=zfs:zroot/ROOT/default"
type = "pvh"

memory = 7168
name = "freebsd-zfs"
vcpus = 3
disk = ['tap:tapdisk:aio:/data/guests/freebsd-zfs/freebsd-zfs.zfs,xvda,w']
vif = [ 'bridge=xenbr0, vifname=fbsdzfs.0',
    'bridge=xenbr0, vifname=fbsdzfs.1' ]

Ready to goReady to go

Now that you got fstab and ttys alright

xl create freebsd-zfs -c

TODOTODO

check about bootfs= of a zpool
check about GELI flags indications from loader

Additional notesAdditional notes

zfs post-install maintenance

mount and rescue from the XEN dom0 host

losetup -a
losetup --find --partscan --show freebsdzfs.img
ls -lF /dev/loop*
fdisk -l /dev/loop0
#/dev/loop0p3 4196352 20969471 16773120    8G FreeBSD ZFS

#zpool import -d /dev -R `pwd`/lala/ zroot freebsdzfs-zroot
zpool import -d /dev -R `pwd`/lala/ -a
#-o readonly=on -f
zpool list
zpool status

zfs mount
zfs mount zroot/ROOT/default

fix and rescue

vi lala/etc/fstab
vi lala/etc/ttys

and when finished

zpool export freebsdzfs-zroot
losetup -d /dev/loop0
ls -lF /dev/loop*

ResourcesResources

boot zfs

Custom Kernel build and ZFS prefetch https://lists.freebsd.org/pipermail/freebsd-questions/2015-February/264111.html

ZFS module parameters in Linux (OR kernel parameter in FreeBSD or zfs-in-kernel OS) https://billtian.github.io/digoal.blog/2014/06/25/03.html

Make ZFS depend on xdr.ko only. It doesn’t need kernel RPC. https://github.com/freebsd/freebsd/commit/77755bd98c42004a010540bd9b6c171a557337a2