XEN FROM SCRATCH

No Budget!

Warning

14.2 is outdated. Use Slackware-current. With 4.13.0-rc6 I get

ERROR: glib-2.48 gthread-2.0 is required to compile QEMU

and Python 3 is required anyhow.

How many cores?

Compilation was done on some enterprise-class IBM servers dating 2007 with 4x4 Xeon processors and 96 GB to 128 GB of RAM. For 16 cores in total and with 1 thread per core (lscpu), I obtained the fastest results with -j16 for xen versus -j24 for tools and stubdom. Anyway, tune the number of make jobs according to your hardware.

Overall requirements (xen + tools + stubdom)

current

in case you’re running a minimal system

    slackpkg install glibc-2 patch-2 pixman pciutils texinfo zlib-1 lzo yajl \
            guile gc python3-3.8

then check

for pkg in \
    bison \
    flex \
    gc \
    gcc \
    git \
    glibc \
    guile \
    make \
    openssl \
    patch \
    pixman \
    pciutils \
    python3 \
    texinfo \
    zlib \
    lzo \
    yajl \
    ; do
    echo -n $pkg:
    ls -1 /var/log/packages/ | grep ^$pkg-[[:digit:]]
    echo ''
done; unset pkg

Compiling XEN (the micro-kernel)

fetch the latest release tarball

cd /data/host/
ver=4.14.0-rc6
wget https://downloads.xenproject.org/release/xen/$ver/xen-$ver.tar.gz
wget https://downloads.xenproject.org/release/xen/$ver/xen-$ver.tar.gz.sig
gpg --recv-key 57E82BD9
gpg xen-$ver.tar.gz.sig

with fingerpring

Primary key fingerprint: 23E3 222C 145F 4475 FA80  60A7 83FE 14C9 57E8 2BD9

if you want current

#git clone git://xenbits.xen.org/xen.git

extract and build

tar xzf xen-$ver.tar.gz
cd xen-$ver/
#nice make -j16 clean-xen >/dev/null && echo CLEAN
time nice make -j16 xen > ../make.xen.log && echo BUILT
#1m18.519s vs 0m24.499s 0m26.116s

# ON ALL NODES
nice make -j16 install-xen > ../make.install-xen.log && echo INSTALLED
ls -lF /xen* /boot/xen*
mv /xen.gz /xen.gz.old
cp -i /boot/xen.gz /xen.gz
cd ../

Tools Requirements

installing the latest ACPICA (20180427 fails), for the xen tools to compile,

#Intel license
#wget https://acpica.org/sites/acpica/files/acpica-unix-20190816.tar.gz

#GPL
#wget https://acpica.org/sites/acpica/files/acpica-unix2-20190816.tar.gz
wget https://acpica.org/sites/acpica/files/acpica-unix2-20200528.tar.gz
tar xzf acpica-unix2-20200528.tar.gz
cd acpica-unix2-20200528/
nice make -j16 > ../make.acpica.log && echo BUILT

# ON ALL NODES
make install > ../make.acpica.install.log && echo INSTALLED
cd ../

installing the corresponding multilib version of gcc and glibc (and installing compat32-tools),

mkdir -p /data/host/multilib/
cd /data/host/multilib/

we DO NOT want the recursive folders with slackware64-compat32

#lftp -c 'open http://slackware.com/~alien/multilib/ ; mirror -c -e 14.2'

we want only the first-level packages from that folder

current

slackpkg install lftp
lftp -c 'open http://slackware.com/~alien/multilib/ ; mget -c current/*'
du -sh ./ #154M

ls -lF gcc-9*.t?z
ls -lF glibc-2*.t?z
ls -lF glibc-solibs-*.t?z
ls -lF compat32-tools-*.t?z

upgradepkg --terse --reinstall gcc-9*.t?z
upgradepkg --terse --reinstall glibc-2*.t?z
upgradepkg --terse --reinstall glibc-solibs-*.t?z
installpkg --terse compat32-tools-*.t?z

ls -lF /var/log/packages/gcc-9*
ls -lF /var/log/packages/glibc-2*
ls -lF /var/log/packages/glibc-solibs-*
ls -lF /var/log/packages/compat32-tools-*
cd ../

and this might be useful for other pieces of software, since you’ve got compat32

vi /etc/profile

export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig

Compiling the Tools

For XEN 4.14+ you will need python 3 – assuming current

ls -lF /usr/bin/python3*

Requirements

slackpkg install gettext-tools pkg-config glib2 pcre google-go-lang

Fetch ipxe & seabios GIT repos through https://

cd /data/host/xen-$ver/
export PATH=$PATH:/usr/lib64/bcc
which bcc-cpp

./configure --enable-githttp

#make -j$j distclean
#time nice make -j$(($j * 2 - 1)) world > ../world.$HOSTNAME.log && echo BUILTWORLD

#make -j$j clean-tools

# ON ALL NODES
time nice make -j12 tools > ../tools.log && echo BUILT
#6m58.954s vs 2m38.416s
nice make -j12 install-tools > ../install-tools.log && echo INSTALLED

#make clean-stubdom
#time make -j12 stubdom > make.stubdom.log && echo done
#time make install-stubdom

cp -pi /etc/ld.so.conf /etc/ld.so.conf.dist
cat >> /etc/ld.so.conf <<-EOF
/usr/local/lib
EOF
cat /etc/ld.so.conf
ldconfig

Tools Setup

without autoballoon nor anything specific (default network type is bridge already)

#mv -i /etc/xen/xl.conf /etc/xen/xl.conf.dist
#grep -vE '^[[:space:]]*(#|$)' /etc/xen/xl.conf.dist #empty
#vi /etc/xen/xl.conf

#https://xenbits.xen.org/docs/4.11-testing/man/xl.conf.5.html
#https://xenbits.xen.org/docs/unstable/man/xl.conf.5.html
#autoballoon="on"
#vif.default.bridge="xenbr0"
#vif.default.script="vif-bridge"
#vif.default.script="vif-openvswitch"

now we want to enable the XEN daemons at boot time alright.

ls -lF /etc/init.d/xen*
vi /etc/rc.d/rc.local

we keep using xendomains because otherwise, guests live alone while dom0 shuts down. also, the default scheduler is now Credit2, since 4.10 or 4.12 – load it AFTER xencommons!

echo XEN
/etc/init.d/xencommons start
/etc/init.d/xendomains start
/usr/local/sbin/xl sched-credit2 --domain=0 --weight=512
/usr/local/sbin/xl sched-credit2
echo

and shutdown

vi /etc/rc.d/rc.local_shutdown

#!/bin/sh

echo XEN DOWN
/etc/init.d/xendomains stop
/etc/init.d/xencommons stop
echo

chmod +x /etc/rc.d/rc.local_shutdown
grep rc.local_shutdown /etc/rc.d/rc.*

Dom0 kernel

slackpkg search kernel

No need for those

removepkg kernel-huge
removepkg kernel-modules
removepkg kernel-source

but it’s good to keep those

slackpkg install kernel-firmware
slackpkg install kernel-headers

and you can now proceed with dom0 kernel building

Booting

Choose either GRUB2 or SYSLINUX – LILO is not an option here (unless you want to play with mbootpack)

Index of /booting/ https://pub.nethence.com/booting/

Networking

see networking

Ready to go

cat /sys/hypervisor/properties/capabilities
cat /proc/xen/capabilities
/etc/init.d/xencommons status
xl li

Handy scripts

It took me a while to figure out with my slackware/XEN servers were lagging and even refusing to provide a prompt when NFS was failing. It was because I had the bad idea to put /data/xen in the PATH. And that was on the NFS share. Therefore it is better to have a separate copy of the repository on each node (here in /root/).

cd ~/
git clone git@github.com:pbraun9/xen.git
git clone https://github.com/pbraun9/xen.git

Tuning

more loops

max_loop=128

Xen dom0 complains about not enough free loop devices when trying to start a new domU or when trying to “mount -o loop” from the cmdline https://wiki.xenproject.org/wiki/Xen_Common_Problems#Xen_dom0_complains_about_not_enough_free_loop_devices_when_trying_to_start_a_new_domU_or_when_trying_to_.22mount_-o_loop.22_from_the_cmdline

How to add more /dev/loop* devices on Fedora 19 https://unix.stackexchange.com/questions/98742/how-to-add-more-dev-loop-devices-on-fedora-19

\3. Adding Support for More Loop Devices https://www.tldp.org/HOWTO/CDServer-HOWTO/addloops.html

libvirt capable (optional)

slackpkg install libpciaccess
sbopkg -i libvirt
sbopkg -i libvirt-python
sbopkg -i virt-manager
sbopkg -i virt-viewer

cat /etc/sysctl.d/60-libvirtd.conf
sysctl fs.aio-max-nr=1048576

/usr/sbin/libvirtd -d
cat >> /etc/rc.local <<-EOF

echo -n starting libvirt...
/usr/sbin/libvirtd -d && echo done
EOF

virsh list

Resources

Dom0 Kernels for Xen https://wiki.xenproject.org/wiki/Dom0_Kernels_for_Xen

Compiling Xen From Source https://wiki.xenproject.org/wiki/Compiling_Xen_From_Source

Xen Project Repositories https://wiki.xenproject.org/wiki/Xen_Project_Repositories

Getting Started with Xen on Slackware http://amree.github.io/2012/09/12/getting-started-with-xen-on-slackware/

OBSOLETE

mounted by xencommons,

cat >> /etc/fstab <<-EOF
xenfs /proc/xen xenfs defaults 0 0
EOF
mount /proc/xen/

eventually fix an anoying bug into seabios,

grep -r BXSSDTSUSP *
vi tools/firmware/seabios-dir-remote/src/fw/ssdt-misc.dsl

BXSSDTSUSP --> BXSSDTSU

grep -r BXSSDTPCIHP *
vi tools/firmware/seabios-dir-remote/src/fw/ssdt-pcihp.dsl

BXSSDTPCIHP --> BXSSDTPC

MOAR TROUBLES

/data/host/xen-4.12.3/tools/libxl/dsdt_pvh.asl   3339:         Processor ( PR7F, 127, 0x0000b010, 0x06 ) {
Warning  3168 -                                                              ^ Legacy Processor() keyword detected. Use Device() keyword instead.

/tmp/cc7ADtII.s: Assembler messages:
/tmp/cc7ADtII.s:3: Error: junk at end of line, first unrecognized character is `{'
Makefile:88: Test harness not built, use newer compiler than "gcc"

HOME | GUIDES | BENCHMARKS | html