XEN FROM SCRATCH

DEPRECATED

this guide was working well a few years ago with slackware 14.2. but with recent XEN versions, python 3 is required. also, we now have GCC 11 in slackware current. also, python-ninja is now required by xen or qemu-xen.

see the new guide for building all this on current

How many cores?

Compilation was done on some enterprise-class IBM servers dating 2007 with 4x4 Xeon processors and 96 GB to 128 GB of RAM. For 16 cores in total and with 1 thread per core (lscpu), I obtained the fastest results with -j16 for xen versus -j24 for tools and stubdom. Anyway, tune the number of make jobs according to your hardware.

Overall requirements (xen + tools + stubdom)

assuming the essential tools for building software installed already

in case you’re running a minimal system

    slackpkg install \
        bison \
        dev86 \
        cmake-3 \
        flex \
        gc \
        git \
        glibc-2 \
        libarchive \
        lzo \
        openssl \
        patch-2 \
        pixman \
        pciutils \
        python3 \
        texinfo \
        zlib-1

    pexpect \
    ptyprocess \
    python3-ninja \
    python3-skbuild \

    syslinux \
    ucl \
    upx \

    wheel \

and additional packages from SBo

    sbopkg -i acpica
    sbopkg -i yajl

Building the micro-kernel

ver=4.x.x

fetch the latest release tarball

wget https://downloads.xenproject.org/release/xen/$ver/xen-$ver.tar.gz
wget https://downloads.xenproject.org/release/xen/$ver/xen-$ver.tar.gz.sig
gpg --keyserver keyserver.ubuntu.com --recv-key 57E82BD9
gpg xen-$ver.tar.gz.sig

with fingerpring

Primary key fingerprint: 23E3 222C 145F 4475 FA80  60A7 83FE 14C9 57E8 2BD9

extract and build the release with the good old cxenstored instead of oxenstored

tar xzf xen-$ver.tar.gz
cd xen-$ver/
./configure \
  --sysconfdir=/etc \
  --localstatedir=/var \
  --with-xenstored=xenstored \
  --disable-sdl \
  --disable-xsmpolicy \
  --disable-systemd \
  --disable-seabios \
  --disable-stubdom \
  --disable-ioemu-stubdom \
  --disable-pv-grub \
  --disable-xenstore-stubdom \
  --disable-ocamltools \
  --disable-qemu-traditional \
  --disable-ovmf \
  --with-extra-qemuu-configure-args="--sysconfdir=/etc --audio-drv-list= --disable-slirp --disable-blobs --without-default-features"

and since we are using a minimal system

--disable-docs \

more options

#  --disable-rombios already disabled by --disable-qemu-traditional
#  --disable-debug
#  --disable-pvshim

finally proceed with the build!

echo $MAKEFLAGS
# make clean-xen >/dev/null && echo CLEAN
time make xen > ../xen.log && echo BUILT
# 0m34.370s on bravo

make install-xen > ../install-xen-${HOSTNAME%%.*}.log && echo DEPLOYED

ln -s boot/xen.gz /xen.gz
cd ../

Multilib

slackware-multilib

Requirements for tools

slackpkg install gettext-tools pkg-config glib2 pcre google-go-lang

Fetch ipxe & seabios GIT repos through https://

export PATH=$PATH:/usr/lib64/bcc
which bcc-cpp

installing the latest ACPICA for xen tools to build

wget https://acpica.org/sites/acpica/files/acpica-unix2-20210730.tar.gz
tar xzf acpica-unix2-20210730.tar.gz
cd acpica-unix2-20210730/

echo $MAKEFLAGS
make > ../acpica.log && echo BUILT

make install > ../make.acpica.install.log && echo INSTALLED
cd ../

Building the tools

(configure was done above)

#make distclean
#time make world > ../world.$HOSTNAME.log && echo WORLD

#make clean-tools

echo $MAKEFLAGS
time make tools > ../tools.log && echo BUILT
make install-tools > ../install-tools.log && echo DEPLOYED

#make clean-stubdom
#time make stubdom > stubdom.log && echo BUILT
#time make install-stubdom

cp -pi /etc/ld.so.conf /etc/ld.so.conf.dist
cat >> /etc/ld.so.conf <<-EOF
/usr/local/lib
EOF
cat /etc/ld.so.conf
ldconfig

Setup

without autoballoon nor anything specific (default network type is bridge already)

#mv -i /etc/xen/xl.conf /etc/xen/xl.conf.dist
#grep -vE '^[[:space:]]*(#|$)' /etc/xen/xl.conf.dist #empty
#vi /etc/xen/xl.conf

#https://xenbits.xen.org/docs/4.11-testing/man/xl.conf.5.html
#https://xenbits.xen.org/docs/unstable/man/xl.conf.5.html
#autoballoon="on"
#vif.default.bridge="xenbr0"
#vif.default.script="vif-bridge"
#vif.default.script="vif-openvswitch"

now we want to enable the XEN daemons at boot time alright.

ls -lF /etc/init.d/xen*
vi /etc/rc.d/rc.local

we keep using xendomains because otherwise, guests live alone while dom0 shuts down. also, the default scheduler is now Credit2, since 4.10 or 4.12 – load it AFTER xencommons!

echo XEN
/etc/init.d/xencommons start
/etc/init.d/xendomains start
/usr/local/sbin/xl sched-credit2 --domain=0 --weight=512
/usr/local/sbin/xl sched-credit2
echo

and shutdown

vi /etc/rc.d/rc.local_shutdown

#!/bin/sh

echo XEN DOWN
/etc/init.d/xendomains stop
/etc/init.d/xencommons stop
echo

chmod +x /etc/rc.d/rc.local_shutdown
grep rc.local_shutdown /etc/rc.d/rc.*

Dom0 kernel

slackpkg search kernel

No need for those

removepkg kernel-huge
removepkg kernel-modules
removepkg kernel-source

but it’s good to keep those

slackpkg install kernel-firmware
slackpkg install kernel-headers

and you can now proceed with dom0 kernel building

Booting

Choose either GRUB2 or SYSLINUX – LILO is not an option here (unless you want to play with mbootpack)

Index of /booting/ https://pub.nethence.com/booting/

Networking

see network

Ready to go

cat /sys/hypervisor/properties/capabilities
cat /proc/xen/capabilities
/etc/init.d/xencommons status
xl li

Handy scripts

It took me a while to figure out with my slackware/XEN servers were lagging and even refusing to provide a prompt when NFS was failing. It was because I had the bad idea to put /data/xen in the PATH. And that was on the NFS share. Therefore it is better to have a separate copy of the repository on each node (here in /root/).

cd ~/
git clone git@github.com:pbraun9/xen.git
git clone https://github.com/pbraun9/xen.git

Tuning

more loops

max_loop=128

Xen dom0 complains about not enough free loop devices when trying to start a new domU or when trying to “mount -o loop” from the cmdline https://wiki.xenproject.org/wiki/Xen_Common_Problems#Xen_dom0_complains_about_not_enough_free_loop_devices_when_trying_to_start_a_new_domU_or_when_trying_to_.22mount_-o_loop.22_from_the_cmdline

How to add more /dev/loop* devices on Fedora 19 https://unix.stackexchange.com/questions/98742/how-to-add-more-dev-loop-devices-on-fedora-19

\3. Adding Support for More Loop Devices https://www.tldp.org/HOWTO/CDServer-HOWTO/addloops.html

libvirt capable (optional)

slackpkg install libpciaccess
sbopkg -i libvirt
sbopkg -i libvirt-python
sbopkg -i virt-manager
sbopkg -i virt-viewer

cat /etc/sysctl.d/60-libvirtd.conf
sysctl fs.aio-max-nr=1048576

/usr/sbin/libvirtd -d
cat >> /etc/rc.local <<-EOF

echo -n starting libvirt...
/usr/sbin/libvirtd -d && echo done
EOF

virsh list

OBSOLETE

mounted by xencommons,

cat >> /etc/fstab <<-EOF
xenfs /proc/xen xenfs defaults 0 0
EOF
mount /proc/xen/

eventually fix an anoying bug into seabios,

grep -r BXSSDTSUSP *
vi tools/firmware/seabios-dir-remote/src/fw/ssdt-misc.dsl

BXSSDTSUSP --> BXSSDTSU

grep -r BXSSDTPCIHP *
vi tools/firmware/seabios-dir-remote/src/fw/ssdt-pcihp.dsl

BXSSDTPCIHP --> BXSSDTPC

MOAR TROUBLES

/data/host/xen-4.12.3/tools/libxl/dsdt_pvh.asl   3339:         Processor ( PR7F, 127, 0x0000b010, 0x06 ) {
Warning  3168 -                                                              ^ Legacy Processor() keyword detected. Use Device() keyword instead.

/tmp/cc7ADtII.s: Assembler messages:
/tmp/cc7ADtII.s:3: Error: junk at end of line, first unrecognized character is `{'
Makefile:88: Test harness not built, use newer compiler than "gcc"

while building tools

ERROR: Cannot find Ninja

–> pip install ninja

while booting dom0

(XEN) ERROR: Will only load images built for the generic loader or Linux images (Not '' and '') or with PHYS32_ENTRY set

–> dom0 kernel is not XEN-capable

Resources

Dom0 Kernels for Xen https://wiki.xenproject.org/wiki/Dom0_Kernels_for_Xen

Compiling Xen https://wiki.xenproject.org/wiki/Compiling_Xen

Compiling Xen From Source https://wiki.xenproject.org/wiki/Compiling_Xen_From_Source

Xen Project Repositories https://wiki.xenproject.org/wiki/Xen_Project_Repositories

Getting Started with Xen on Slackware http://amree.github.io/2012/09/12/getting-started-with-xen-on-slackware/

Xen Documentation https://xenbits.xen.org/docs/unstable/


HOME | GUIDES | LECTURES | LAB | SMTP HEALTH | NU | HTML5
Copyright © 2022 Pierre-Philipp Braun