Setting up KVM/Libvirt

Install

grep -E --color=always 'vmx|svm| lm ' /proc/cpuinfo | uniq

Ubuntu

apt install qemu-kvm bridge-utils
apt install virtinst libosinfo-bin libguestfs-tools

Debian

apt-get install qemu-system libvirt-clients libvirt-daemon-system
apt-get install --no-install-recommends qemu-system libvirt-clients libvirt-daemon-system

make virsh commands available to some non-root user

usermod -aG libvirt USER

check for hardware accel and other things

lsmod | grep kvm
#rmmod kvm
#modprobe -a kvm

kvm-ok
virt-host-validate

#ls -lF /sys/hypervisor/
#cat /sys/hypervisor/properties/capabilities

ls -lF /var/run/libvirt/libvirt-sock # root:libvirt
ls -lF /dev/kvm # root:kvm

systemctl status libvirtd
systemctl status libvirt-guests
ps auxww | grep libvirtd$
virsh list --all

virsh nodeinfo
osinfo-query os
libguestfs-test-tool

to enable nested virtualization e.g. to setup XEN dom0 within KVM

vi /etc/modprobe.d/kvm.conf

options kvm_amd nested=1

rmmod kvm_amd
modprobe kvm_amd
    modinfo kvm_amd | grep nested

Storage

assuming /data/ is an NFS mount point already, enable it as KVM storage

virsh pool-list
virsh pool-dumpxml default > pool.xml
virsh pool-destroy default
vi pool.xml

    <path>/data/kvm</path>

mkdir /data/kvm
virsh pool-create pool.xml
virsh pool-list

Network

without libvirt

Create and manage the bridge [without libvirt]/xen/networking.html)

Then point the guests against the XEN bridge

--network bridge=xenbr0,model=virtio \

with libvirt

This gets enabled by default on Ubuntu. But on a Debian system you need to proceed manually.

virsh net-list --all
virsh net-start default
virsh net-autostart default

#virsh net-destroy default
#virsh net-autostart default --disable

Then point the guests against the Libvirt bridge

--network bridge=virbr0,model=virtio \

Ready to go

be ready for virt-install

mkdir -p /data/kvm/

ifconfig xenbr0
ifconfig virbr0

osinfo-query os | grep -i centos
osinfo-query os | grep -i openbsd

and install some guests

Live-migration

to be able to talk to the other hypervisors and enable the farm, you will have to share a few SSH keys.

to migrate a guest, make sure the machine type is compatible on both ends,

kvm -M ?

and fix it (DO NOT USE the pc alias that is pointing to different latest versions depending on the hypervisors – choose the latest available on all hypervisors e.g. pc-i440fx-2.5) if required,

virsh shutdown GUESTNAME
virsh edit GUESTNAME

  <os>
    <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type>
    <boot dev='hd'/>
  </os>

the available virtual network devices can be displayed as such,

qemu-system-x86_64 -net nic,model=? /dev/null

e.g.,

qemu: Supported NIC models: ne2k_pci,i82551,i82557b,i82559er,rtl8139,e1000,pcnet,virtio

replacing rtl8139 with virtio,

    <interface type='bridge'>
      <mac address='xx:xx:xx:xx:xx:xx'/>
      <source bridge='xenbr0'/>
      <model type='virtio'/>
            <driver name='vhost' txmode='iothread' ioeventfd='on' event_idx='off' queues='5' rx_queue_size='256' tx_queue_size='256'>
              <host csum='off' gso='off' tso4='off' tso6='off' ecn='off' ufo='off' mrg_rxbuf='off'/>
              <guest csum='off' tso4='off' tso6='off' ecn='off' ufo='off'/>
            </driver>
    </interface>

you may also have to disable caching on the virtual drive,

  <driver name='qemu' type='qcow2' cache='none'/>

refs https://access.redhat.com/solutions/158363

you will then be able to live-migrate,

virsh -c qemu+ssh://ANOTHER-KVM-SRV/system list
virsh migrate GUESTNAME qemu+ssh://ANOTHER-KVM-SRV/system

TRASH

prepare a new bridge config for libvirt and get rid of the crapy defaults,

virsh net-dumpxml default > net-dumpxml.xml
virsh net-destroy default
#virsh net-undefine default
service libvirtd restart

cp net-dumpxml.xml br0.xml
vi br0.xml

<network>
  <name>br0</name>
  <uuid>SOME-UUID-HERE</uuid>
  <bridge name='br0' stp='off' delay='0'/>
</network>

virsh net-create br0.xml

https://help.ubuntu.com/community/KVM/Networking

https://wiki.libvirt.org/page/VirtualNetworking

https://www.cyberciti.biz/faq/linux-kvm-disable-virbr0-nat-interface/

Resources

install

https://help.ubuntu.com/lts/serverguide/libvirt.html

https://help.ubuntu.com/community/KVM/Installation

https://wiki.xen.org/wiki/Libvirt

https://www.cyberciti.biz/faq/installing-kvm-on-ubuntu-16-04-lts-server/

https://wiki.debian.org/KVM

https://wiki.debian.org/libvirt

https://computingforgeeks.com/virsh-commands-cheatsheet/

tuning

Tuning KVM https://www.linux-kvm.org/page/Tuning_KVM

Tuning Kernel https://www.linux-kvm.org/page/Tuning_Kernel

Memory https://www.linux-kvm.org/page/Memory

Chapter 6. Overcommitting with KVM https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/virtualization_administration_guide/chap-virtualization-tips_and_tricks-overcommitting_with_kvm

Chapter 7. KSM https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/virtualization_administration_guide/chap-KSM

How to use the Kernel Samepage Merging feature https://www.kernel.org/doc/Documentation/vm/ksm.txt

KVM host won’t use memory for caching https://serverfault.com/questions/710172/kvm-host-wont-use-memory-for-caching

Virtualization (KVM) and excessive cached RAM https://unix.stackexchange.com/questions/241142/virtualization-kvm-and-excessive-cached-ram

Kernel same-page merging https://en.wikipedia.org/wiki/Kernel_same-page_merging

virt-install

https://raymii.org/s/articles/virt-install_introduction_and_copy_paste_distro_install_commands.html#Ubuntu_14.04

network

https://wiki.debian.org/LXC/LibVirtDefaultNetwork

https://wiki.debian.org/KVM#Libvirt_default_network

https://unix.stackexchange.com/questions/340521/create-virbr0-for-virsh

https://linuxconfig.org/how-to-use-bridged-networking-with-libvirt-and-kvm

live-migration

https://mike42.me/blog/libvirt-migrate-a-vm-from-qemusession-to-qemusystem https://wiki.libvirt.org/page/Failed_to_connect_to_the_hypervisor http://blog.programster.org/kvm-offline-migration https://libvirt.org/migration.html https://libvirt.org/formatdomain.html https://libvirt.org/formatdomain.html#elementsNICS https://wiki.ubuntu.com/QemuKVMMigration https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1617214 https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1637438 https://www.linux-kvm.org/page/Management_Tools https://www.linux-kvm.org/page/Networking https://www.redhat.com/archives/libvirt-users/2015-February/msg00024.html https://access.redhat.com/errata/RHSA-2016:2577

storage

http://ask.xmodulo.com/change-default-location-libvirt-vm-images.html

Trash

those don’t exist anymore

libvirt-bin ubuntu-vm-builder

also deprecated, apparently

systemctl status libvirt-bin.service

GUIDES | LECTURES | BENCHMARKS | SMTP HEALTH