**Debian, KVM and libvirt**
# Objectives
Working in pairs, you will build a Linux-based virtualization host server on
hardware provided in the class.
* Install Debian Linux. (This is because Ganeti, which we will be using later,
is better supported under Debian than Ubuntu)
* Configure LVM during installation with manual partitioning so that most
of the volume group is unused
* Use KVM with the `virt-manager` GUI and the `virsh` command-line interface
to install and manage a VM.
NOTE: please take care to install LVM as shown, creating some small logical
volumes for the system but leaving most of the volume group unused, because
later exercises will need to create more logical volumes from the free space.
Please also name your volume group "ganeti", because this has to be consistent
throughout a cluster of machines.
# Install Debian
It may be necessary to press a key (e.g. F12) to get the option to boot
from the USB stick.
If you get the Debian splash screen, select "install" instead of "graphical
install".
* Language: English
* Country: as appropriate
* Locale settings: United States
* Keymap: American English
* Hostname: as allocated by instructor (nodeX where X is a number)
* Domain name: virt.nsrc.org
* Root password: use the class root password
* Full name for new user: NSRC
* Username: "nsrc"
* Password: class user password
* Select your time zone if prompted
## Partitioning
* Select "Manual" (and hit Enter)
* Select your disk drive (sda)
* (If there is no partition table, you'll be prompted to create one)
* Delete existing partition(s)
* Instructors will tell you the best approach
* If there's a large Windows partition, delete it
* If there's a Recovery partition, keep it
* Select "pri/log FREE SPACE"
* Press ENTER to create a new partition
* New partition size: 1G (this is the /boot filesystem)
* Primary
* Beginning
* Use as: Ext4 journaling file system
* Mount point: /boot - static files of the boot loader
* Bootable flag: on
* Done setting up the partition
* Select "pri/log FREE SPACE"
* Press ENTER to create a new partition
* Accept whatever size is shown (the rest of the disk)
* Primary
* Use as: physical volume for LVM
* Done setting up the partition
* Configure the Logical Volume Manager
* Write changes to disks and configure LVM? Yes
* Create volume group
* Volume group name: ganeti
* Devices for the new volume group: select only the LVM partition
(probably /dev/sda2) and hit SPACE to display '[*]', then hit
Enter to continue.
* Create logical volume
* Volume group: ganeti
* Logical volume name: root
* Logical volume size: 4GB
* Create logical volume
* Volume group: ganeti
* Logical volume name: swap
* Logical volume size: 4GB
* Create logical volume
* Volume group: ganeti
* Logical volume name: var
* Logical volume size: 20GB
* Finish
* Select the entry #1 underneath LV *root*
~~~
LVM VG ganeti, LV root - 4.0 GB Linux device-mapper (linear)
#1 4.0 GB <<<<<<<< select this <<<<<<<
~~~
* Select:
* Use as: Ext4 journaling file system
* Mount point: / - the root filesystem
* Done setting up the partition
* Select the entry #1 underneath LV *swap*
* Use as: swap area
* Done setting up the partition
* Select the entry #1 underneath LV *var*
* Use as: Ext4 journalling file system
* Mount point: /var - variable data
* Done setting up the partition
* Finish partitioning and write changes to disk
(might need to scroll down to see this option)
* Write the changes to disks? Yes
## Software selection
* Use a network mirror? Yes
* Check with the instructor which country/mirror to use. In the
absence of any guidance, [use these](https://wiki.debian.org/DebianGeoMirror)
* Country: United States
* Mirror: cdn.debian.net
* HTTP proxy: `http://apt.virt.nsrc.org:3142/`
* Participate in the package usage survey? No
* Software to install. Select only these:
* [X] SSH server
* [X] Standard system utilities
* (Do not select "Debian desktop environment" or you will get tons of
applications including LibreOffice, Iceweasel, Gimp, etc)
* (To save time, do not select "Laptop" even if this is a laptop. You can
install package "task-laptop" later if you wish)
* Install the GRUB boot loader to the master boot record? Yes
* Installation complete - hit Continue
You should now boot into the system.
## Basic configuration
Login as root.
The first thing you will need to do is to get the installer to forget about
the CD-ROM.
As root, edit the file `/etc/apt/sources.list`, for example using vi:
~~~
# editor /etc/apt/sources.list
~~~
Comment out any line which begins "deb cdrom:..." by inserting a hash (`#`)
in front of it, changing it from this:
~~~
deb cdrom:[Debian GNU/Linux 7.7.0 [...]
~~~
To this:
~~~
# deb cdrom:[Debian GNU/Linux 7.7.0 [...]
~~~
Then save the file.
Install `sudo`, and add the 'nsrc' user into the 'sudo' group
~~~
# apt-get update
# apt-get install sudo
# usermod -G sudo -a nsrc
~~~
Also install `vim` which is an improved version of the vi editor
~~~
# apt-get install vim
~~~
## Network configuration
Your server may have picked up a dynamic IP address via DHCP, which you
can find out using the `ifconfig` command:
~~~
# ifconfig eth0
~~~
Now you are going to give it a static IP address and also configure bridging
so that your virtual machines can share the same network interface.
Your machine will be given IP address 10.10.10.X
### Edit `/etc/hostname`
~~~
nodeX.virt.nsrc.org
~~~
### Edit `/etc/hosts`, and change it to look like this:
~~~
127.0.0.1 localhost
10.10.10.X nodeX.virt.nsrc.org nodeX
~~~
### Install additional networking packages
~~~
# apt-get install bridge-utils vlan
~~~
### Edit `/etc/network/interfaces`
Change the file so that it looks like this. This removes the (dynamic) IP
address from eth0, and instead creates a bridge interface "br-lan" with a
static IP address, and eth0 a member of the bridge.
~~~
# The loopback network interface
auto lo
iface lo inet loopback
# Management interface
auto eth0
iface eth0 inet manual
auto br-lan
iface br-lan inet static
address 10.10.10.X
netmask 255.255.255.0
gateway 10.10.10.254
bridge_ports eth0
bridge_stp off
bridge_fd 0
bridge_maxwait 0
~~~
You can activate your changes like this:
~~~
# ifdown eth0
# killall dhclient
# ifup br-lan
# brctl show
bridge name bridge id STP enabled interfaces
br-lan 8000.xxxxxxxxxxxx no eth0
~~~
You should see your new IP address on `ifconfig br-lan`, and you should
still be able to ping out (e.g. `ping 8.8.8.8`)
### Edit `/etc/resolv.conf`
~~~
domain virt.nsrc.org
nameserver 10.10.10.241
~~~
Check you can still resolve names (e.g. `ping apt.virt.nsrc.org`)
### Reboot
As a final check, reboot your server.
~~~
# reboot
~~~
It should come up on the same IP address again.
### Test ssh
You should now be able to ssh into your server as user "nsrc" (e.g. using
putty under Windows). Check you are able to do this. ssh to
either `nodeX.virt.nsrc.org` or `10.10.10.X`
Both people in the pair should be able to connect to the server from their
laptop, so they don't need to share the console.
NOTE: get into the habit of logging into the server as a normal user, and
try to avoid logging in as "root". You will see a prompt which ends with
`$`. It's advisable to disable logging in as root directly over SSH (not
covered here).
If there is a single command you wish to run as root, prefix it with `sudo`.
If you have a series of commands to run as root, start a root shell with
`sudo -s`, and the prompt will change to `#`. When you have finished, type
`exit` to leave the root shell.
# Virtual machine tools
## Install KVM and `libvirt`
~~~
# apt-get install qemu-kvm libvirt-bin
~~~
Now add the 'nsrc' user into the 'libvirt' group.
~~~
# usermod -G libvirt -a nsrc
~~~
This is needed so you can talk to the libvirt daemon. If you are currently
logged in as 'nsrc' you will need to logout and login again to pick up
this group. The command "id" will show you what groups you are a member of.
## Check for hardware virtualization support
At a command line prompt, type the following:
~~~
$ egrep '(vmx|svm)' /proc/cpuinfo
~~~
If you get one or more lines containing "vmx" or "svm" then the processor
has hardware virtualization capabilities. However this doesn't tell you if
it's enabled in the BIOS.
To verify, do:
~~~
$ ls -l /dev/kvm
~~~
If you see:
~~~
crw-rw---- 1 root kvm 10, 232 Jan 17 21:24 /dev/kvm
~~~
Then it's OK [^1]
[^1]: On Ubuntu, The `kvm-ok` utility, included in the package `cpu-checker`,
will tell you.
If KVM acceleration is not available, then you may need to reboot into the
BIOS settings and enable VT-x (Intel) or AMD-V (AMD).
## Desktop environment
Install an X11 desktop (which we want just so that we can run virt-manager)
~~~
# apt-get install xorg lxde lightdm ssh-askpass --no-install-recommends
~~~
This will take a while - be patient!
Once the install is finished, start the graphical environment:
~~~
# service lightdm start
~~~
You should now be able to login to the graphical environment as the "nsrc"
user. You can get a shell window using `Start > Accessories > LXTerminal`.
You can also switch between text and graphical consoles using Ctrl-Alt-F1
and Ctrl-Alt-F7 respectively.
## Install virt-manager GUI
~~~
# apt-get install virt-manager
~~~
This installs a lot of extra packages because it depends on a lot of
graphics libraries.
# Create a VM
On the virtualization host you have just built, you are going to install
Ubuntu Server in a VM, using the virt-manager GUI which is similar to the
VirtualBox GUI.
## Copy Ubuntu ISO image
First, copy the file `ubuntu-14.04.1-server-i386.iso` into the directory
`/var/lib/libvirt/images` on your host.
How to do this depends on where you are copying the image from.
* If you are copying from a USB stick: insert the stick. If you have a GUI
it will ask you if you want to open with the File Manager. Say yes, and
it should mount the USB stick automatically (on `/media/XXX`), and open
a window.
(XXX will be a random number, look at the title of the window)
You can then copy the ISO:
~~~
# cp /media/XXX/iso/ubuntu-14.04.1-server-i386.iso /var/lib/libvirt/images/
~~~
If it doesn't work, you may need to mount it by hand:
~~~
# dmesg | tail # find the ID of the device just inserted, e.g. sdb
# mount /dev/sdb1 /mnt
# ls /mnt # find the file
# ls /mnt/iso # find the file
# cp /mnt/iso/ubuntu-14.04.1-server-i386.iso /var/lib/libvirt/images/
# umount /mnt
~~~
* If you are copying from a web server:
~~~
# cd /var/lib/libvirt/images
# wget http://download.virt.nsrc.org/ubuntu-14.04.1-server-i386.iso
~~~
## Start the virt-manager GUI
Use `Start > System Tools > Virtual Machine Manager`, or just type
`virt-manager` at a terminal prompt while in the GUI.
NOTE: to release the keyboard and mouse, press left-CTRL and left-ALT
together.
## Create a VM
In the Virtual Machine Manager window:
* Click "New" (this is the PC-shaped button under the File menu)
* Step 1 of 5
* Enter a name, e.g. "ubuntu1"
* Select "Local install media (ISO image or CDROM)"
* Click Forward
* Step 2 of 5
* Check "Use ISO image" is selected
* Click "Browse", and select `ubuntu-14.04.1-server-i386.iso`
(if it's not there, click "Browse Local", choose `File System` on the
left, and browse to where you copied/downloaded the Ubuntu ISO image.
* OS type: Linux
* Version: select the nearest, which may be "Ubuntu 11.10"
* Click Forward
* Step 3 of 5
* Memory: 512 MB
* CPUs: 1
* Click Forward
* Step 4 of 5
* Create a disk image on the computer's hard drive
* 4.0 GB
* Uncheck "Allocate entire disk now"
* Click Forward
* Step 5 of 5
* Open "Advanced options"
* Check that the interface selected is "Host device eth0 (Bridge 'br-lan')"
* Click Finish
At this point the VM will start and the console should appear. Continue to
install your Ubuntu virtual machine however you like. Follow the instructions
from the VirtualBox exercise if you wish. We recommend using "Guided -
use entire disk" for partitioning, and please enter
`http://apt.virt.nsrc.org:3142/` for the HTTP proxy. Install the "OpenSSH
Server" package.
When it has finished and rebooted, your VM should come back up. Login,
type "ifconfig" to find what IP address it has come up on. This should be
an address on 10.10.10.X which has been picked dynamically via DHCP.
SSH into this address from your laptop (e.g. using Putty) and login.
If you didn't select OpenSSH Server during the installation, then you can
add it using `apt-get install openssh-server`
Congratulations: you have a created a working VM, it is on the network, and
you have remote access to it!
## Look at the KVM process
You should be able to find the running `kvm` (or qemu-system) process like
this:
~~~
$ ps auxwww | egrep '(kvm|qemu-system)'
~~~
Note the very large number of command line parameters given to kvm. Now
you can see why we need `libvirt` to manage this for us :-)
## Find the disk image file
The disk image file is in the directory `/var/lib/libvirt/images` and you
can find it like this:
~~~
# cd /var/lib/libvirt/images
# ls -slh
~~~
Note that the total size of the file is 4.0GB, but the disk space used (the
left-hand column on the line) is less than this. This means it is a
"sparse" file, because we de-selected "allocate entire disk now".
To view how much space is really used, you can use the `du` (disk use)
command, like so:
~~~
# du -m *
~~~
-m = megabytes
# Using virsh CLI
When you are logged into the VM host platform remotely, e.g. over ssh, it's
often far easier and quicker to interact using the command line rather than
trying to pull back a graphical desktop and GUI.
## Setup
You should be able to run virsh as a non-root user (remember, it's always
a good idea to run as few commands as root as possible). Under Debian you need
to create a config file to make this work.
When logged in as user "nsrc", do the following to create a config file
`.config/libvirt/libvirt.conf` in your home directory, containing one line:
~~~
$ cd
$ mkdir -p .config/libvirt # it may already exist
$ editor .config/libvirt/libvirt.conf
uri_default = "qemu:///system"
~~~
> Note: this is for libvirt from backports. On older libvirt versions it
> is `.libvirt/libvirt.conf` instead
## Simple commands
On the host server (either a terminal window or logged in using ssh),
try the following commands:
~~~
$ virsh list
$ virsh list --all
~~~
The first shows only running VMs, the second shows all defined VMs
(including halted ones).
You can send a shutdown signal like this:
~~~
$ virsh shutdown ubuntu1
~~~
After a few seconds, try `virsh list` again to see if it has shut down.
(If it hasn't - this means `acpid` is not running inside the guest)
To restart the VM:
~~~
$ virsh start ubuntu1
~~~
You can find all the parameters of the VM, such as how much memory and CPU
cores it has, and what disk images are attached, by looking at the XML:
~~~
$ virsh dumpxml ubuntu1 | less
~~~
See if you can find the path to the disk image file in the XML.
(Hit space to advance to next page, 'b' to go back, and 'q' to quit)
The XML files are actually stored under `/etc/libvirt/qemu/`, but it is
safer to manipulate them using the `virsh` commands, which test the
XML file for errors before saving it.
## Adding a serial console
On your host server, try connecting to the serial port of your VM:
~~~
$ virsh console ubuntu1
~~~
Hit Enter a few times and you should find nothing happens. This is because
the VM isn't configured to give a login prompt on the serial port. Hit ctrl
and `]` (right-hand square bracket) to exit.
Firstly, let's check that VM does have an emulated serial port, by editing
the XML:
~~~
$ virsh edit ubuntu1
~~~
Scroll down and check that it contains the following 6 lines:
~~~
~~~
If it does, you can exit the editor without saving. If not, then scroll down
until you find the section defining the mouse:
~~~
~~~
and insert the 6 lines of serial and console XML from above, *before* this
line. Exit the editor, then reboot the virtual machine.
Now login to the virtual machine itself (using the graphical console as you
have been doing so far). Create a file `/etc/init/ttyS0.conf` (as root)
with the following contents:
~~~
start on stopped rc RUNLEVEL=[2345]
stop on runlevel [!2345]
respawn
exec /sbin/getty -L 115200 ttyS0 xterm
~~~
Once you have created this, you should be able to check the status of the
console getty process and start it like this:
~~~
# initctl status ttyS0
# initctl start ttyS0
# initctl status ttyS0
~~~
Check that there is now a "getty" process running on ttyS0:
~~~
# ps auxwww | grep ttyS0
~~~
Now go back to a command line on the *host* server. On there, type:
~~~
$ virsh console ubuntu1
~~~
and then hit Enter. You should this time get a login prompt. Congratulations,
you are now using the emulated serial port (which doesn't require any GUI
to get into). Why use a serial console ? In case you lose network access
to your virtual machine, and you cannot easily get access to the graphical
desktop of your Host machine (if it is for example in a server room somewhere),
then the virtual console will let you access your Linux server.
Login to test, then hit Ctrl and `]` (right-hand square bracket) to
disconnect from the emulated serial console.
# Additional exercises
Please feel free to try these if time is available, or use them as
reference material.
## VNC remote access to virt-manager desktop
Unfortunately, virt-manager can only run under Linux. However it is possible
to create and access a remote Linux desktop so you can use virt-manager
remotely.
Firstly, install the required package:
~~~
# apt-get install vnc4server
~~~
(If you have not installed a desktop environment like LXDE, then you should
also install a simple window manager like "openbox" or "fluxbox")
Now as the "nsrc" user - not as root! - type "vncserver" to start a server.
~~~
nsrc@nodeX:~$ vncserver
You will require a password to access your desktops.
Password:
Verify:
New 'nodeX.virt.nsrc.org:1 (nsrc)' desktop is nodeX.virt.nsrc.org:1
Creating default startup script /home/nsrc/.vnc/xstartup
Starting applications specified in /home/nsrc/.vnc/xstartup
Log file is /home/nsrc/.vnc/nodeX.virt.nsrc.org:1.log
~~~
Note the VNC screen number, in the above example it is :1. (The actual TCP
port number is this value plus 5900).
Now connect using a VNC client on your laptop to `:`,
which would be `nodeX:1` in the above example. You should be prompted for the
password.
If you just get a blank screen, then you are probably in the 'openbox' window
manager. Right-click, select Terminal from the menu, and in the terminal
type 'virt-manager'.
Note that you can disconnect and reconnect from the desktop with VNC, and
it will be just as you left it.
If you want the VNC desktop to have the full LXDE desktop environment, you
need to edit a config file, comment out a couple of lines and add a new line.
~~~
$ cd
$ editor .vnc/xstartup
...
#x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
#x-window-manager &
x-session-manager &
~~~
Now restart your VNC server:
~~~
$ vncserver -kill :1
$ vncserver
~~~
and reconnect.
Note 1: If you reboot your VM server, then you will have to login over ssh and
type "vncserver" again to restart the VNC server. However it will remember
the password you used before.
Note 2: This is *not* the way we recommend to manage Linux systems, for many
reasons. However it does give you a usable way to run virt-manager over
the network.
If you had a cluster of machines, you can run virt-manager on one machine,
and then use `File > Add Connection` to get it to communicate with libvirt
on other machines. Then only the manager machine needs to have a graphical
desktop. [^2]
[^2]: If you're running Linux on your own computer, you could manage
libvirt on other servers from your own machine. More on this
[here](http://libvirt.org/uri.html).
## libvirt storage pools
You can create 'storage pools' to make them available to libvirt. Here is
how to set up an LVM pool. Do this as root. You can create the file "ganeti.xml"
wherever you want. We'll create the file in the /root directory.
~~~
# cd
# editor ganeti.xml
ganeti/dev/ganeti
# virsh pool-define ganeti.xml
# virsh pool-list --all
# virsh pool-start ganeti
# virsh pool-autostart ganeti
# virsh pool-list
~~~
Now you have the option of creating VMs within the "ganeti" pool, which
will create logical volumes in that volume group.
## virt-install
The virt-install tool allows you to create and start a VM, allocate disk
and attach CD image, without having to write XML and without having to
use the virt-manager GUI.
This is very useful if you want to do all your management using virsh CLI.
~~~
# apt-get install virtinst
# virt-install --name foo --ram 256 \
--cdrom /var/lib/libvirt/images/ubuntu-14.04.1-server-i386.iso \
--disk pool=default,size=4 --network=bridge:br-lan \
--noautoconsole --graphics vnc,listen=0.0.0.0,password=xyzzy
~~~
(Use `pool=ganeti,size=4` if you have set up an LVM storage pool and want
to use a logical volume rather than an image file)
With `--noautoconsole` it does not attempt to start the `virt-viewer` X11
application to attach to the console. Instead, you can use a VNC viewer on
your laptop to connect to the console.
To find out which VNC port to connect to, type:
~~~
# virsh vncdisplay foo
~~~
This will return something like `:0` or `:2` which is the VNC display number.
(Add 5900 to get the VNC TCP port number).
Start your VNC viewer, tell it to connect to `nodeX.virt.nsrc.org:` and
you should have a password-protected graphical console onto the VM.
## Install VM with QCOW2 disk iamge
By default, virt-manager creates raw files (sparse or pre-allocated)
If you want to create a different format, e.g. QCOW2, or create an image
in a different storage pool, this is what you do.
When creating a virtual machine, at step 4 of 5 ("Enable storage for this
virtual machine") click "Select managed or other existing storage", then
"Browse".
At this point you can select a storage pool from the left-hand pane and
click New Volume. This allows you to choose a name for the volume and its
format (e.g. QCOW2) and size. Note that you cannot choose the format if
you choose an LVM based storage pool - therefore choose the `default`
pool to create a disk image in QCOW2 format.
When the volume has been created, select it in the right-hand pane, then
click "Choose Volume" to continue with the installation using this volume.
## libvirt snapshots
Snaphots are supported using the qcow2 format; the disk image file contains
both the disk snapshots and the CPU/RAM state. You can try them out.
virt-manager GUI support for snapshots is [planned](https://fedoraproject.org/wiki/Changes/Virt_Manager_Snapshots)
but for now use the command line:
~~~
virsh snapshot-create
virsh snapshot-list
virsh snapshot-revert
~~~
You can try creating temporary files in the '/run' directory (which is a
RAM disk) and watching how they change back when you revert a snapshot.
If a machine is shutdown, you can use `snapshot-revert` to start it
immediately at the point where the snapshot was taken.