**Debian, KVM and libvirt** # Objectives Working in pairs, you will build a Linux-based virtualization host server on hardware provided in the class. * Install Debian Linux. (This is because Ganeti, which we will be using later, is better supported under Debian than Ubuntu) * Configure LVM during installation with manual partitioning so that most of the volume group is unused * Use KVM with the `virt-manager` GUI and the `virsh` command-line interface to install and manage a VM. NOTE: please take care to install LVM as shown, creating some small logical volumes for the system but leaving most of the volume group unused, because later exercises will need to create more logical volumes from the free space. Please also name your volume group "ganeti", because this has to be consistent throughout a cluster of machines. # Install Debian It may be necessary to press a key (e.g. F12) to get the option to boot from the USB stick. If you get the Debian splash screen, select "install" instead of "graphical install". * Language: English * Country: as appropriate * Locale settings: United States * Keymap: American English * Hostname: as allocated by instructor (nodeX where X is a number) * Domain name: virt.nsrc.org * Root password: use the class root password * Full name for new user: NSRC * Username: "nsrc" * Password: class user password * Select your time zone if prompted ## Partitioning * Select "Manual" (and hit Enter) * Select your disk drive (sda) * (If there is no partition table, you'll be prompted to create one) * Delete existing partition(s) * Instructors will tell you the best approach * If there's a large Windows partition, delete it * If there's a Recovery partition, keep it * Select "pri/log FREE SPACE" * Press ENTER to create a new partition * New partition size: 1G (this is the /boot filesystem) * Primary * Beginning * Use as: Ext4 journaling file system * Mount point: /boot - static files of the boot loader * Bootable flag: on * Done setting up the partition * Select "pri/log FREE SPACE" * Press ENTER to create a new partition * Accept whatever size is shown (the rest of the disk) * Primary * Use as: physical volume for LVM * Done setting up the partition * Configure the Logical Volume Manager * Write changes to disks and configure LVM? Yes * Create volume group * Volume group name: ganeti * Devices for the new volume group: select only the LVM partition (probably /dev/sda2) and hit SPACE to display '[*]', then hit Enter to continue. * Create logical volume * Volume group: ganeti * Logical volume name: root * Logical volume size: 4GB * Create logical volume * Volume group: ganeti * Logical volume name: swap * Logical volume size: 4GB * Create logical volume * Volume group: ganeti * Logical volume name: var * Logical volume size: 20GB * Finish * Select the entry #1 underneath LV *root* ~~~ LVM VG ganeti, LV root - 4.0 GB Linux device-mapper (linear) #1 4.0 GB <<<<<<<< select this <<<<<<< ~~~ * Select: * Use as: Ext4 journaling file system * Mount point: / - the root filesystem * Done setting up the partition * Select the entry #1 underneath LV *swap* * Use as: swap area * Done setting up the partition * Select the entry #1 underneath LV *var* * Use as: Ext4 journalling file system * Mount point: /var - variable data * Done setting up the partition * Finish partitioning and write changes to disk (might need to scroll down to see this option) * Write the changes to disks? Yes ## Software selection * Use a network mirror? Yes * Check with the instructor which country/mirror to use. In the absence of any guidance, [use these](https://wiki.debian.org/DebianGeoMirror) * Country: United States * Mirror: cdn.debian.net * HTTP proxy: `http://apt.virt.nsrc.org:3142/` * Participate in the package usage survey? No * Software to install. Select only these: * [X] SSH server * [X] Standard system utilities * (Do not select "Debian desktop environment" or you will get tons of applications including LibreOffice, Iceweasel, Gimp, etc) * (To save time, do not select "Laptop" even if this is a laptop. You can install package "task-laptop" later if you wish) * Install the GRUB boot loader to the master boot record? Yes * Installation complete - hit Continue You should now boot into the system. ## Basic configuration Login as root. The first thing you will need to do is to get the installer to forget about the CD-ROM. As root, edit the file `/etc/apt/sources.list`, for example using vi: ~~~ # editor /etc/apt/sources.list ~~~ Comment out any line which begins "deb cdrom:..." by inserting a hash (`#`) in front of it, changing it from this: ~~~ deb cdrom:[Debian GNU/Linux 7.7.0 [...] ~~~ To this: ~~~ # deb cdrom:[Debian GNU/Linux 7.7.0 [...] ~~~ Then save the file. Install `sudo`, and add the 'nsrc' user into the 'sudo' group ~~~ # apt-get update # apt-get install sudo # usermod -G sudo -a nsrc ~~~ Also install `vim` which is an improved version of the vi editor ~~~ # apt-get install vim ~~~ ## Network configuration Your server may have picked up a dynamic IP address via DHCP, which you can find out using the `ifconfig` command: ~~~ # ifconfig eth0 ~~~ Now you are going to give it a static IP address and also configure bridging so that your virtual machines can share the same network interface. Your machine will be given IP address 10.10.10.X ### Edit `/etc/hostname` ~~~ nodeX.virt.nsrc.org ~~~ ### Edit `/etc/hosts`, and change it to look like this: ~~~ 127.0.0.1 localhost 10.10.10.X nodeX.virt.nsrc.org nodeX ~~~ ### Install additional networking packages ~~~ # apt-get install bridge-utils vlan ~~~ ### Edit `/etc/network/interfaces` Change the file so that it looks like this. This removes the (dynamic) IP address from eth0, and instead creates a bridge interface "br-lan" with a static IP address, and eth0 a member of the bridge. ~~~ # The loopback network interface auto lo iface lo inet loopback # Management interface auto eth0 iface eth0 inet manual auto br-lan iface br-lan inet static address 10.10.10.X netmask 255.255.255.0 gateway 10.10.10.254 bridge_ports eth0 bridge_stp off bridge_fd 0 bridge_maxwait 0 ~~~ You can activate your changes like this: ~~~ # ifdown eth0 # killall dhclient # ifup br-lan # brctl show bridge name bridge id STP enabled interfaces br-lan 8000.xxxxxxxxxxxx no eth0 ~~~ You should see your new IP address on `ifconfig br-lan`, and you should still be able to ping out (e.g. `ping 8.8.8.8`) ### Edit `/etc/resolv.conf` ~~~ domain virt.nsrc.org nameserver 10.10.10.241 ~~~ Check you can still resolve names (e.g. `ping apt.virt.nsrc.org`) ### Reboot As a final check, reboot your server. ~~~ # reboot ~~~ It should come up on the same IP address again. ### Test ssh You should now be able to ssh into your server as user "nsrc" (e.g. using putty under Windows). Check you are able to do this. ssh to either `nodeX.virt.nsrc.org` or `10.10.10.X` Both people in the pair should be able to connect to the server from their laptop, so they don't need to share the console. NOTE: get into the habit of logging into the server as a normal user, and try to avoid logging in as "root". You will see a prompt which ends with `$`. It's advisable to disable logging in as root directly over SSH (not covered here). If there is a single command you wish to run as root, prefix it with `sudo`. If you have a series of commands to run as root, start a root shell with `sudo -s`, and the prompt will change to `#`. When you have finished, type `exit` to leave the root shell. # Virtual machine tools ## Install KVM and `libvirt` ~~~ # apt-get install qemu-kvm libvirt-bin ~~~ Now add the 'nsrc' user into the 'libvirt' group. ~~~ # usermod -G libvirt -a nsrc ~~~ This is needed so you can talk to the libvirt daemon. If you are currently logged in as 'nsrc' you will need to logout and login again to pick up this group. The command "id" will show you what groups you are a member of. ## Check for hardware virtualization support At a command line prompt, type the following: ~~~ $ egrep '(vmx|svm)' /proc/cpuinfo ~~~ If you get one or more lines containing "vmx" or "svm" then the processor has hardware virtualization capabilities. However this doesn't tell you if it's enabled in the BIOS. To verify, do: ~~~ $ ls -l /dev/kvm ~~~ If you see: ~~~ crw-rw---- 1 root kvm 10, 232 Jan 17 21:24 /dev/kvm ~~~ Then it's OK [^1] [^1]: On Ubuntu, The `kvm-ok` utility, included in the package `cpu-checker`, will tell you. If KVM acceleration is not available, then you may need to reboot into the BIOS settings and enable VT-x (Intel) or AMD-V (AMD). ## Desktop environment Install an X11 desktop (which we want just so that we can run virt-manager) ~~~ # apt-get install xorg lxde lightdm ssh-askpass --no-install-recommends ~~~ This will take a while - be patient! Once the install is finished, start the graphical environment: ~~~ # service lightdm start ~~~ You should now be able to login to the graphical environment as the "nsrc" user. You can get a shell window using `Start > Accessories > LXTerminal`. You can also switch between text and graphical consoles using Ctrl-Alt-F1 and Ctrl-Alt-F7 respectively. ## Install virt-manager GUI ~~~ # apt-get install virt-manager ~~~ This installs a lot of extra packages because it depends on a lot of graphics libraries. # Create a VM On the virtualization host you have just built, you are going to install Ubuntu Server in a VM, using the virt-manager GUI which is similar to the VirtualBox GUI. ## Copy Ubuntu ISO image First, copy the file `ubuntu-14.04.1-server-i386.iso` into the directory `/var/lib/libvirt/images` on your host. How to do this depends on where you are copying the image from. * If you are copying from a USB stick: insert the stick. If you have a GUI it will ask you if you want to open with the File Manager. Say yes, and it should mount the USB stick automatically (on `/media/XXX`), and open a window. (XXX will be a random number, look at the title of the window) You can then copy the ISO: ~~~ # cp /media/XXX/iso/ubuntu-14.04.1-server-i386.iso /var/lib/libvirt/images/ ~~~ If it doesn't work, you may need to mount it by hand: ~~~ # dmesg | tail # find the ID of the device just inserted, e.g. sdb # mount /dev/sdb1 /mnt # ls /mnt # find the file # ls /mnt/iso # find the file # cp /mnt/iso/ubuntu-14.04.1-server-i386.iso /var/lib/libvirt/images/ # umount /mnt ~~~ * If you are copying from a web server: ~~~ # cd /var/lib/libvirt/images # wget http://download.virt.nsrc.org/ubuntu-14.04.1-server-i386.iso ~~~ ## Start the virt-manager GUI Use `Start > System Tools > Virtual Machine Manager`, or just type `virt-manager` at a terminal prompt while in the GUI. NOTE: to release the keyboard and mouse, press left-CTRL and left-ALT together. ## Create a VM In the Virtual Machine Manager window: * Click "New" (this is the PC-shaped button under the File menu) * Step 1 of 5 * Enter a name, e.g. "ubuntu1" * Select "Local install media (ISO image or CDROM)" * Click Forward * Step 2 of 5 * Check "Use ISO image" is selected * Click "Browse", and select `ubuntu-14.04.1-server-i386.iso` (if it's not there, click "Browse Local", choose `File System` on the left, and browse to where you copied/downloaded the Ubuntu ISO image. * OS type: Linux * Version: select the nearest, which may be "Ubuntu 11.10" * Click Forward * Step 3 of 5 * Memory: 512 MB * CPUs: 1 * Click Forward * Step 4 of 5 * Create a disk image on the computer's hard drive * 4.0 GB * Uncheck "Allocate entire disk now" * Click Forward * Step 5 of 5 * Open "Advanced options" * Check that the interface selected is "Host device eth0 (Bridge 'br-lan')" * Click Finish At this point the VM will start and the console should appear. Continue to install your Ubuntu virtual machine however you like. Follow the instructions from the VirtualBox exercise if you wish. We recommend using "Guided - use entire disk" for partitioning, and please enter `http://apt.virt.nsrc.org:3142/` for the HTTP proxy. Install the "OpenSSH Server" package. When it has finished and rebooted, your VM should come back up. Login, type "ifconfig" to find what IP address it has come up on. This should be an address on 10.10.10.X which has been picked dynamically via DHCP. SSH into this address from your laptop (e.g. using Putty) and login. If you didn't select OpenSSH Server during the installation, then you can add it using `apt-get install openssh-server` Congratulations: you have a created a working VM, it is on the network, and you have remote access to it! ## Look at the KVM process You should be able to find the running `kvm` (or qemu-system) process like this: ~~~ $ ps auxwww | egrep '(kvm|qemu-system)' ~~~ Note the very large number of command line parameters given to kvm. Now you can see why we need `libvirt` to manage this for us :-) ## Find the disk image file The disk image file is in the directory `/var/lib/libvirt/images` and you can find it like this: ~~~ # cd /var/lib/libvirt/images # ls -slh ~~~ Note that the total size of the file is 4.0GB, but the disk space used (the left-hand column on the line) is less than this. This means it is a "sparse" file, because we de-selected "allocate entire disk now". To view how much space is really used, you can use the `du` (disk use) command, like so: ~~~ # du -m * ~~~ -m = megabytes # Using virsh CLI When you are logged into the VM host platform remotely, e.g. over ssh, it's often far easier and quicker to interact using the command line rather than trying to pull back a graphical desktop and GUI. ## Setup You should be able to run virsh as a non-root user (remember, it's always a good idea to run as few commands as root as possible). Under Debian you need to create a config file to make this work. When logged in as user "nsrc", do the following to create a config file `.config/libvirt/libvirt.conf` in your home directory, containing one line: ~~~ $ cd $ mkdir -p .config/libvirt # it may already exist $ editor .config/libvirt/libvirt.conf uri_default = "qemu:///system" ~~~ > Note: this is for libvirt from backports. On older libvirt versions it > is `.libvirt/libvirt.conf` instead ## Simple commands On the host server (either a terminal window or logged in using ssh), try the following commands: ~~~ $ virsh list $ virsh list --all ~~~ The first shows only running VMs, the second shows all defined VMs (including halted ones). You can send a shutdown signal like this: ~~~ $ virsh shutdown ubuntu1 ~~~ After a few seconds, try `virsh list` again to see if it has shut down. (If it hasn't - this means `acpid` is not running inside the guest) To restart the VM: ~~~ $ virsh start ubuntu1 ~~~ You can find all the parameters of the VM, such as how much memory and CPU cores it has, and what disk images are attached, by looking at the XML: ~~~ $ virsh dumpxml ubuntu1 | less ~~~ See if you can find the path to the disk image file in the XML. (Hit space to advance to next page, 'b' to go back, and 'q' to quit) The XML files are actually stored under `/etc/libvirt/qemu/`, but it is safer to manipulate them using the `virsh` commands, which test the XML file for errors before saving it. ## Adding a serial console On your host server, try connecting to the serial port of your VM: ~~~ $ virsh console ubuntu1 ~~~ Hit Enter a few times and you should find nothing happens. This is because the VM isn't configured to give a login prompt on the serial port. Hit ctrl and `]` (right-hand square bracket) to exit. Firstly, let's check that VM does have an emulated serial port, by editing the XML: ~~~ $ virsh edit ubuntu1 ~~~ Scroll down and check that it contains the following 6 lines: ~~~ ~~~ If it does, you can exit the editor without saving. If not, then scroll down until you find the section defining the mouse: ~~~ ~~~ and insert the 6 lines of serial and console XML from above, *before* this line. Exit the editor, then reboot the virtual machine. Now login to the virtual machine itself (using the graphical console as you have been doing so far). Create a file `/etc/init/ttyS0.conf` (as root) with the following contents: ~~~ start on stopped rc RUNLEVEL=[2345] stop on runlevel [!2345] respawn exec /sbin/getty -L 115200 ttyS0 xterm ~~~ Once you have created this, you should be able to check the status of the console getty process and start it like this: ~~~ # initctl status ttyS0 # initctl start ttyS0 # initctl status ttyS0 ~~~ Check that there is now a "getty" process running on ttyS0: ~~~ # ps auxwww | grep ttyS0 ~~~ Now go back to a command line on the *host* server. On there, type: ~~~ $ virsh console ubuntu1 ~~~ and then hit Enter. You should this time get a login prompt. Congratulations, you are now using the emulated serial port (which doesn't require any GUI to get into). Why use a serial console ? In case you lose network access to your virtual machine, and you cannot easily get access to the graphical desktop of your Host machine (if it is for example in a server room somewhere), then the virtual console will let you access your Linux server. Login to test, then hit Ctrl and `]` (right-hand square bracket) to disconnect from the emulated serial console. # Additional exercises Please feel free to try these if time is available, or use them as reference material. ## VNC remote access to virt-manager desktop Unfortunately, virt-manager can only run under Linux. However it is possible to create and access a remote Linux desktop so you can use virt-manager remotely. Firstly, install the required package: ~~~ # apt-get install vnc4server ~~~ (If you have not installed a desktop environment like LXDE, then you should also install a simple window manager like "openbox" or "fluxbox") Now as the "nsrc" user - not as root! - type "vncserver" to start a server. ~~~ nsrc@nodeX:~$ vncserver You will require a password to access your desktops. Password: Verify: New 'nodeX.virt.nsrc.org:1 (nsrc)' desktop is nodeX.virt.nsrc.org:1 Creating default startup script /home/nsrc/.vnc/xstartup Starting applications specified in /home/nsrc/.vnc/xstartup Log file is /home/nsrc/.vnc/nodeX.virt.nsrc.org:1.log ~~~ Note the VNC screen number, in the above example it is :1. (The actual TCP port number is this value plus 5900). Now connect using a VNC client on your laptop to `:`, which would be `nodeX:1` in the above example. You should be prompted for the password. If you just get a blank screen, then you are probably in the 'openbox' window manager. Right-click, select Terminal from the menu, and in the terminal type 'virt-manager'. Note that you can disconnect and reconnect from the desktop with VNC, and it will be just as you left it. If you want the VNC desktop to have the full LXDE desktop environment, you need to edit a config file, comment out a couple of lines and add a new line. ~~~ $ cd $ editor .vnc/xstartup ... #x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" & #x-window-manager & x-session-manager & ~~~ Now restart your VNC server: ~~~ $ vncserver -kill :1 $ vncserver ~~~ and reconnect. Note 1: If you reboot your VM server, then you will have to login over ssh and type "vncserver" again to restart the VNC server. However it will remember the password you used before. Note 2: This is *not* the way we recommend to manage Linux systems, for many reasons. However it does give you a usable way to run virt-manager over the network. If you had a cluster of machines, you can run virt-manager on one machine, and then use `File > Add Connection` to get it to communicate with libvirt on other machines. Then only the manager machine needs to have a graphical desktop. [^2] [^2]: If you're running Linux on your own computer, you could manage libvirt on other servers from your own machine. More on this [here](http://libvirt.org/uri.html). ## libvirt storage pools You can create 'storage pools' to make them available to libvirt. Here is how to set up an LVM pool. Do this as root. You can create the file "ganeti.xml" wherever you want. We'll create the file in the /root directory. ~~~ # cd # editor ganeti.xml ganeti /dev/ganeti # virsh pool-define ganeti.xml # virsh pool-list --all # virsh pool-start ganeti # virsh pool-autostart ganeti # virsh pool-list ~~~ Now you have the option of creating VMs within the "ganeti" pool, which will create logical volumes in that volume group. ## virt-install The virt-install tool allows you to create and start a VM, allocate disk and attach CD image, without having to write XML and without having to use the virt-manager GUI. This is very useful if you want to do all your management using virsh CLI. ~~~ # apt-get install virtinst # virt-install --name foo --ram 256 \ --cdrom /var/lib/libvirt/images/ubuntu-14.04.1-server-i386.iso \ --disk pool=default,size=4 --network=bridge:br-lan \ --noautoconsole --graphics vnc,listen=0.0.0.0,password=xyzzy ~~~ (Use `pool=ganeti,size=4` if you have set up an LVM storage pool and want to use a logical volume rather than an image file) With `--noautoconsole` it does not attempt to start the `virt-viewer` X11 application to attach to the console. Instead, you can use a VNC viewer on your laptop to connect to the console. To find out which VNC port to connect to, type: ~~~ # virsh vncdisplay foo ~~~ This will return something like `:0` or `:2` which is the VNC display number. (Add 5900 to get the VNC TCP port number). Start your VNC viewer, tell it to connect to `nodeX.virt.nsrc.org:` and you should have a password-protected graphical console onto the VM. ## Install VM with QCOW2 disk iamge By default, virt-manager creates raw files (sparse or pre-allocated) If you want to create a different format, e.g. QCOW2, or create an image in a different storage pool, this is what you do. When creating a virtual machine, at step 4 of 5 ("Enable storage for this virtual machine") click "Select managed or other existing storage", then "Browse". At this point you can select a storage pool from the left-hand pane and click New Volume. This allows you to choose a name for the volume and its format (e.g. QCOW2) and size. Note that you cannot choose the format if you choose an LVM based storage pool - therefore choose the `default` pool to create a disk image in QCOW2 format. When the volume has been created, select it in the right-hand pane, then click "Choose Volume" to continue with the installation using this volume. ## libvirt snapshots Snaphots are supported using the qcow2 format; the disk image file contains both the disk snapshots and the CPU/RAM state. You can try them out. virt-manager GUI support for snapshots is [planned](https://fedoraproject.org/wiki/Changes/Virt_Manager_Snapshots) but for now use the command line: ~~~ virsh snapshot-create virsh snapshot-list virsh snapshot-revert ~~~ You can try creating temporary files in the '/run' directory (which is a RAM disk) and watching how they change back when you revert a snapshot. If a machine is shutdown, you can use `snapshot-revert` to start it immediately at the point where the snapshot was taken.