10. Creating virtual machines
10.1. Motivation and plan
Things are converging to where we will always deploy on virtual machines …
Plan: heavy reliance on the number 3.
Choose VM guest OSes (example: Debian unstable, Ubuntu 16.04, CentOS 7)
Choose up to three “killer apps” – web-based applications to run off of these machines (examples: OwnCloud, ArchivesSpace, Trac, Jenkins, MediaWiki, Snipe-IT, a simple Django backend you can write up, a backup server…)
10.2. Physical base
We write these instructions using Ubuntu 16.04 as the host operating system. This can obviously be changed.
You host the VMs on your own computer. Make sure you have enough RAM and hard disk space to handle creating one to three VMs.
Also make sure that hardware virtualization support is set. This is special hardware
egrep '(vmx|svm)' --color=always /proc/cpuinfo
You should see strings with “vmx” or “svm” in them. If you don’t it means that your hardware is not configured to support virtualization, which means that running a VM would be very slow.
The place to configure this is in the BIOS settings: enable the option for hardware virtualization support and reboot and check again.
10.3. ISO images for the guest operating systems
Our goal is to flexibly experiment with installing a variety of operating systems. Let’s get some ISO images handy for this work.
Debian weekly snapshot “netinst” image, about 300 megabytes.
Ubuntu 16.04.4 LTS (Xenial Xerus), about 1.5 gigabytes. Let’s also get the server image, which lends itself better to automated installation and doesn’t have unnecessary software. The server image is about 850 meg.
For good measure, also get the analogous Ubuntu 18.04 images (Bionic Beaver).
CentOS 7 “DVD” ISO, about 4 gigabytes.
You can download them with the following instructions, which you can paste in all together.
sudo mkdir -p /usr/local/src
sudo chown $LOGNAME /usr/local/src
mkdir -p /usr/local/src/cd-images
cd /usr/local/src/cd-images/
wget --continue https://cdimage.debian.org/cdimage/weekly-builds/amd64/iso-cd/debian-testing-amd64-netinst.iso
wget --continue http://releases.ubuntu.com/16.04/ubuntu-16.04.4-server-amd64.iso
wget --continue http://releases.ubuntu.com/18.04/ubuntu-18.04-server-amd64.iso
wget --continue http://centos.s.uw.edu/centos/7/isos/x86_64/CentOS-7-x86_64-DVD-1708.iso
10.4. Install VMs from a GUI
Let us first experiment with installing a VM using the virt-manager GUI. This should be analogous to such installations from other virtual machine systems.
Start by installing virt-manager and qemu-kvm:
sudo apt install virt-manager qemu-kvm
Make sure that you add yourself to the group libvirtd, and then reboot and log in again. This will make sure that you are a member of libvirtd when you log in again.
Now, as yourself, run:
virt-manager &
and use the GUI to create yourself a VM with, let’s say, 2gig of RAM and an 80gig hard drive.
Then use the GUI to point the virtual host’s CDROM to
/usr/local/src/cd-images/ubuntu-server-16.04.4-amd64.iso
Now you can start the VM and install the operating system. Then you could exercise your new guest OS by following the instructions in some of the other chapters of this book where we set up a variety of services.
10.5. Automated installation
[ added 2019-04-30: found this URL that seems to do it all https://raymii.org/s/articles/virt-install_introduction_and_copy_paste_distro_install_commands.html
more info on kickstart files and proxies is at http://vijayjt.blogspot.com/2017/03/kvm-automation.html
there are problems when you’re behind a proxy. this question puts it well, but it has no answers yet: https://serverfault.com/questions/941671/redirect-all-web-trafic-from-network-through-proxy-server-to-a-specific-url
]
10.5.1. virt-manager
GUI installation is pleasant, but it is slow. There is a program
virt-install which does the actual “under the hood” job of
installing a VM.
https://linuxconfig.org/how-to-create-and-manage-kvm-virtual-machines-from-cli
A typical command to create a VM might look like this:
virt-install --name=linuxconfig-vm \
--name debian-test \
--vcpus=1 \
--memory=1024 \
--cdrom=/usr/local/src/cd-images/debian-testing-amd64-netinst.iso \
--disk size=5 \
--os-variant=debian8
to boot, stop and remove-and-cleanup:
virsh boot debian-teset
virsh destroy debian-test
virsh undefine debian-test
or like this:
qemu-img create -f qcow2 ./centos7.qcow2 8G
virt-install --name centos7 --ram 1024 --disk path=./centos7.qcow2,size=8 --vcpus 1 --os-type linux --os-variant rhel7 --network network=default --graphics none --console pty,target_type=serial --location 'http://repo.miserver.it.umich.edu/centos/7.6.1810/os/x86_64/' --extra-args 'console=ttyS0,115200n8'
to boot, stop and remove-and-cleanup:
virsh boot centos7
virsh destroy centos7
virsh undefine centos7
The first approach is problematic because it does not carry out the installation and you still have to do a lot by hand. The second approach has the same problem and in addition it installs entirely from the web instead of from a local CD, so it will be unusable from a slow network connection.
To really achive unattended installation look at the procedures in Section 10.6.2 and Section 10.6.3
Tip
(this tip is out of sequence; should be when we do ks/preseed) To generate an encrypted password to use in a kickstarter or preseed file you can do this:
python -c 'import crypt,getpass; print(crypt.crypt(getpass.getpass(), crypt.mksalt(crypt.METHOD_SHA512)))'
Other sites:
10.5.2. virtualbox
Preparation:
Installation from GUI
Automated installation
10.6. Unattended installation of some operating systems
10.6.1. Concepts
The parts of an unattended installation are:
Preparing a hard disk image.
Invoking virt-install with the following parameters:
A naming of the VM within virt-manager.
Memory.
Injection of a kickstarter or preseed file.
Location from which to install the OS. This can be a CD image or a web location or a location in the filesystem.
A specification of how to answer the installation questions. On debian-based systems (such as debian and ubuntu) this is in a file called
preseed.cfg, while on redhat-based systems it’s in a kickstarter file which often ends in.ks
The hard disk image can be prepared with the qemu-img create
command. An example of the syntax is qemu-img create -f qcow2
$DISK_IMAGE 100G.
The naming of the VM is done with the --name option, and RAM with
the --memory option.
The location from which to install is specified with either the
--cdrom option or the --location option. There are subtle
differences and rules on how to use these options, so I usually figure
out a recipe and just follow it.
10.6.2. Unattended Ubuntu installation
I usually use the script make-minimal-ubuntu.sh in
Listing 10.6.2.1 to make a new unattended ubuntu
VM. This will make use of a preseed file.
preseed.cfg, shown in Listing 10.6.2.2.#! /bin/sh
#HOST_ID=unattendedU
HOST_ID=tryu3
virsh destroy ${HOST_ID}
virsh undefine ${HOST_ID}
## ubuntu does not need us to mount the ISO file, so we just point to
## the ISO
INSTALL_ISO=/usr/local/src/cd-images/ubuntu-16.04.4-server-amd64.iso
#INSTALL_ISO=/usr/local/src/cd-images/ubuntu-18.04-live-server-amd64.iso
#INSTALL_ISO=/usr/local/src/cd-images/ubuntu-18.04-desktop-amd64.iso
#INSTALL_ISO=/usr/local/src/cd-images/ubuntu-18.04-mini.iso
DISK_IMAGE=/data1/vm-images/${HOST_ID}.qcow2
/bin/rm -f $DISK_IMAGE
qemu-img create -f qcow2 $DISK_IMAGE 100G
sudo virt-install \
--name ${HOST_ID} \
--vcpus 2 \
--memory 4096 \
--disk size=100,bus=virtio,format=qcow2 \
--boot cdrom,hd \
--network network=default \
--graphics vnc \
--location $INSTALL_ISO \
--initrd-inject=preseed.cfg \
--extra-args="auto=true netcfg/use_autoconfig=true netcfg/disable_dhcp=false netcfg/get_hostname=${HOST_ID} file=file:/preseed.cfg"
## this tiny list is based on:
## https://askubuntu.com/questions/806820/how-do-i-create-a-completely-unattended-install-of-ubuntu-desktop-16-04-1-lts
### Partitioning
# d-i partman-auto/disk string /dev/sda
d-i partman-auto/method string lvm
d-i partman-auto/method string regular
d-i partman-lvm/device_remove_lvm boolean true
d-i partman-md/device_remove_md boolean true
d-i partman-auto/choose_recipe select atomic
# This makes partman automatically partition without confirmation
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true
# Locale
d-i debian-installer/locale string en_US
d-i console-setup/ask_detect boolean false
d-i console-setup/layoutcode string us
# Network
d-i netcfg/get_hostname string unassigned-hostname
d-i netcfg/get_domain string unassigned-domain
d-i netcfg/choose_interface select auto
# Clock
d-i clock-setup/utc-auto boolean true
d-i clock-setup/utc boolean true
d-i time/zone string US/Mountain
d-i clock-setup/ntp boolean true
# Packages, Mirrors, Image
d-i base-installer/kernel/override-image string linux-server
d-i base-installer/kernel/override-image string linux-image-amd64
d-i mirror/http/mirror select us.archive.ubuntu.com
d-i mirror/country string manual
d-i mirror/http/proxy string
#d-i mirror/http/proxy string http://proxyout.lanl.gov:8080/
d-i mirror/http/hostname string us.archive.ubuntu.com
d-i mirror/http/directory string /ubuntu
apt-mirror-setup apt-setup/restricted boolean true
apt-mirror-setup apt-setup/universe boolean true
apt-mirror-setup apt-setup/multiverse boolean true
apt-mirror-setup apt-setup/backports boolean true
apt-mirror-setup apt-setup/partner boolean false
d-i pkgsel/update-policy select none
# tasksel tasksel/first multiselect ubuntu-desktop
d-i apt-setup/restricted boolean true
d-i apt-setup/universe boolean true
d-i pkgsel/install-language-support boolean false
d-i pkgsel/include string openssh-server cryptsetup build-essential libssl-dev libreadline-dev zlib1g-dev linux-source dkms nfs-common
d-i pkgsel/include string openssh-server
d-i pkgsel/include string wget
d-i pkgsel/include string zip
d-i pkgsel/include string unzip
d-i pkgsel/include string build-essential
d-i pkgsel/include string wget curl firewalld
tasksel tasksel/first multiselect standard
# Users
d-i passwd/user-fullname string ubuntu
d-i passwd/username string ubuntu
d-i passwd/user-password password ubuntu
d-i passwd/user-password-again password ubuntu
# d-i passwd/user-password-crypted password [crpyt 3]
d-i passwd/root-login boolean true
d-i passwd/root-password ubuntu
d-i passwd/root-password-again ubuntu
d-i user-setup/allow-password-weak boolean true
# Set to true if you want to encrypt the first user's home directory.
d-i user-setup/encrypt-home boolean false
# Grub
d-i grub-installer/grub2_instead_of_grub_legacy boolean true
d-i grub-installer/only_debian boolean true
d-i finish-install/reboot_in_progress note
# Custom Commands
You now have a new VM, ready to work with, which was installed with no
human intervention after the virt-install command. It’s running
ubuntu 16.04.4
You will find this VM in the virt-manager list of VMs. You can bring up its console and log in to find out what the IP address is, and to install openssh-server.
10.6.3. Unattended CentOS7/RHEL7 installation
[incomplete; I have the minimal-el7.ks file and the make-minimal-el7.sh files, but I have not yet fixed the write-up in this section.]
After a minimal install of CentOS7 you don’t even have the network coming up automatically. Log in as root and do:
sed -i 's/ONBOOT=no/ONBOOT=yes/' /etc/sysconfig/network-scripts/ifcfg-ens3
reboot
Then log in as root again and you will have a network. Install some basic s/w before following the procedure for Archivesspace.
## for good measure, update the system
yum update -y
sudo yum install -y net-tools curl wget openssh-server spice-vdagent
sudo yum install -y telnet zip unzip lynx links
ifconfig ## then take not of our ip address
At this point you can bypass the console and ssh in from your workstation, which is more convenient, with:
ssh root@ip_address_found_with_ifconfig
Then log in from a terminal on your own workstation.
ssh ip_address_found_with_ifconfig
10.6.3.1. Install Apache, PHP and MariaDB
and on the CentOS7 VM we now prepare the stack:
sudo yum -y install epel-release
sudo yum -y update
sudo shutdown -r now
## after reboot, ssh back in and do:
sudo yum -y install httpd
## Start Apache and enable it to automatically run at boot time.
sudo systemctl start httpd.service
sudo systemctl enable httpd.service