Creating virtual machiness on Linux --- the hard way

13 minutes read (About 1986 words)

Two ways of using KVM-based vms on Linux are introduced. One is using virt-manager GUI, another is using the raw qemu-system.

I. Prerequisites

1
2
pacman -S qemu bridge-utils iptables dnsmasq ebtables
pacman -S libvirt

check for module loading:

1
2
lsmod | grep kvm
lsmod | grep virtio

To auto load a module at boot, create a file like ‘/etc/modules-load.d/virtio.conf’

1
2
# Load virtio at boot
virtio

II. Virt Manager

1
pacman -S virt-manager
1
2
sudo usermod --append --groups libvirt $(whoami)
sudo systemctl start libvirtd

check for nat:

1
2
sudo virsh net-list --all
# sudo virsh net-start default

Then install using GUI.
if you want a raw image, create the img file by yourself

1
qemu-img create -f raw imgfile 20G

II.A Win7 VM installation and usage

aur install virtio-win
pacman install dmidecode

Under hard disk OS-images directory, find the file windows7_Ultimate_x64.iso and copy it into /var/lib/libvirt/images.
This is a ghost OS image with some preinstall software. I use this because the activation tool is inside.
Screenshot-win7-activation.png

Just put the script and Windows Loader.ext in the same directory and then run the script in Administrator.

Real time clock

In virt manager, enable XML editing.
Then replace <clock offset='utc'> with <clock offset='localtime'>
Administrator open cmd:

1
bcdedit /set {default} USEPLATFORMCLOCK on

Office 2016 pack

mount the iso file: Office_2016_x64_ZH_16.0.7466.2038.iso
off-line installer:
Screenshot-office16.png
start one of the Office app
convert and activate

QXL Acceleration

1
2
lsmod|grep qxl # check
sudo modprobe qxl bochs_drm # if not mod

To auto load a module at boot, create a file like ‘/etc/modules-load.d/qxl.conf’

1
2
3
# Load qxl at boot
qxl
bochs_drm

This is not enough. Because we just build a base on the server, but do nothing on the Windows guest. Look at this:
before-install.png

Next install spice-guest-tools-latest.exe downloaded from https://www.spice-space.org/download.html.

done installed qxl driver

比较不好的一个bug是虚拟机内部的复制粘贴不好用了，所以对于跨机的复制粘贴，还是采用原始的文件方式来实现。

make sure sshd on host is on
open one term on host and type tty to get device name，say /dev/pts/0

ssh to host using PuTTY from win7 client
echo "copied text" >> /dev/pts/0

You will see “copied text” on the host terminal.

II.B Win10 VM Installation

Because of two bugs of Win7 VM: the first is that copy-and-pasteing for office object always throws an error in VM; the second is that Office2016 on win7 cannot open remote-location file. I change to win10 vm, and I found that’s cool.

II.B.a Launch VM as a desktop program

Go to Virt-manager > Connection Details to find Libvirt URI, here mine is qemu:///system

libvirt-uri

Then open virtual machine, go to View > Details to find the name of the virtual machine, here mine is win10

vm-name

Now we can test the direct-open command:

1
virt-manager --connect qemu:///system --show-domain-console win10

Then create your desktop file under ~/.local/share/applications:

1
2
3
4
5
6
7
8
9
[Desktop Entry]
Version=1.0
Type=Application
Name=win10
Comment=Start Win10 VM
Exec=virt-manager --connect qemu:///system --show-domain-console win10
Icon=your/logo/position/windows.svg
Terminal=false
StartupNotify=false

II.B.b Expand block image size

To add 5G to my win10 raw format volume, first check the state:

1
2
3
4
# virsh list --all
 Id   Name    State
------------------------
 -    win10   shut off

Then go to increase

1
2
3
# cd /var/lib/libvirt/images/
# ls -lh win10vol.img
# qemu-img resize win10vol.img +5G

Then start win10 VM and right click ‘Start Menu’ -> Disk management -> Extend Volumn C:\
addvol.png
ok.
afteradd.png

II.B.c PCI硬件设备直通

Prerequisite: Enable IOMMU(Input/Output Memory Management Unit)
Intel CPU上开启IOMMU需要VT-d的支持。VT-d(Intel Virtualization Technology for Direct I/O)是一个位于CPU、内存和I/O设备之间的硬件设备，它可以拦截I/O设备产生的中断请求和通过DMA方式对虚拟机内存的访问，然后重定向到通过虚拟机监视器（VMM）设定的虚拟机中。VT-d技术支持了虚拟机内部驱动程序直接与硬件设备通信，替代了传统的软件模拟方式，大大提升了虚拟化的I/O性能。

开启方式可分为临时开启和写配置文件开启两种方式：
方式一（临时开启方法）：在计算机启动的grub页面按e进入kernal模式，在启动vmlinuz的参数后面添加 intel_iommu=on 和 iommu=pt
grub-edit.jpg
这种开启方法在重启之后配置又会变成原来的样子。

方式二（写配置文件开启方法）：

1
2
su
vim /etc/default/grub

在 GRUB_CMDLINE_LINUX_DEFAULT 参数中添加 intel_iommu=on iommu=pt

GRUB_CMDLINE_LINUX_DEFAULT=”loglevel=3 intel_iommu=on iommu=pt”

1
grub-mkconfig -o /boot/grub/grub.cfg

1. 配置无线网卡直通
默认的虚拟机使用NAT模式上网，对一些其他上网模式也是支持的。目前我对于桥接模式的配置没有成功，所以这里记录只成功了的网卡直通模式。
在该模式下，虚拟机直接使用无线网卡联网，宿主机可以用有线连接，当时配置该模式只是为了暂时同步一下微信聊天记录（因为必须在同一wifi下）

成功开机后root使用下面命令检查无线网卡的pci地址：

1
2
3
4
5
6
7
shopt -s nullglob
for g in `find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V`; do
    echo "IOMMU Group ${g##*/}:"
    for d in $g/devices/*; do
        echo -e "\t$(lspci -nns ${d##*/})"
    done;
done;

我的是

IOMMU Group 15:
3a:00.0 Network controller [0280]: Qualcomm Atheros QCA6174 802.11ac Wireless Network Adapter [168c:003e] (rev 32)

再检查一下地址是否正确和是否支持iommu

1
2
lspci -n -s 3a:00.0
dmesg | grep -i iommu | grep 3a:00

显示是支持的：

[ 0.452590] pci 0000:3a:00.0: Adding to iommu group 15

将无线网卡与宿主机解绑：

1
echo "0000:3a:00.0" > /sys/bus/pci/devices/0000\:3a\:00.0/driver/unbind

然后，打开虚拟机软件，添加pci device，与虚拟机绑定:
bind-adapter-addr.png

参考：https://www.cnblogs.com/xia-dong/p/11542771.html

2. 配置USB直通
一般情况下，QEMU的USB软件重定向方式可以满足U盘等设备的使用需求，但是在一些需要特定驱动的方式下（比如无线投影、USB转VGA），需要开启USB直通功能。

1
2
3
4
5
[root@arch wsd]# lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 0c45:670c Microdia Integrated Webcam HD
Bus 001 Device 002: ID 0cf3:e300 Qualcomm Atheros Communications QCA61x4 Bluetooth 4.0
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

通过lsusb命令可以看到，USB接口和蓝牙、摄像头都挂在USB Bus 001下。

检查USB设置是否支持RESET功能：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
[root@arch wsd]# for iommu_group in $(find /sys/kernel/iommu_groups/ -maxdepth 1 -mindepth 1 -type d);do echo "IOMMU group $(basename "$iommu_group")"; for device in $(\ls -1 "$iommu_group"/devices/); do if [[ -e "$iommu_group"/devices/"$device"/reset ]]; then echo -n "[RESET]"; fi; echo -n $'\t';lspci -nns "$device"; done; done
IOMMU group 7
[RESET] 00:1c.4 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port #5 [8086:9d14] (rev f1)
IOMMU group 5
        00:16.0 Communication controller [0780]: Intel Corporation Sunrise Point-LP CSME HECI #1 [8086:9d3a] (rev 21)
IOMMU group 13
[RESET] 3c:00.0 Non-Volatile memory controller [0108]: SK hynix PC401 NVMe Solid State Drive 256GB [1c5c:1527]
IOMMU group 3
        00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-LP USB 3.0 xHCI Controller [8086:9d2f] (rev 21)
        00:14.2 Signal processing controller [1180]: Intel Corporation Sunrise Point-LP Thermal subsystem [8086:9d31] (rev 21)
IOMMU group 11
[RESET] 3a:00.0 Network controller [0280]: Qualcomm Atheros QCA6174 802.11ac Wireless Network Adapter [168c:003e] (rev 32)
IOMMU group 1
[RESET] 00:02.0 VGA compatible controller [0300]: Intel Corporation UHD Graphics 620 [8086:5917] (rev 07)
IOMMU group 8
[RESET] 00:1c.5 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port #6 [8086:9d15] (rev f1)
IOMMU group 6
[RESET] 00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port #1 [8086:9d10] (rev f1)
IOMMU group 4
        00:15.0 Signal processing controller [1180]: Intel Corporation Sunrise Point-LP Serial IO I2C Controller #0 [8086:9d60] (rev 21)
        00:15.1 Signal processing controller [1180]: Intel Corporation Sunrise Point-LP Serial IO I2C Controller #1 [8086:9d61] (rev 21)
IOMMU group 12
[RESET] 3b:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS525A PCI Express Card Reader [10ec:525a] (rev 01)
IOMMU group 2
        00:04.0 Signal processing controller [1180]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Thermal Subsystem [8086:1903] (rev 08)
IOMMU group 10
        00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point LPC Controller/eSPI Controller [8086:9d4e] (rev 21)
        00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-LP PMC [8086:9d21] (rev 21)
        00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-LP HD Audio [8086:9d71] (rev 21)
        00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-LP SMBus [8086:9d23] (rev 21)
IOMMU group 0
        00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers [8086:5914] (rev 08)
IOMMU group 9
[RESET] 00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port #9 [8086:9d18] (rev f1)

每行前面带有RESET字样就是可以支持的，可以看到”00:14.0 USB controller”是不支持RESET的。

II.B.d CPU绑定

通过 virsh vcpuinfo win10 可以查看当前使用的CPU情况

1
2
3
4
5
6
7
8
9
10
11
12
wsd@xps13 ~ % sudo virsh vcpuinfo win10                                                                                                                                                                         [1]
VCPU:           0
CPU:            5
State:          running
CPU time:       45.5s
CPU Affinity:   yyyyyyyy

VCPU:           1
CPU:            1
State:          running
CPU time:       42.7s
CPU Affinity:   yyyyyyyy

通过virsh edit win10，在<vcpu placement='static'>2</vcpu>下一行添加：

1
2
3
4
<cputune>
    <vcpupin vcpu='0' cpuset='6'/>
    <vcpupin vcpu='1' cpuset='7'/>
</cputune>

将虚拟机两个vCPU分别绑定在宿主机6和7号CPU上
再次检查CPU使用情况：

1
2
3
4
5
6
7
8
9
10
11
12
[root@xps13 wsd]# virsh vcpuinfo win10
VCPU:           0
CPU:            6
State:          running
CPU time:       165.3s
CPU Affinity:   ------y-

VCPU:           1
CPU:            7
State:          running
CPU time:       105.4s
CPU Affinity:   -------y

References

https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF

#Linux #VM #Win10 #qemu #kvm
Raspberry Pi parallel computing and cluster computing experiment C++ examples of leveldb

Comments

You forgot to set the shortname for Disqus. Please set it in _config.yml.
Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now

×