目录
- 1. 虚拟化介绍
- 2. kvm介绍
- 3. kvm部署
- 3.1kvm安装
- 3.2 kvm web管理界面安装
- 3.3 kvm web界面管理
- 3.3.1 kvm连接管理
- 3.3.2 kvm存储管理
- 3.3.3 kvm网络管理
- 3.3.4 实例管理
- 4.故障案例
- 4.1 案例1
- 4.2 案例2
1. 虚拟化介绍
虚拟化是云计算的基础。简单的说，虚拟化使得在一台物理的服务器上可以跑多台虚拟机，虚拟机共享物理机的 CPU、内存、IO 硬件资源，但逻辑上虚拟机之间是相互隔离的。
物理机我们一般称为宿主机（Host），宿主机上面的虚拟机称为客户机（Guest）。
那么 Host 是如何将自己的硬件资源虚拟化，并提供给 Guest 使用的呢？
这个主要是通过一个叫做 Hypervisor 的程序实现的。
根据 Hypervisor 的实现方式和所处的位置，虚拟化又分为两种：
- 全虚拟化
- 半虚拟化
全虚拟化：
半虚拟化：
物理机上首先安装常规的操作系统，比如 Redhat、Ubuntu 和 Windows。Hypervisor 作为 OS 上的一个程序模块运行，并对管理虚拟机进行管理。KVM、VirtualBox 和 VMWare Workstation 都属于这个类型
理论上讲：
全虚拟化一般对硬件虚拟化功能进行了特别优化，性能上比半虚拟化要高；
半虚拟化因为基于普通的操作系统，会比较灵活，比如支持虚拟机嵌套。嵌套意味着可以在KVM虚拟机中再运行KVM。
2. kvm介绍
kVM 全称是 Kernel-Based Virtual Machine。也就是说 KVM 是基于 Linux 内核实现的。
KVM有一个内核模块叫 kvm.ko，只用于管理虚拟 CPU 和内存。
那 IO 的虚拟化，比如存储和网络设备则是由 Linux 内核与Qemu来实现。
作为一个 Hypervisor，KVM 本身只关注虚拟机调度和内存管理这两个方面。IO 外设的任务交给 Linux 内核和 Qemu。
大家在网上看 KVM 相关文章的时候肯定经常会看到 Libvirt 这个东西。
Libvirt 就是 KVM 的管理工具。
其实，Libvirt 除了能管理 KVM 这种 Hypervisor，还能管理 Xen，VirtualBox 等。
- Libvirt 包含 3 个东西：后台 daemon 程序 libvirtd、API 库和命令行工具 virsh
- libvirtd是服务程序，接收和处理 API 请求；
- API 库使得其他人可以开发基于 Libvirt 的高级工具，比如 virt-manager，这是个图形化的 KVM 管理工具；
- virsh 是我们经常要用的 KVM 命令行工具
3. kvm部署
环境说明：
| 环境类型 | IP |
|---|---|
| RHEL7 | 192.168.159.144 |
3.1kvm安装
部署前请确保你的CPU虚拟化功能已开启。分为两种情况：
- 虚拟机要关机设置CPU虚拟化
- 物理机要在BIOS里开启CPU虚拟化
//关闭防火墙和SELINUX
[root@localhost ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: inactive (dead)
[root@localhost ~]# getenforce
Disabled
//配置网络源
[root@localhost yum.repos.d]# curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
[root@localhost yum.repos.d]# sed -i 's/\$releasever/7/g' /etc/yum.repos.d/CentOS7-Base.repo
[root@localhost yum.repos.d]# wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
//安装一些常用的工具
[root@localhost ~]# yum -y install vim wget net-tools unzip zip gcc gcc-c++
//检查CPU是否支持KVM；如果结果中有vmx（Intel）或svm(AMD)字样，就说明CPU的支持的
[root@localhost ~]# egrep -o 'vmx|svm' /proc/cpuinfo
vmx
vmx
vmx
vmx
//kvm安装
[root@localhost ~]# yum -y install qemu-kvm qemu-kvm-tools qemu-img virt-manager libvirt libvirt-python libvirt-client virt-install virt-viewer bridge-utils libguestfs-tools
//因为虚拟机中网络，我们一般都是和公司的其他服务器是同一个网段，所以我们需要把 \
KVM服务器的网卡配置成桥接模式。这样的话KVM的虚拟机就可以通过该桥接网卡和公司内部 \
其他服务器处于同一网段
//此处我的网卡是eth0，所以用br0来桥接eth0网卡
[root@localhost network-scripts]# ls
ifcfg-br0 ifdown-ipv6 ifdown-tunnel ifup-isdn ifup-TeamPort
ifcfg-eth0 ifdown-isdn ifup ifup-plip ifup-tunnel
ifcfg-lo ifdown-post ifup-aliases ifup-plusb ifup-wireless
ifdown ifdown-ppp ifup-bnep ifup-post init.ipv6-global
ifdown-bnep ifdown-routes ifup-eth ifup-ppp network-functions
ifdown-eth ifdown-sit ifup-ib ifup-routes network-functions-ipv6
ifdown-ib ifdown-Team ifup-ippp ifup-sit
ifdown-ippp ifdown-TeamPort ifup-ipv6 ifup-Team
[root@localhost network-scripts]# cat ifcfg-br0
TYPE=Bridge
NM_CONTROLLED=no
BOOTPROTO=static
NAME=br0
DEVICE=br0
ONBOOT=yes
IPADDR=192.168.159.144
NETMASK=255.255.255.0
GATEWAY=192.168.159.2
DNS1=114.114.114.114
[root@localhost network-scripts]# cat ifcfg-eth0
TYPE=Ethernet
BOOTPROTO=static
NAME=eth0
DEVICE=eth0
ONBOOT=yes
BRIDGE=br0
NM_CONTROLLED=no
[root@localhost network-scripts]# systemctl restart network
[root@localhost network-scripts]# ip a
1: lo: mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast master br0 state UP qlen 1000
link/ether 00:50:56:23:3e:ef brd ff:ff:ff:ff:ff:ff
inet6 fe80::250:56ff:fe23:3eef/64 scope link
valid_lft forever preferred_lft forever
3: br0: mtu 1500 qdisc noqueue state UP
link/ether 00:50:56:23:3e:ef brd ff:ff:ff:ff:ff:ff
inet 192.168.159.144/24 brd 192.168.159.255 scope global br0
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe23:3eef/64 scope link
valid_lft forever preferred_lft forever
//启动服务
[root@localhost ~]# systemctl enable --now libvirtd
[root@localhost ~]# ss -anlt
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 5 192.168.122.1:53 *:*
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
//验证安装结果
[root@localhost ~]# lsmod|grep kvm
kvm_intel 162153 0
kvm 525259 1 kvm_intel
irqbypass 13503 1 kvm
//测试并验证安装结果
[root@localhost ~]# virsh -c qemu:///system list
Id Name State
----------------------------------------------------
[root@localhost ~]# virsh --version
4.5.0
[root@localhost ~]# virt-install --version
1.5.0
[root@localhost ~]# ln -s /usr/libexec/qemu-kvm /usr/bin/qemu-kvm
[root@localhost ~]# ll /usr/bin/qemu-kvm
lrwxrwxrwx 1 root root 21 Aug 4 02:43 /usr/bin/qemu-kvm -> /usr/libexec/qemu-kvm
[root@localhost ~]# lsmod |grep kvm
kvm_intel 162153 0
kvm 525259 1 kvm_intel
irqbypass 13503 1 kvm
//查看网桥信息
[root@localhost ~]# brctl show
bridge name bridge id STP enabled interfaces
br0 8000.005056233eef no eth0
virbr0 8000.000000000000 yes virbr0-nic
3.2 kvm web管理界面安装
//安装依赖包
[root@localhost ~]# yum -y install git python-pip libvirt-python libxml2-python python-websockify supervisor nginx python-devel
//升级pip
[root@localhost ~]# pip install --upgrade pip
Collecting pip
Downloading https://files.pythonhosted.org/packages/36/74/38c2410d688ac7b48afa07d413674afc1f903c1c1f854de51dc8eb2367a5/pip-20.2-py2.py3-none-any.whl (1.5MB)
100% |████████████████████████████████| 1.5MB 6.5kB/s
Installing collected packages: pip
Found existing installation: pip 8.1.2
Uninstalling pip-8.1.2:
Successfully uninstalled pip-8.1.2
Successfully installed pip-20.2
//从github上下载webvirtmgr代码
[root@localhost ~]# cd /usr/local/src/
[root@localhost src]# git clone git://github.com/retspen/webvirtmgr.git
Cloning into 'webvirtmgr'...
remote: Enumerating objects: 5614, done.
remote: Total 5614 (delta 0), reused 0 (delta 0), pack-reused 5614
Receiving objects: 100% (5614/5614), 2.98 MiB | 14.00 KiB/s, done.
Resolving deltas: 100% (3602/3602), done.
//安装webvirtmgr
[root@localhost src]# cd webvirtmgr/
[root@localhost webvirtmgr]# pip install -r requirements.txt
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting django==1.5.5
Downloading Django-1.5.5.tar.gz (8.1 MB)
|████████████████████████████████| 8.1 MB 133 kB/s
Collecting gunicorn==19.5.0
Downloading gunicorn-19.5.0-py2.py3-none-any.whl (113 kB)
|████████████████████████████████| 113 kB 127 kB/s
Collecting lockfile>=0.9
Downloading lockfile-0.12.2-py2.py3-none-any.whl (13 kB)
Using legacy 'setup.py install' for django, since package 'wheel' is not installed.
Installing collected packages: django, gunicorn, lockfile
Running setup.py install for django ... done
Successfully installed django-1.5.5 gunicorn-19.5.0 lockfile-0.12.2
//检查sqlite3是否安装
[root@localhost webvirtmgr]# python
Python 2.7.5 (default, Apr 2 2020, 13:16:51)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> exit()
//初始化帐号信息
[root@localhost webvirtmgr]# python manage.py syncdb
WARNING:root:No local_settings file found.
Creating tables ...
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_groups
Creating table auth_user_user_permissions
Creating table auth_user
Creating table django_content_type
Creating table django_session
Creating table django_site
Creating table servers_compute
Creating table instance_instance
Creating table create_flavor
You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): yes //问你是否创建超级管理员帐号
Username (leave blank to use 'root'): admin //指定超级管理员帐号用户名，默认留空为root
Email address: [email protected] //设置超级管理员邮箱
Password: //设置超级管理员密码
Password (again): //再次输入超级管理员密码
Superuser created successfully.
Installing custom SQL ...
Installing indexes ...
Installed 6 object(s) from 1 fixture(s)
//拷贝web网页至指定目录
[root@localhost webvirtmgr]# mkdir /var/www
[root@localhost webvirtmgr]# cp -r /usr/local/src/webvirtmgr /var/www/
[root@localhost webvirtmgr]# chown -R nginx.nginx /var/www/webvirtmgr/
//生成密钥
[root@localhost ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
70:f1:5e:67:74:65:b5:1b:17:81:18:01:ca:0d:ab:bb [email protected]
The key's randomart image is:
+--[ RSA 2048]----+
| ....o+ ooO|
| . =o . o oo|
| .+... . oo.|
| .o . . o +|
| . S . . |
| . |
| . |
| . |
| E |
+-----------------+
//由于这里webvirtmgr和kvm服务部署在同一台机器，所以这里本地信任。如果kvm部署在其他机器，那么这个是它的ip
[root@localhost ~]# ssh-copy-id 192.168.159.144
The authenticity of host '192.168.159.144 (192.168.159.144)' can't be established.
ECDSA key fingerprint is 37:f2:8a:06:d0:c7:ee:24:97:46:50:f3:bf:87:96:06.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
[email protected]'s password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh '192.168.159.144'"
and check to make sure that only the key(s) you wanted were added.
//配置端口转发
[root@localhost ~]# ssh 192.168.159.144 -L localhost:8000:localhost:8000 -L localhost:6080:localhost:60
Last login: Tue Aug 4 03:07:28 2020 from 192.168.159.1
[root@localhost ~]# ss -anlt
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:111 *:*
LISTEN 0 5 192.168.122.1:53 *:*
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 127.0.0.1:6010 *:*
LISTEN 0 128 127.0.0.1:6080 *:*
LISTEN 0 128 127.0.0.1:8000 *:*
LISTEN 0 128 :::111 :::*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
LISTEN 0 128 ::1:6010 :::*
LISTEN 0 128 ::1:6080 :::*
LISTEN 0 128 ::1:8000 :::*
//配置nginx
[root@localhost ~]# vim /etc/nginx/nginx.conf
[root@localhost ~]# cat /etc/nginx/nginx.conf
# For more information on configuration, see:
# * Official English Documentation: http://nginx.org/en/docs/
# * Official Russian Documentation: http://nginx.org/ru/docs/
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
server {
listen 80;
server_name localhost;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
root html;
index index.html index.html
}
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
# Settings for a TLS enabled server.
#
# server {
# listen 443 ssl http2 default_server;
# listen [::]:443 ssl http2 default_server;
# server_name _;
# root /usr/share/nginx/html;
#
# ssl_certificate "/etc/pki/nginx/server.crt";
# ssl_certificate_key "/etc/pki/nginx/private/server.key";
# ssl_session_cache shared:SSL:1m;
# ssl_session_timeout 10m;
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_prefer_server_ciphers on;
#
# # Load configuration files for the default server block.
# include /etc/nginx/default.d/*.conf;
#
# location / {
# }
#
# error_page 404 /404.html;
# location = /40x.html {
# }
#
# error_page 500 502 503 504 /50x.html;
# location = /50x.html {
# }
# }
}
[root@localhost ~]# vim /etc/nginx/conf.d/webvirtmgr.conf
[root@localhost ~]# cat /etc/nginx/conf.d/webvirtmgr.conf
//添加下面内容
[include]
files = supervisord.d/*.ini
[program:webvirtmgr]
command=/usr/bin/python2 /var/www/webvirtmgr/manage.py run_gunicorn -c /var/www/webvirtmgr/conf/gunicorn.conf.py
directory=/var/www/webvirtmgr
autostart=true
autorestart=true
logfile=/var/log/supervisor/webvirtmgr.log
log_stderr=true
user=nginx
[program:webvirtmgr-console]
command=/usr/bin/python2 /var/www/webvirtmgr/console/webvirtmgr-console
directory=/var/www/webvirtmgr
autostart=true
autorestart=true
stdout_logfile=/var/log/supervisor/webvirtmgr-console.log
redirect_stderr=true
user=nginx
//启动supervisor并设置开机自启
[root@localhost ~]# systemctl start supervisord
[root@localhost ~]# systemctl enable supervisord
Created symlink from /etc/systemd/system/multi-user.target.wants/supervisord.service to /usr/lib/systemd/system/supervisord.service.
[root@localhost ~]# ss -anlt
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:111 *:*
LISTEN 0 128 *:80 *:*
LISTEN 0 5 192.168.122.1:53 *:*
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 127.0.0.1:6010 *:*
LISTEN 0 128 127.0.0.1:6080 *:*
LISTEN 0 128 127.0.0.1:8000 *:*
LISTEN 0 128 :::111 :::*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
LISTEN 0 128 ::1:6010 :::*
LISTEN 0 128 ::1:6080 :::*
LISTEN 0 128 ::1:8000 :::*
//配置nginx用户
[root@localhost ~]# su - nginx -s /bin/bash
-bash-4.2$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/var/lib/nginx/.ssh/id_rsa):
Created directory '/var/lib/nginx/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /var/lib/nginx/.ssh/id_rsa.
Your public key has been saved in /var/lib/nginx/.ssh/id_rsa.pub.
The key fingerprint is:
9d:a1:44:f5:a1:5a:75:72:dc:7b:eb:fe:eb:9a:c3:49 [email protected]
The key's randomart image is:
+--[ RSA 2048]----+
| ... +.o. |
| . + =. .|
| . + . .|
| . = o ..|
| S o o|
| E . |
| o o |
| +.. |
| o=++|
+-----------------+
-bash-4.2$ touch ~/.ssh/config && echo -e "StrictHostKeyChecking=no\nUserKnownHostsFile=/dev/null" >> ~/.ssh/config
-bash-4.2$ chmod 0600 ~/.ssh/config
-bash-4.2$ ssh-copy-id [email protected]
/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
Warning: Permanently added '192.168.159.144' (ECDSA) to the list of known hosts.
[email protected]'s password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added.
-bash-4.2$ exit
logout
[root@localhost ~]# vim /etc/polkit-1/localauthority/50-local.d/50-libvirt-remote-access.pkla
[root@localhost ~]# cat /etc/polkit-1/localauthority/50-local.d/50-libvirt-remote-access.pkla
[Remote libvirt SSH access]
Identity=unix-user:root
Action=org.libvirt.unix.manage
ResultAny=yes
ResultInactive=yes
ResultActive=yes
[root@localhost ~]# chown -R root.root /etc/polkit-1/localauthority/50-local.d/50-libvirt-remote-access.pkla
[root@localhost ~]# systemctl restart nginx
[root@localhost ~]# systemctl restart libvirtd
3.3 kvm web界面管理
通过ip地址在浏览器上访问kvm，例如我这里就是：http://192.168.159.144
3.3.1 kvm连接管理
创建SSH连接：
3.3.2 kvm存储管理
创建存储：
通过远程连接软件上传ISO镜像文件至存储目录/var/lib/libvirt/images/
[root@localhost ~]# cd /var/lib/libvirt/images
[root@localhost images]# ls
CentOS-7-x86_64-DVD-1511.iso
在 web 界面查看ISO镜像是否存在
创建系统安装镜像
添加成功如下图所示
3.3.3 kvm网络管理
添加桥接网络
3.3.4 实例管理
实例(虚拟机)创建
虚拟机插入光盘
设置在 web 上访问虚拟机的密码
虚拟机安装
4.故障案例
4.1 案例1
第一次通过web访问kvm时可能会一直访问不了，一直转圈，而命令行界面一直报错(too many open files)
此时需要对nginx进行配置
[root@localhost ~]# vim /etc/nginx/nginx.conf
[root@localhost ~]# cat /etc/nginx/nginx.conf
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
worker_rlimit_nofile 655350; //添加此行
然后对系统参数进行设置
[root@localhost ~]# vim /etc/security/limits.conf
[root@localhost ~]# cat /etc/security/limits.conf
//在最后添加下面内容
* soft nofile 655350
* hard nofile 655350
[root@localhost ~]# systemctl restart nginx
4.2 案例2
web界面配置完成后可能会出现以下错误界面
[root@localhost ~]# yum -y install novnc
[root@localhost ~]# ll /etc/rc.local
lrwxrwxrwx 1 root root 13 Aug 4 02:30 /etc/rc.local -> rc.d/rc.local
[root@localhost ~]# ll /etc/rc.d/rc.local
-rw-r--r-- 1 root root 473 Jun 30 23:11 /etc/rc.d/rc.local
[root@localhost ~]# chmod +x /etc/rc.d/rc.local
[root@localhost ~]# ll /etc/rc.d/rc.local
-rwxr-xr-x 1 root root 473 Jun 30 23:11 /etc/rc.d/rc.local
[root@localhost ~]# vim /etc/rc.d/rc.local
[root@localhost ~]# cat /etc/rc.d/rc.local
#!/bin/bash
# THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES
#
# It is highly advisable to create own systemd services or udev rules
# to run scripts during boot instead of using this file.
#
# In contrast to previous versions due to parallel execution during boot
# this script will NOT be run after all other services.
#
# Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure
# that this script will be executed during boot.
touch /var/lock/subsys/local
nohup novnc_server 172.16.12.128:5920 &
[root@localhost ~]# . /etc/rc.d/rc.local
[root@localhost ~]# nohup: ignoring input and appending output to ‘nohup.out’
做完以上操作后再次访问即可正常访问
安装完成了
