paky_du

LXC的网络结构和端口映射

LXC，其名称来自Linux软件容器的缩写，一种操作系统层虚拟化技术，为Linux内核容器功能的一个用户空间接口。它将应用软件系统打包成一个软件容器，内含应用软件本身的代码，以及所需要的操作系统核心和库。我们可以把LXC创建出来的容器理解成一个应用层的虚拟，它和宿主机共用一个kernel层，仅仅是应用层程序和运行环境的隔离。所以可想而知她的性能要优于我们常见的虚拟机形式的应用。Docker，大家应该都听说过，其实Docker可以理解成： Docker = LXC + 友好的用户接口。所以技术的本质还是LXC的思想。

LXC的网络接口如下：

paky@paky-VirtualBox14-32:~$ ifconfig

br0 Link encap:Ethernet HWaddrfe:b6:31:c3:77:dd

inet addr:10.0.3.1 Bcast:10.0.3.255 Mask:255.255.255.0

inet6 addr:fe80::9022:5fff:fe5c:84cc/64 Scope:Link

UP BROADCAST RUNNINGMULTICAST MTU:1500 Metric:1

RX packets:319 errors:0dropped:0 overruns:0 frame:0

TX packets:472 errors:0dropped:0 overruns:0 carrier:0

collisions:0txqueuelen:1000

RX bytes:43425 (43.4 KB) TX bytes:60075 (60.0 KB)

eth0 Link encap:Ethernet HWaddr08:00:27:84:7b:88

inet addr:192.168.199.160 Bcast:192.168.199.255 Mask:255.255.255.0

inet6 addr:fe80::a00:27ff:fe84:7b88/64 Scope:Link

UP BROADCAST RUNNINGMULTICAST MTU:1500 Metric:1

RX packets:12616 errors:0dropped:0 overruns:0 frame:0

TX packets:13057 errors:0dropped:0 overruns:0 carrier:0

collisions:0txqueuelen:1000

RX bytes:4161274 (4.1 MB) TX bytes:8688599 (8.6 MB)

eth1 Link encap:Ethernet HWaddr08:00:27:73:aa:62

inet addr:192.168.56.101 Bcast:192.168.56.255 Mask:255.255.255.0

inet6 addr:fe80::a00:27ff:fe73:aa62/64 Scope:Link

UP BROADCAST RUNNINGMULTICAST MTU:1500 Metric:1

RX packets:43 errors:0dropped:0 overruns:0 frame:0

TX packets:106 errors:0dropped:0 overruns:0 carrier:0

collisions:0txqueuelen:1000

RX bytes:9941 (9.9 KB) TX bytes:16404 (16.4 KB)

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0

inet6 addr: ::1/128Scope:Host

UP LOOPBACK RUNNING MTU:65536 Metric:1

RX packets:1046 errors:0dropped:0 overruns:0 frame:0

TX packets:1046 errors:0dropped:0 overruns:0 carrier:0

collisions:0txqueuelen:1

RX bytes:100160 (100.1KB) TX bytes:100160 (100.1 KB)

vethNK7NLR Link encap:Ethernet HWaddrfe:b6:31:c3:77:dd

inet6 addr:fe80::fcb6:31ff:fec3:77dd/64 Scope:Link

UP BROADCAST RUNNINGMULTICAST MTU:1500 Metric:1

RX packets:319 errors:0dropped:0 overruns:0 frame:0

TX packets:416 errors:0dropped:0 overruns:0 carrier:0

collisions:0txqueuelen:1000

RX bytes:47891 (47.8 KB) TX bytes:51255 (51.2 KB)

paky@paky-VirtualBox14-32:~$ route -n

Kernel IP routing table

Destination Gateway    Genmask Flags Metric Ref UseIface

0.0.0.0 192.168.199.1 0.0.0.0 UG 0 0 0 eth0

10.0.3.0 0.0.0.0    255.255.255.0 U 0 0 0 br0

192.168.56.0 0.0.0.0    255.255.255.0 U 1 0    0 eth1

192.168.199.0 0.0.0.0 255.255.255.0 U 1 0    0 eth0

其中br0（缺省情况下， LXC创建的名字是lxcbr0，这个名字可以修改。我就把把它修改为了br0）就是lxc的桥接口，宿主机中的所以容器实例都是和这个网络口挂接在一起的。如果你有一定的网络知识，那就可以把这个br0理解成我们一般的家用的路由器的LAN的网关接口，每个LXC的容器实例如果家里的每个电脑设备， LXC服务给每个LXC实例分配IP地址（LXC使用dnsmasq实现）。每个容器实例通过br0这个网关在宿主机上做NAT的网络转化，这样每个容器实例也就可以访问互联网了。

1. 关于br0的网络的配置文件：/etc/default/lxc-net

pakydu@pakydu-VirtualBox:~$ cat /etc/default/lxc-net
# This file is auto-generated by lxc.postinst if it does not
# exist. Customizations will not be overridden.
# Leave USE_LXC_BRIDGE as "true" if you want to use lxcbr0 for your
# containers. Set to "false" if you'll use virbr0 or another existing
# bridge, or mavlan to your host's NIC.
USE_LXC_BRIDGE="true" //Paky：这个是启用/关闭LXC的Bridge interface。

# If you change the LXC_BRIDGE to something other than lxcbr0, then
# you will also need to update your /etc/lxc/default.conf as well as the
# configuration (/var/lib/lxc//config) for any containers
# already created using the default config to reflect the new bridge
# name.
# If you have the dnsmasq daemon installed, you'll also have to update
# /etc/dnsmasq.d/lxc and restart the system wide dnsmasq daemon.
LXC_BRIDGE="br0" //Paky：这个是LXC的bridge interface name，可以修改
LXC_ADDR="10.0.3.1"   //Paky：这个是LXC的bridge interface IP，相当于LAN的网关
LXC_NETMASK="255.255.255.0"
LXC_NETWORK="10.0.3.0/24"
LXC_DHCP_RANGE="10.0.3.2,10.0.3.101"
LXC_DHCP_MAX="100"
# Uncomment the next line if you'd like to use a conf-file for the lxcbr0
# dnsmasq. For instance, you can use 'dhcp-host=mail1,10.0.3.100' to have
# container 'mail1' always get ip address 10.0.3.100.
#LXC_DHCP_CONFILE=/etc/lxc/dnsmasq.conf

# Uncomment the next line if you want lxcbr0's dnsmasq to resolve the .lxc
# domain. You can then add "server=/lxc/10.0.3.1' (or your actual )
# to /etc/dnsmasq.conf, after which 'container1.lxc' will resolve on your
# host.
#LXC_DOMAIN="lxc"
pakydu@pakydu-VirtualBox:~$

pakydu@pakydu-VirtualBox:~$ brctl show
bridge name bridge id STP enabled interfaces
br0 8000.fe1feebdf13d no vethRVPP5G
vethRW2KPW
pakydu@pakydu-VirtualBox:~$ sudo lxc-ls -f //Paky：我的宿主机上启动了两个容器实例。
NAME STATE IPV4 IPV6 AUTOSTART
---------------------------------------------------
paky-ss RUNNING 10.0.3.63 - NO
paky-test RUNNING 10.0.3.175 - NO

2. 关于宿主机如何实现NAT的配置：iptables的配置

paky@paky-VirtualBox14-32:~$ sudo iptables-save

# Generated by iptables-save v1.4.21 on Wed Mar 14 23:21:412018

*mangle

:PREROUTING ACCEPT [13423:4117489]

:INPUT ACCEPT [13003:4066947]

:FORWARD ACCEPT [420:50542]

:OUTPUT ACCEPT [12717:8284678]

:POSTROUTING ACCEPT [13345:8366195]

-A POSTROUTING -o br0 -p udp -m udp --dport 68 -j CHECKSUM--checksum-fill

COMMIT

# Completed on Wed Mar 14 23:21:41 2018

# Generated by iptables-save v1.4.21 on Wed Mar 14 23:21:412018

*nat

:PREROUTING ACCEPT [23:5355]

:INPUT ACCEPT [23:5355]

:OUTPUT ACCEPT [30:2438]

:POSTROUTING ACCEPT [31:2490]

-A POSTROUTING -s 10.0.3.0/24 ! -d 10.0.3.0/24 -j MASQUERADE

COMMIT

# Completed on Wed Mar 14 23:21:41 2018

# Generated by iptables-save v1.4.21 on Wed Mar 14 23:21:412018

*filter

:INPUT ACCEPT [12985:4065493]

:FORWARD ACCEPT [0:0]

:OUTPUT ACCEPT [12719:8284778]

-A INPUT -i br0 -p tcp -m tcp --dport 53 -j ACCEPT

-A INPUT -i br0 -p udp -m udp --dport 53 -j ACCEPT

-A INPUT -i br0 -p tcp -m tcp --dport 67 -j ACCEPT

-A INPUT -i br0 -p udp -m udp --dport 67 -j ACCEPT

-A FORWARD -o br0 -j ACCEPT

-A FORWARD -i br0 -j ACCEPT

COMMIT

# Completed on Wed Mar 14 23:21:41 2018

paky@paky-VirtualBox14-32:~$

接下来进入另外一个主题，如果从外边的网络访问宿主机中的容器实例里面的服务，如容器10.0.3.63的http和ssh。

既然LXC用的是iptables来实现网络的NAT，那我们也就可以用iptables的规则来实现端口映射，这样就可以实现从外部访问容器内部的服务了。如下图所示，下图中的192.168.199.xx是个私有的IP，我们假设他是个共有IP来说明我们的网络拓扑图。

下图是一个测试截图：

最后再一docker如何实现端口映射的作为结尾，其实它就是对用户更好的封装，更容易使用docker。

paky 3297 0.0 0.3 419960 9340 ? Sl 20:52 0:00/usr/lib/gvfs/gvfsd-fuse /run/user/1000/gvfs -f -o big_writes

paky 3303 0.0 0.1 178532 4764 ? Sl 20:52 0:00/usr/lib/dconf/dconf-service

paky 3309 0.0 0.2 353464 8008 ? Sl 20:52 0:00/usr/lib/at-spi2-core/at-spi-bus-launcher

paky 3318 0.0 0.1 42768 3528 ? S 20:52 0:00 /usr/bin/dbus-daemon --config-file=/etc/at-spi2/accessibility.conf--nofork --print-address 3

paky 3320 0.0 0.1 206976 5356 ? Sl 20:52 0:00/usr/lib/at-spi2-core/at-spi2-registryd --use-gnome-session

paky 3323 0.1 1.8 60979658216 ? Sl 20:52 0:00 /usr/bin/python3/usr/bin/blueman-applet

paky 3333 0.1 1.1 64849634456 ? Sl 20:52 0:00 nm-applet

paky 3349 0.0 0.4 31420413004 ? Ssl 20:52 0:00xfce4-power-manager

paky 3350 0.0 0.8 50544826228 ? Sl 20:52 0:00 update-notifier

paky 3351 0.0 0.6 33954421492 ? Sl 20:52 0:00/usr/lib/policykit-1-gnome/polkit-gnome-authentication-agent-1

paky 3396 0.0 0.5 30413617520 ? SNl 20:52 0:00/usr/lib/x86_64-linux-gnu/tumbler-1/tumblerd

root 3397 0.0 0.3 348896 9960 ? Ssl 20:52 0:00/usr/lib/upower/upowerd

paky 3421 0.0 0.8 33994425604 ? Sl 20:52 0:00/usr/lib/x86_64-linux-gnu/xfce4/panel/wrapper-1.0/usr/lib/x86_64-linux-gnu/xfce4/panel/plugins/libwhiskermenu.so 1 2306871

paky 3431 0.0 0.4 16047614060 ? S 20:52 0:00/usr/lib/x86_64-linux-gnu/xfce4/panel/wrapper-1.0 /usr/lib/x86_64-linux-gnu/xfce4/panel/plugins/libsystray.so4 23068717 sy

paky 3432 0.0 0.6 32753619468 ? Sl 20:52 0:00/usr/lib/x86_64-linux-gnu/xfce4/panel/wrapper-1.0/usr/lib/x86_64-linux-gnu/xfce4/panel/plugins/libxfce4powermanager.so 5 2

paky 3433 0.0 0.9 45044829600 ? Sl 20:52 0:00/usr/lib/x86_64-linux-gnu/xfce4/panel/wrapper-2.0/usr/lib/x86_64-linux-gnu/xfce4/panel/plugins/libindicator-plugin.so 6 23

paky 3444 0.0 0.3 298000 9764 ? Sl 20:52 0:00 /usr/lib/gvfs/gvfs-udisks2-volume-monitor

root 3448 0.0 0.3 382276 9728 ? Ssl 20:52 0:00/usr/lib/udisks2/udisksd --no-debug

paky 3477 0.0 0.1 273472 5260 ? Sl 20:52 0:00/usr/lib/gvfs/gvfs-gphoto2-volume-monitor

paky 3480 0.0 0.1 47700 4480 ? S 20:52 0:00 upstart --user --startup-event indicator-services-start

paky 3482 0.0 0.3 371708 9560 ? Ssl 20:52 0:00/usr/lib/x86_64-linux-gnu/indicator-messages/indicator-messages-service

paky 3483 0.0 0.4 75990813208 ? Ssl 20:52 0:00/usr/lib/x86_64-linux-gnu/indicator-sound/indicator-sound-service

paky 3484 0.0 0.4 40314412872 ? Ssl 20:52 0:00/usr/lib/x86_64-linux-gnu/indicator-application/indicator-application-service

paky 3503 0.0 0.2 405360 9176 ? Sl 20:52 0:00/usr/lib/gvfs/gvfs-afc-volume-monitor

paky 3510 0.0 0.1 261276 4588 ? Sl 20:52 0:00/usr/lib/gvfs/gvfs-mtp-volume-monitor

paky 3518 0.0 0.1 259284 4292 ? Sl 20:52 0:00/usr/lib/gvfs/gvfs-goa-volume-monitor

paky 3527 0.0 0.2 365392 9012 ? Sl 20:52 0:00/usr/lib/gvfs/gvfsd-trash --spawner :1.15 /org/gtk/gvfs/exec_spaw/0

paky 3535 0.0 0.1 187692 4556 ? Sl 20:52 0:00/usr/lib/gvfs/gvfsd-metadata

root 3649 0.0 0.1 16128 3792 ? S 20:52 0:00 /sbin/dhclient -d -q -sf /usr/lib/NetworkManager/nm-dhcp-helper -pf/var/run/dhclient-enp0s3.pid -lf /var/lib/NetworkManage

nobody 3687 0.0 0.1 54616 4368 ? S 20:52 0:00/usr/sbin/dnsmasq --no-resolv --keep-in-foreground --no-hosts --bind-interfaces--pid-file=/var/run/NetworkManager/dnsmasq.

paky 4010 0.2 0.8 40040425940 ? Sl 20:52 0:00/usr/bin/xfce4-terminal

paky 4016 0.0 0.0 14872 1892 ? S 20:52 0:00 gnome-pty-helper

paky 4017 0.0 0.1 24372 5508 pts/6 Ss 20:52 0:00 bash

paky 4115 0.0 0.3 15030410016 ? S 20:52 0:00/usr/lib/bluetooth/obexd

root 4394 0.0 0.1 117096 3580 ? Sl 20:53 0:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port10022 -container-ip 172.17.0.2 -container-port 22

root 4400 0.0 0.1 207624 4976 ? Sl 20:53 0:00containerd-shim a1157cef062f3075c787dedb98e28370e900d232e3adfc086bd7eb122e3dde1f/var/run/docker/libcontainerd/a1157cef062f

root 4418 0.0 0.0 18020 2592 ? Ss 20:53 0:00/bin/bash /run.sh

root 4460 0.0 0.1 65504 5160 ? S 20:53 0:00 /usr/sbin/sshd -D

paky 4485 0.0 0.1 39112 3556 pts/6 R+ 20:55 0:00 ps aux

paky@paky-VirtualBox:~$ iptables

iptables iptables-apply iptables-restore iptables-save iptables-xml

paky@paky-VirtualBox:~$ iptables

iptables iptables-apply iptables-restore iptables-save iptables-xml

paky@paky-VirtualBox:~$ iptables-save

paky@paky-VirtualBox:~$ iptables -t nat -nL

iptables v1.6.0: can't initialize iptables table `nat':Permission denied (you must be root)

Perhaps iptables or your kernel needs to be upgraded.

paky@paky-VirtualBox:~$ sudo iptables -t nat -nL

Chain PREROUTING (policy ACCEPT)