docker高级应用之智能添加与修改防火墙规则
资料简介： 如果你有以下痛苦：
1、使用默认docker0桥接方式；
2、修改防火墙规则的话，使用手动修改配置；
3、并且修改时候还得计算来源端口，防止重复端口使用户登陆错误容器；
4、并当容器意外重启，内网ip变化后还得修改规则
那么你可以看看本文了，对你这些痛处都有解决方法。
目前docker容器设置访问规则的话，就2个方法
1、在docker容器创建的时候，使用-p来设置
2、在容器运行中，获取容器的ip，然后在宿主机的iptables力通过nat链做dnat设置
我之前一直使用第2个方法，但随着我docker项目的增加（目前我这里研发使用docker的容器做测试机），防火墙的访问规则设置起来十分麻烦，并且之前规划没有弄好，容器的网络还是默认的docker0桥接方式，这样容器一挂或者异常问题、docker daemon重启，都会导致容器的ip变更，变更后就得修改防火墙策略，十分的麻烦。
为了解决这个问题，我开发了2个程序，1个是持久化固定容器ip，另外一个是智能防火墙，下面是关于智能防火墙功能的介绍。
一、介绍
1、编写语言
python
2、运行环境
容器需要使用我之前写的持久化固定ip方式来创建
需要额外安装的python模块
etcd
docker
nmap
3、基本宿主机防火墙（包含filter链与nat链）
默认在/root/firewall里有个基础的宿主机防火墙，里面包含filter链与nat链，我的防火墙程序先获取这个文件，然后在从etcd里获取各容器的防火墙结合后是新的规则，如下面是我的
4、代码
二、运行
1、使用帮助
-l是展示当前容器的所有防火墙规则，后面不加值
-a是进行增加容器防火墙规则，后面不加值
-r是删除容器规则，后面是防火墙的id值
-m是防火墙规则的模式，有internal（内部模式）与external（外部模式），如果你使用跟宿主机一样出口ip的话，选择internal，如果使用独立的外网ip，就选择external
-s是来源ip，后面是ip/netmask模式
-sp是来源访问端口，后面需要输入允许外边访问进来的端口
-d是指定独立的外网ip，后面需要输入ip值
-dp是指定访问容器的端口，后面是属于端口值
-pm是指定来源端口的模式，有dynamic与manual模式，如果选择dynamic就不需要你指定来源端口值，会自动生成一个值，使用manual的话，就需要你手动输入一个来源端口值
-e是设置好防火墙后，应用此防火墙策略，也就是在iptables里生效，后面不加值
-ap是在本机增加独立的外网ip，后面是ip/netmask模式
-rp是删除本机独立外网ip，后面是ip模式
2、查看test1的规则
目前我本机有test1-3这3个容器
现在使用智能防火墙查看test1 的防火墙
3、添加规则
先给test1添加内部模式规则-m internal，这样能使用docker宿主机的外网ip+port访问
其中test1的容器ip自动查看，不需要收到查找与手动输入
说明
Mode是运行的模式
Container_name是对应容器名
Source_port是来源端口
Port_mode是指端口模式为动态获取
Local_port是目的端口
Source_ip是来源ip
Id是指防火墙规则的id，后面可以通过指定id来删除规则
Container_ip是容器的ip
以上数据均是在使用持久化故障ip脚本生成容器的时候，程序自动把数据写入到etcd里，然后只能防火墙也通过etcd获取数据
在给test1添加一个使用手动模式设置来源端口
4、防火墙规则生效
先查看当前宿主机防火墙
现在使用-e参数来生效
在去配置文件里看看
5、删除防火墙规则
当前test1 的规则
6、添加额外的外网ip
使用-ap来添加
在通过修改防火墙来设置外部模式策略
然后生效
7、测试批量生效防火墙策略
目前仅有test1有防火墙测试了，现在给test2也设置
8、删除额外的外网ip
使用rp参数
目前docker方面的文章，已经完成了：
1、安装docker
2、动态扩容docker容器的空间
3、动态绑定volume
4、docker多主机网络互联
5、docker持久化固定ip
6、docker智能添加与修改防火墙
1、使用默认docker0桥接方式；
2、修改防火墙规则的话，使用手动修改配置；
3、并且修改时候还得计算来源端口，防止重复端口使用户登陆错误容器；
4、并当容器意外重启，内网ip变化后还得修改规则
那么你可以看看本文了，对你这些痛处都有解决方法。
目前docker容器设置访问规则的话，就2个方法
1、在docker容器创建的时候，使用-p来设置
2、在容器运行中，获取容器的ip，然后在宿主机的iptables力通过nat链做dnat设置
我之前一直使用第2个方法，但随着我docker项目的增加（目前我这里研发使用docker的容器做测试机），防火墙的访问规则设置起来十分麻烦，并且之前规划没有弄好，容器的网络还是默认的docker0桥接方式，这样容器一挂或者异常问题、docker daemon重启，都会导致容器的ip变更，变更后就得修改防火墙策略，十分的麻烦。
为了解决这个问题，我开发了2个程序，1个是持久化固定容器ip，另外一个是智能防火墙，下面是关于智能防火墙功能的介绍。
一、介绍
1、编写语言
python
2、运行环境
容器需要使用我之前写的持久化固定ip方式来创建
需要额外安装的python模块
etcd
docker
nmap
3、基本宿主机防火墙（包含filter链与nat链）
默认在/root/firewall里有个基础的宿主机防火墙，里面包含filter链与nat链，我的防火墙程序先获取这个文件，然后在从etcd里获取各容器的防火墙结合后是新的规则，如下面是我的
[root@docker-test3 firewall]# cat /root/firewall/iptables_base.txt *filter :INPUT DROP [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [1:83] -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -i em1 -j ACCEPT -A INPUT -i ovs1 -j ACCEPT #forllow is room network -A INPUT -s 117.121.x.0/24 -p tcp -m multiport --dports 50020 -j ACCEPT -A INPUT -p tcp -j REJECT --reject-with tcp-reset -A FORWARD -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK RST -m limit --limit 1/sec -j ACCEPT COMMIT # Completed on Fri Dec 6 10:59:13 2013 *nat :PREROUTING ACCEPT [2:269] :POSTROUTING ACCEPT [1739:127286] :OUTPUT ACCEPT [1739:127286] :DOCKER - [0:0] -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER -A POSTROUTING -s 172.16.0.0/16 ! -d 172.16.0.0/16 -j MASQUERADE -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER COMMIT其中50020是ssh端口,117.121.x.0/24是允许的网段，x的意思保密，不让你们看我的网络。
4、代码
#!/usr/bin/env python #-*- coding: utf-8 -*- #author:Deng Lei #email: [email protected] import os import sys import argparse import etcd import time import socket, struct, fcntl from docker import Client import subprocess import shutil import nmap def get_local_ip(iface = 'em1'): sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sockfd = sock.fileno() SIOCGIFADDR = 0x8915 ifreq = struct.pack('16sH14s', iface, socket.AF_INET, 'x00'*14) try: res = fcntl.ioctl(sockfd, SIOCGIFADDR, ifreq) except: return None ip = struct.unpack('16sH2x4s8x', res)[2] return socket.inet_ntoa(ip) def docker_container_all(): docker_container=docker_client.containers(all=True) container_name=[] container_stop_name=[] for i in docker_container: container_name.append(i['Names']) for b in container_name: for c in b: container_stop_name.append(c) return container_stop_name def docker_container_run(): docker_container=docker_client.containers() container_name=[] container_stop_name=[] for i in docker_container: container_name.append(i['Names']) for b in container_name: for c in b: container_stop_name.append(c[1::]) return container_stop_name if __name__ == "__main__": #follow is help info p = argparse.ArgumentParser(description='It is userful tool to modify docker container firewall') p.add_argument("container_name",help="list local docker container name") p.add_argument("-l","--list",help="show container firewall rules",action="store_true") p.add_argument("-a","--add",help="add container firewall rules",action="store_true") p.add_argument("-r","--rm",help="rm container firewall rules") p.add_argument("-m","--mode",choices=["internal","external"],help="set container firewall mode") p.add_argument("-s","--source",help="source ip view container firewall rules") p.add_argument("-sp","--sport",help="source port view container firewall rules") p.add_argument("-d","--dest",help="destination ip container firewall rules") p.add_argument("-dp","--dport",help="destination port view container firewall rules") p.add_argument("-pm","--portmode",choices=["dynamic","manual"],help="set container port mode") p.add_argument("-e","--effect",help="effect container firewall rules",action="store_true") p.add_argument("-ap","--addip",help="add external ip to container") p.add_argument("-rp","--rmip",help="rm external ip to container") args = p.parse_args() local_ip=get_local_ip('ovs1') docker_etcd_key='/app/docker/' etcd_client=etcd.Client(host='127.0.0.1', port=4001) docker_client = Client(base_url='unix://var/run/docker.sock', version='1.15', timeout=10) docker_container_all_name=docker_container_all() portmode='manual' container_ip='' #get container ip r = etcd_client.read('%s%s'%(docker_etcd_key,local_ip), recursive=True, sorted=True) for child in r.children: if child.dir is not True and args.container_name in child.key and 'firewall' not in child.key: container_ip=eval(child.value)['Container_ip'] if len(container_ip) == 0 and args.container_name != "all": print 'This container:%s info is not in etcd!'%args.container_name sys.exit(1) if '/'+args.container_name not in docker_container_all_name and args.container_name != "all": print 'local host docker is not container:%s!'%args.container_name sys.exit(1) if args.list: try: now_firewall_rule=etcd_client.read('%s%s/firewall/nat-%s'%(docker_etcd_key,local_ip,args.container_name)).value except KeyError: print 'This container:%s is not firewall rule!'%args.container_name sys.exit(1) if len(now_firewall_rule) >0: now_firewall_rule=eval(now_firewall_rule) print 'Follow is container:%s firewall rule!'%args.container_name for i in now_firewall_rule: print i else: print 'This container:%s is not firewall rule!'%args.container_name sys.exit(1) if args.portmode=="dynamic": try: now_port=etcd_client.read('%s%s/firewall/now_port'%(docker_etcd_key,local_ip)).value except KeyError: now_port='40000' now_port=int(now_port) + 1 key='%s%s/firewall/now_port'%(docker_etcd_key,local_ip) etcd_client.write(key,now_port) portmode=args.portmode elif args.portmode=="manual": if len(args.sport)>0: now_port=args.sport else: print 'no input source port' key='%s%s/firewall/now_port'%(docker_etcd_key,local_ip) etcd_client.write(key,now_port) #add docker container firewall rule if args.add: if args.mode: if args.source: if args.source == "all": source_ip='0.0.0.0/0.0.0.0' else: source_ip=args.source if args.portmode=="dynamic": sport=now_port else: sport=args.sport if args.dport: dport=args.dport else: print 'please input dest port!This port is container local port!' sys.exit(1) try: now_id=len(eval(etcd_client.read('%s%s/firewall/nat-%s'%(docker_etcd_key,local_ip,args.container_name)).value)) except KeyError: now_id='0' except SyntaxError: now_id='0' now_id = int(now_id) + 1 if args.mode=="internal": msg={'Id':now_id,'Mode':args.mode,'Container_name':args.container_name,'Source_ip':source_ip,'Port_mode':portmode,'Source_port':'%s'%sport,'Local_port':dport,'Container_ip':container_ip} else: if args.dest: msg={'Id':now_id,'Mode':args.mode,'Container_name':args.container_name,'Source_ip':source_ip,'Destination_ip':args.dest,'Port_mode':portmode,'Source_port':'%s'%sport,'Local_port':dport,'Container_ip':container_ip} else: print 'please input destination ip' sys.exit(1) #add rule to iptables try: now_firewall_rule=etcd_client.read('%s%s/firewall/nat-%s'%(docker_etcd_key,local_ip,args.container_name)).value now_firewall_rule=eval(now_firewall_rule) except KeyError: now_firewall_rule=[] except SyntaxError: now_firewall_rule=[] for i in now_firewall_rule: if msg['Local_port'] == i['Local_port'] and msg['Source_ip'] == i['Source_ip'] and msg['Mode'] == i['Mode'] and msg['Container_name'] == i['Container_name'] and msg['Source_port'] == i['Source_port']: print 'This rule had exist!' sys.exit(1) now_firewall_rule.append(msg) key='%s%s/firewall/nat-%s'%(docker_etcd_key,local_ip,args.container_name) etcd_client.write(key,now_firewall_rule) for i in now_firewall_rule: print i #del exist firewall rule if args.rm: try: now_info=eval(etcd_client.read('%s%s/firewall/nat-%s'%(docker_etcd_key,local_ip,args.container_name)).value) except KeyError: print 'This Container:%s is not firewall rule!'%args.container_name sys.exit(1) except SyntaxError: print 'This container:%s is not firewall rule!'%args.container_name sys.exit(1) old_id=[i['Id'] for i in now_info] if args.rm != 'all': if int(args.rm) not in old_id: print 'you input rule id %s is not exit!'%args.rm sys.exit(1) for i in now_info: if int(args.rm) == i['Id']: now_info.remove(i) print 'Follow is container_name:%s new firewall rule!'%args.container_name for i in now_info: print i key='%s%s/firewall/nat-%s'%(docker_etcd_key,local_ip,args.container_name) etcd_client.write(key,now_info) sys.exit(0) else: now_info='' key='%s%s/firewall/nat-%s'%(docker_etcd_key,local_ip,args.container_name) etcd_client.write(key,now_info) print 'This container_name:%s is not firewall rule!'%args.container_name sys.exit(0) #effect container firewall rule if args.effect: #check firewall filter exist config_dir='/root/firewall' iptables_config='iptables_base.txt' if os.path.exists(config_dir) is False: os.mkdir(config_dir) if os.path.isfile('%s/%s'%(config_dir,iptables_config)) is False: print 'no found base iptables config in %s/%s!'%(config_dir,iptables_config) sys.exit(1) docker_container_run=docker_container_run() etcd_exist_firewall=[] if args.container_name != "all": container_name=args.container_name try: now_info=eval(etcd_client.read('%s%s/firewall/nat-%s'%(docker_etcd_key,local_ip,args.container_name)).value) msg=[] msg.append('#follow is container:%s firewall rulen'%args.container_name) for i in now_info: if 'Destination_ip' not in i: text='-A DOCKER -s %s ! -i ovs2 -p tcp -m tcp --dport %s -j DNAT --to-destination %s:%s'%(i['Source_ip'],i['Source_port'],i['Container_ip'].split('/')[0],i['Local_port']) msg.append('%sn'%text) else: text='-A DOCKER -s %s -d %s ! -i ovs2 -p tcp -m tcp --dport %s -j DNAT --to-destination %s:%s'%(i['Source_ip'],i['Destination_ip'],i['Source_port'],i['Container_ip'].split('/')[0],i['Local_port']) msg.append('%sn'%text) except SyntaxError: msg='' #wirte container firewall rule iptables_new_config='iptables_nat_%s.txt'%args.container_name f=open('%s/%s'%(config_dir,iptables_new_config),'w') for i in msg: f.write(i) f.close() else: r = etcd_client.read('%s%s/firewall'%(docker_etcd_key,local_ip), recursive=True, sorted=True) for child in r.children: if child.dir is not True and 'nat' in child.key and child.key.split('/')[-1].split('nat-')[-1] in docker_container_run: #etcd_exist_firewall.append(child.key.split('/')[-1].split('nat-')[-1]) try: now_info=eval(etcd_client.read(child.key).value) msg=[] msg.append('#follow is container:%s firewall rulen'%child.key.split('/')[-1].split('nat-')[-1]) for i in now_info: if 'Destination_ip' not in i: text='-A DOCKER -s %s ! -i ovs2 -p tcp -m tcp --dport %s -j DNAT --to-destination %s:%s'%(i['Source_ip'],i['Source_port'],i['Container_ip'].split('/')[0],i['Local_port']) msg.append('%sn'%text) else: text='-A DOCKER -s %s -d %s ! -i ovs2 -p tcp -m tcp --dport %s -j DNAT --to-destination %s:%s'%(i['Source_ip'],i['Destination_ip'],i['Source_port'],i['Container_ip'].split('/')[0],i['Local_port']) msg.append('%sn'%text) except SyntaxError: msg='' #wirte container firewall rule iptables_new_config='iptables_nat_%s.txt'%child.key.split('/')[-1].split('nat-')[-1] f=open('%s/%s'%(config_dir,iptables_new_config),'w') for i in msg: f.write(i) f.close() #get now all container firewall rule all_firewall_file=[] for parent,dirnames,filenames in os.walk(config_dir): for filename in filenames: if 'iptables_nat' in filename: all_firewall_file.append(os.path.join(parent,filename)) #get iptables base file line count = len(open('%s/%s'%(config_dir,iptables_config),'rU').readlines()) modify_post=int(count)-1 f=open('%s/%s'%(config_dir,iptables_config),'r+') flist=f.readlines() flist[modify_post]='' f=open for i in all_firewall_file: f=open(i) try: container_text=f.read() finally: f.close() flist.append(container_text) flist.append('COMMITn') f=open('%s/temp_iptables.txt'%config_dir,'w') for i in flist: f.write(i) f.close() #apply new firewall rule shutil.copy('%s/temp_iptables.txt'%config_dir,'/etc/sysconfig/iptables') #restart firewall firewall_status=((subprocess.Popen("systemctl restart iptables &>>/dev/null && echo 0 || echo 1",shell=True,stdout=subprocess.PIPE)).stdout.readlines()[0]).strip('n') if firewall_status != "0": print 'firewall rule has problem!' sys.exit(1) else: print 'config firewall rule is success!' sys.exit(0) if args.addip: if '/' not in args.addip: print 'please input ip:netmask!' sys.exit(1) external_ip=args.addip.split('/')[0] external_ip_netmask=args.addip.split('/')[1] #nmap ip exist! nm = nmap.PortScanner() nmap_result=nm.scan(external_ip,'60020')['nmap']['scanstats']['uphosts'] if int(nmap_result) == 1: print 'you input ip:%s is online!'%external_ip sys.exit(1) try: now_ip=eval(etcd_client.read('%s%s/external_ip/%s'%(docker_etcd_key,local_ip,external_ip)).value) if now_ip['Container_name'] != args.container_name: print 'this is external ip:%s is has used by container:%s.if you want to use it again,please delete this key:%s.'%(args.addip,now_ip['Container_name'],'%s%s/external_ip/%s'%(docker_etcd_key,local_ip,external_ip)) sys.exit(1) except KeyError: pass #get device info try: now_device=etcd_client.read('%s%s/external_ip/device'%(docker_etcd_key,local_ip)).value except KeyError: now_device='em2:0' new_device=now_device.split(':')[0]+':'+str(int(now_device.split(':')[1])+1) key='%s%s/external_ip/device'%(docker_etcd_key,local_ip) etcd_client.write(key,new_device) #add new external ip in localhost if int(external_ip_netmask) == 8: external_ip_netmask='255.0.0.0' elif int(external_ip_netmask) == 16: external_ip_netmask='255.255.0.0' elif int(external_ip_netmask) == 24: external_ip_netmask='255.255.255.0' elif int(external_ip_netmask) == 32: external_ip_netmask='255.255.255.255' else: print 'you input netmask %s i can not calculate'%external_ip_netmask sys.exit(1) add_external_ip_status=((subprocess.Popen("/sbin/ifconfig %s %s netmask %s up &>>/dev/null && echo 0 || echo 1"%(new_device,external_ip,external_ip_netmask),shell=True,stdout=subprocess.PIPE)).stdout.readlines()[0]).strip('n') if add_external_ip_status != "0": print 'add external ip:%s is fail!'%args.addip sys.exit(1) else: print 'add external ip:%s is success!'%args.addip key='%s%s/external_ip/%s'%(docker_etcd_key,local_ip,external_ip) info={'Ip':external_ip,'Netmask':external_ip_netmask,'Container_name':args.container_name,'Device':new_device,'Date':time.strftime('%Y.%m.%d-%T')} etcd_client.write(key,info) sys.exit(0) if args.rmip: try: now_ip=eval(etcd_client.read('%s%s/external_ip/%s'%(docker_etcd_key,local_ip,args.rmip)).value) except KeyError: print 'This external ip:%s is not use in etcd!'%args.rmip sys.exit(1) if now_ip['Container_name'] != args.container_name: print 'this is external ip:%s is has used by container:%s.if you want to delete it,please input correct container:%s and external ip:%s.'%(args.rmip,now_ip['Container_name'],now_ip['Container_name'],now_ip['Ip']) sys.exit(1) #delete use external ip in localhost delete_external_ip_status=((subprocess.Popen("/sbin/ifconfig %s down &>>/dev/null && echo 0 || echo 1"%(now_ip['Device']),shell=True,stdout=subprocess.PIPE)).stdout.readlines()[0]).strip('n') if delete_external_ip_status != "0": print 'delete external ip:%s is fail!'%args.rmip sys.exit(1) else: print 'delete external ip:%s is success!'%args.rmip key='%s%s/external_ip/%s'%(docker_etcd_key,local_ip,args.rmip) etcd_client.delete(key) sys.exit(0) sys.exit(1)建议根据自己实际环境来修改上面代码，我这个仅是让大家参考我的运行方式，以及可以使用这个东东来解决自己的问题，大家如果有其他的方法也可以使用。
二、运行
1、使用帮助
[root@docker-test3 code]# python modify_docker_container_firewall.py -h
usage: modify_docker_container_firewall.py [-h] [-l] [-a] [-r RM]
[-m {internal,external}]
[-s SOURCE] [-sp SPORT] [-d DEST]
[-dp DPORT] [-pm {dynamic,manual}]
[-e] [-ap ADDIP] [-rp RMIP]
container_name
It is userful tool to modify docker container firewall
positional arguments:
container_name list local docker container name
optional arguments:
-h, --help show this help message and exit
-l, --list show container firewall rules
-a, --add add container firewall rules
-r RM, --rm RM rm container firewall rules
-m {internal,external}, --mode {internal,external}
set container firewall mode
-s SOURCE, --source SOURCE
source ip view container firewall rules
-sp SPORT, --sport SPORT
source port view container firewall rules
-d DEST, --dest DEST destination ip container firewall rules
-dp DPORT, --dport DPORT
destination port view container firewall rules
-pm {dynamic,manual}, --portmode {dynamic,manual}
set container port mode
-e, --effect effect container firewall rules
-ap ADDIP, --addip ADDIP
add external ip to container
-rp RMIP, --rmip RMIP
rm external ip to container 说明：
-l是展示当前容器的所有防火墙规则，后面不加值
-a是进行增加容器防火墙规则，后面不加值
-r是删除容器规则，后面是防火墙的id值
-m是防火墙规则的模式，有internal（内部模式）与external（外部模式），如果你使用跟宿主机一样出口ip的话，选择internal，如果使用独立的外网ip，就选择external
-s是来源ip，后面是ip/netmask模式
-sp是来源访问端口，后面需要输入允许外边访问进来的端口
-d是指定独立的外网ip，后面需要输入ip值
-dp是指定访问容器的端口，后面是属于端口值
-pm是指定来源端口的模式，有dynamic与manual模式，如果选择dynamic就不需要你指定来源端口值，会自动生成一个值，使用manual的话，就需要你手动输入一个来源端口值
-e是设置好防火墙后，应用此防火墙策略，也就是在iptables里生效，后面不加值
-ap是在本机增加独立的外网ip，后面是ip/netmask模式
-rp是删除本机独立外网ip，后面是ip模式
2、查看test1的规则
目前我本机有test1-3这3个容器
[root@docker-test3 code]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES c07b67c2382a docker.ops-chukong.com:5000/centos6-http:new "/usr/bin/supervisor 40 hours ago Up 40 hours test2 7c31bbfe0091 docker.ops-chukong.com:5000/centos6-http:new "/usr/bin/supervisor 3 days ago Up 15 hours test1 5eede798f189 docker.ops-chukong.com:5000/centos6-http:new "/usr/bin/supervisor 3 days ago Exited (0) 24 hours ago test3其中test1与test2都使用持久化固定ip了，test3没有
现在使用智能防火墙查看test1 的防火墙
[root@docker-test3 code]# python modify_docker_container_firewall.py test1 -l This container:test1 is not firewall rule! [root@docker-test3 code]# python modify_docker_container_firewall.py test2 -l This container:test2 is not firewall rule! [root@docker-test3 code]# python modify_docker_container_firewall.py test3 -l This container:test3 info is not in etcd!可以看到test1与test2都是没有规则的，而test3每一使用持久化固定ip，所以没办法查看数据
3、添加规则
先给test1添加内部模式规则-m internal，这样能使用docker宿主机的外网ip+port访问
[root@docker-test3 code]# python modify_docker_container_firewall.py test1 -a -m internal -s 1.1.1.1/24 -pm dynamic -dp 22
{'Mode': 'internal', 'Container_name': 'test1', 'Source_port': '40030', 'Port_mode': 'dynamic', 'Local_port': '22', 'Source_ip': '1.1.1.1/24', 'Id': 1, 'Container_ip': '172.16.1.2/24'} 这样是给与test1容器设置，允许1.1.1.1/24通过40030（使用动态模式自动生成访问端口），访问test1的22端口
其中test1的容器ip自动查看，不需要收到查找与手动输入
说明
Mode是运行的模式
Container_name是对应容器名
Source_port是来源端口
Port_mode是指端口模式为动态获取
Local_port是目的端口
Source_ip是来源ip
Id是指防火墙规则的id，后面可以通过指定id来删除规则
Container_ip是容器的ip
以上数据均是在使用持久化故障ip脚本生成容器的时候，程序自动把数据写入到etcd里，然后只能防火墙也通过etcd获取数据
在给test1添加一个使用手动模式设置来源端口
[root@docker-test3 code]# python modify_docker_container_firewall.py test1 -a -m internal -s 1.1.1.1/24 -pm manual -sp 400031 -dp 22
{'Mode': 'internal', 'Container_name': 'test1', 'Source_port': '40030', 'Source_ip': '1.1.1.1/24', 'Local_port': '22', 'Port_mode': 'dynamic', 'Id': 1, 'Container_ip': '172.16.1.2/24'}
{'Mode': 'internal', 'Container_name': 'test1', 'Source_port': '400031', 'Port_mode': 'manual', 'Local_port': '22', 'Source_ip': '1.1.1.1/24', 'Id': 2, 'Container_ip': '172.16.1.2/24'} 如果指定的来源端口相同，并且来源ip也相同会报错
[root@docker-test3 code]# python modify_docker_container_firewall.py test1 -a -m internal -s 1.1.1.1/24 -pm manual -sp 40031 -dp 22 This rule had exist!现在通过-l参数来查看当前test1的规则
[root@docker-test3 code]# python modify_docker_container_firewall.py test1 -l
Follow is container:test1 firewall rule!
{'Mode': 'internal', 'Container_name': 'test1', 'Source_port': '40030', 'Port_mode': 'dynamic', 'Local_port': '22', 'Source_ip': '1.1.1.1/24', 'Id': 1, 'Container_ip': '172.16.1.2/24'}
{'Mode': 'internal', 'Container_name': 'test1', 'Source_port': '40031', 'Source_ip': '1.1.1.1/24', 'Local_port': '22', 'Port_mode': 'manual', 'Id': 2, 'Container_ip': '172.16.1.2/24'} 可以看到有2个刚才输入的规则
4、防火墙规则生效
先查看当前宿主机防火墙
[root@docker-test3 code]# iptables -t nat -L -nv Chain PREROUTING (policy ACCEPT 117K packets, 11M bytes) pkts bytes target prot opt in out source destination 15431 914K DOCKER all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL Chain INPUT (policy ACCEPT 11906 packets, 716K bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 1462 packets, 86817 bytes) pkts bytes target prot opt in out source destination 24 1424 DOCKER all -- * * 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL Chain POSTROUTING (policy ACCEPT 4994 packets, 234K bytes) pkts bytes target prot opt in out source destination 0 0 MASQUERADE all -- * * 172.16.0.0/16 !172.16.0.0/16 Chain DOCKER (2 references) pkts bytes target prot opt in out source destination可以看到只有默认的nat规则，其他的没有
现在使用-e参数来生效
[root@docker-test3 code]# python modify_docker_container_firewall.py test1 -e config firewall rule is success!可以看到运行成功
[root@docker-test3 code]# iptables -t nat -L -nv Chain PREROUTING (policy ACCEPT 53 packets, 5242 bytes) pkts bytes target prot opt in out source destination 5 300 DOCKER all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL Chain INPUT (policy ACCEPT 5 packets, 300 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 DOCKER all -- * * 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 MASQUERADE all -- * * 172.16.0.0/16 !172.16.0.0/16 Chain DOCKER (2 references) pkts bytes target prot opt in out source destination 0 0 DNAT tcp -- !ovs2 * 1.1.1.0/24 0.0.0.0/0 tcp dpt:40030 to:172.16.1.2:22 0 0 DNAT tcp -- !ovs2 * 1.1.1.0/24 0.0.0.0/0 tcp dpt:40031 to:172.16.1.2:22规则已经运行了
在去配置文件里看看
[root@docker-test3 code]# tail -n 12 /etc/sysconfig/iptables *nat :PREROUTING ACCEPT [2:269] :POSTROUTING ACCEPT [1739:127286] :OUTPUT ACCEPT [1739:127286] :DOCKER - [0:0] -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER -A POSTROUTING -s 172.16.0.0/16 ! -d 172.16.0.0/16 -j MASQUERADE -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER #follow is container:test1 firewall rule -A DOCKER -s 1.1.1.1/24 ! -i ovs2 -p tcp -m tcp --dport 40030 -j DNAT --to-destination 172.16.1.2:22 -A DOCKER -s 1.1.1.1/24 ! -i ovs2 -p tcp -m tcp --dport 40031 -j DNAT --to-destination 172.16.1.2:22 COMMIT也是跟我们之前配置的一样，并且每个容器规则上面都有标示下面规则是属于哪个容器的，方便查看
5、删除防火墙规则
当前test1 的规则
[root@docker-test3 code]# python modify_docker_container_firewall.py test1 -l
Follow is container:test1 firewall rule!
{'Mode': 'internal', 'Container_name': 'test1', 'Source_port': '40030', 'Port_mode': 'dynamic', 'Local_port': '22', 'Source_ip': '1.1.1.1/24', 'Id': 1, 'Container_ip': '172.16.1.2/24'}
{'Mode': 'internal', 'Container_name': 'test1', 'Source_port': '40031', 'Source_ip': '1.1.1.1/24', 'Local_port': '22', 'Port_mode': 'manual', 'Id': 2, 'Container_ip': '172.16.1.2/24'} 使用-r来删除，-r后面输入id
[root@docker-test3 code]# python modify_docker_container_firewall.py test1 -r 2
Follow is container_name:test1 new firewall rule!
{'Mode': 'internal', 'Container_name': 'test1', 'Source_port': '40030', 'Port_mode': 'dynamic', 'Local_port': '22', 'Source_ip': '1.1.1.1/24', 'Id': 1, 'Container_ip': '172.16.1.2/24'} 然后-e生效
[root@docker-test3 code]# python modify_docker_container_firewall.py test1 -e config firewall rule is success! [root@docker-test3 code]# iptables -t nat -L -nv Chain PREROUTING (policy ACCEPT 4 packets, 200 bytes) pkts bytes target prot opt in out source destination 3 168 DOCKER all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL Chain INPUT (policy ACCEPT 2 packets, 120 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 DOCKER all -- * * 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL Chain POSTROUTING (policy ACCEPT 1 packets, 40 bytes) pkts bytes target prot opt in out source destination 0 0 MASQUERADE all -- * * 172.16.0.0/16 !172.16.0.0/16 Chain DOCKER (2 references) pkts bytes target prot opt in out source destination 0 0 DNAT tcp -- !ovs2 * 1.1.1.0/24 0.0.0.0/0 tcp dpt:40030 to:172.16.1.2:22 [root@docker-test3 code]# tail -n 12 /etc/sysconfig/iptables # Completed on Fri Dec 6 10:59:13 2013 *nat :PREROUTING ACCEPT [2:269] :POSTROUTING ACCEPT [1739:127286] :OUTPUT ACCEPT [1739:127286] :DOCKER - [0:0] -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER -A POSTROUTING -s 172.16.0.0/16 ! -d 172.16.0.0/16 -j MASQUERADE -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER #follow is container:test1 firewall rule -A DOCKER -s 1.1.1.1/24 ! -i ovs2 -p tcp -m tcp --dport 40030 -j DNAT --to-destination 172.16.1.2:22 COMMIT可以看到已经智能自动修改了
6、添加额外的外网ip
使用-ap来添加
[root@docker-test3 code]# python modify_docker_container_firewall.py test1 -ap 117.121.x.99/24 add external ip:117.121.x.99/24 is success! [root@docker-test3 code]# ping 117.121.x.99 -c 2 PING 117.121.x.99 (117.121.x.99) 56(84) bytes of data. 64 bytes from 117.121.x.99: icmp_seq=1 ttl=64 time=0.039 ms 64 bytes from 117.121.x.99: icmp_seq=2 ttl=64 time=0.032 ms --- 117.121.x.99 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 999ms rtt min/avg/max/mdev = 0.032/0.035/0.039/0.006 ms可以看到已经添加成功，并且能ping通，安全起见，我把额外的外网ip第三位添加个x，防止坏人扫描。
在通过修改防火墙来设置外部模式策略
[root@docker-test3 code]# python modify_docker_container_firewall.py test1 -a -m external -pm dynamic -s all -d 117.121.x.99 -dp 22
{'Mode': 'internal', 'Container_name': 'test1', 'Source_port': '40030', 'Source_ip': '1.1.1.1/24', 'Local_port': '22', 'Port_mode': 'dynamic', 'Id': 1, 'Container_ip': '172.16.1.2/24'}
{'Destination_ip': '117.121.x.99', 'Mode': 'external', 'Container_name': 'test1', 'Source_port': '40033', 'Port_mode': 'dynamic', 'Local_port': '22', 'Source_ip': '0.0.0.0/0.0.0.0', 'Id': 2, 'Container_ip': '172.16.1.2/24'} 备注：其中-s all是允许所有公网ip访问
然后生效
[root@docker-test3 code]# python modify_docker_container_firewall.py test1 -e config firewall rule is success! [root@docker-test3 code]# iptables -t nat -L -nv Chain PREROUTING (policy ACCEPT 16 packets, 1308 bytes) pkts bytes target prot opt in out source destination 1 60 DOCKER all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL Chain INPUT (policy ACCEPT 1 packets, 60 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 DOCKER all -- * * 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 MASQUERADE all -- * * 172.16.0.0/16 !172.16.0.0/16 Chain DOCKER (2 references) pkts bytes target prot opt in out source destination 0 0 DNAT tcp -- !ovs2 * 1.1.1.0/24 0.0.0.0/0 tcp dpt:40030 to:172.16.1.2:22 0 0 DNAT tcp -- !ovs2 * 0.0.0.0/0 117.121.x.99 tcp dpt:40033 to:172.16.1.2:22然后ssh登陆试试
[root@docker-test3 code]# ssh 117.121.x.99 -l root -p 40033 The authenticity of host '[117.121.x.99]:40033 ([117.121.x.99]:40033)' can't be established. RSA key fingerprint is 39:7c:13:9f:d4:b0:d7:63:fc:ff:ae:e3:46:a4:bf:6b. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '[117.121.x.99]:40033' (RSA) to the list of known hosts. [email protected]'s password: Last login: Thu Mar 12 11:04:04 2015 from 211.151.20.221 root@7c31bbfe0091:~ 11:04:57 # ifconfig eth1 Link encap:Ethernet HWaddr 66:17:20:C3:4E:21 inet addr:172.16.1.2 Bcast:0.0.0.0 Mask:255.255.255.0 inet6 addr: fe80::6417:20ff:fec3:4e21/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:606 errors:0 dropped:2 overruns:0 frame:0 TX packets:411 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:52692 (51.4 KiB) TX bytes:45451 (44.3 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)可以设置的外网ip可以正常登陆到test1的容器里
7、测试批量生效防火墙策略
目前仅有test1有防火墙测试了，现在给test2也设置
[root@docker-test3 code]# python modify_docker_container_firewall.py test2 -a -m internal -s 1.1.1.1/24 -pm dynamic -dp 22
{'Mode': 'internal', 'Container_name': 'test2', 'Source_port': '40034', 'Port_mode': 'dynamic', 'Local_port': '22', 'Source_ip': '0.0.0.0/0.0.0.0', 'Id': 1, 'Container_ip': '172.16.1.4/24'}
{'Mode': 'internal', 'Container_name': 'test2', 'Source_port': '40035', 'Source_ip': '0.0.0.0/0.0.0.0', 'Local_port': '22', 'Port_mode': 'dynamic', 'Id': 2, 'Container_ip': '172.16.1.4/24'}
{'Mode': 'internal', 'Container_name': 'test2', 'Source_port': '40036', 'Port_mode': 'dynamic', 'Local_port': '22', 'Source_ip': '1.1.1.1/24', 'Id': 3, 'Container_ip': '172.16.1.4/24'} 然后我在修改一下test1的
[root@docker-test3 code]# python modify_docker_container_firewall.py test1 -a -m internal -s 2.2.2.2/24 -pm dynamic -dp 22
{'Mode': 'internal', 'Container_name': 'test1', 'Source_port': '40030', 'Port_mode': 'dynamic', 'Local_port': '22', 'Source_ip': '1.1.1.1/24', 'Id': 1, 'Container_ip': '172.16.1.2/24'}
{'Destination_ip': '117.121.x.99', 'Mode': 'external', 'Container_name': 'test1', 'Source_port': '40033', 'Source_ip': '0.0.0.0', 'Local_port': '22', 'Port_mode': 'dynamic', 'Id': 2, 'Container_ip': '172.16.1.2/24'}
{'Mode': 'internal', 'Container_name': 'test1', 'Source_port': '40037', 'Port_mode': 'dynamic', 'Local_port': '22', 'Source_ip': '2.2.2.2/24', 'Id': 3, 'Container_ip': '172.16.1.2/24'} 之前如果想动态生效需要知道容器名，下面可以把容器名那里输入all，就可以生效所有已经设置的规则
[root@docker-test3 code]# python modify_docker_container_firewall.py all -e config firewall rule is success! [root@docker-test3 code]# iptables -t nat -L -nv Chain PREROUTING (policy ACCEPT 20 packets, 1914 bytes) pkts bytes target prot opt in out source destination 1 60 DOCKER all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL Chain INPUT (policy ACCEPT 1 packets, 60 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 DOCKER all -- * * 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 MASQUERADE all -- * * 172.16.0.0/16 !172.16.0.0/16 Chain DOCKER (2 references) pkts bytes target prot opt in out source destination 0 0 DNAT tcp -- !ovs2 * 1.1.1.0/24 0.0.0.0/0 tcp dpt:40030 to:172.16.1.2:22 0 0 DNAT tcp -- !ovs2 * 0.0.0.0 117.121.x.99 tcp dpt:40033 to:172.16.1.2:22 0 0 DNAT tcp -- !ovs2 * 2.2.2.0/24 0.0.0.0/0 tcp dpt:40037 to:172.16.1.2:22 0 0 DNAT tcp -- !ovs2 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:40034 to:172.16.1.4:22 0 0 DNAT tcp -- !ovs2 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:40035 to:172.16.1.4:22 0 0 DNAT tcp -- !ovs2 * 1.1.1.0/24 0.0.0.0/0 tcp dpt:40036 to:172.16.1.4:22 [root@docker-test3 code]# tail -n 15 /etc/sysconfig/iptables :POSTROUTING ACCEPT [1739:127286] :OUTPUT ACCEPT [1739:127286] :DOCKER - [0:0] -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER -A POSTROUTING -s 172.16.0.0/16 ! -d 172.16.0.0/16 -j MASQUERADE -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER #follow is container:test1 firewall rule -A DOCKER -s 1.1.1.1/24 ! -i ovs2 -p tcp -m tcp --dport 40030 -j DNAT --to-destination 172.16.1.2:22 -A DOCKER -s 0.0.0.0 -d 117.121.x.99 ! -i ovs2 -p tcp -m tcp --dport 40033 -j DNAT --to-destination 172.16.1.2:22 -A DOCKER -s 2.2.2.2/24 ! -i ovs2 -p tcp -m tcp --dport 40037 -j DNAT --to-destination 172.16.1.2:22 #follow is container:test2 firewall rule -A DOCKER -s 0.0.0.0/0.0.0.0 ! -i ovs2 -p tcp -m tcp --dport 40034 -j DNAT --to-destination 172.16.1.4:22 -A DOCKER -s 0.0.0.0/0.0.0.0 ! -i ovs2 -p tcp -m tcp --dport 40035 -j DNAT --to-destination 172.16.1.4:22 -A DOCKER -s 1.1.1.1/24 ! -i ovs2 -p tcp -m tcp --dport 40036 -j DNAT --to-destination 172.16.1.4:22 COMMIT很简单吧，比以前需要找到容器的ip，然后收到去防火墙里修改，并且修改的时候，还得计算给予的来源端口，现在是否舒服很多，以后有平台想使用的话，直接调用就可以。
8、删除额外的外网ip
使用rp参数
[root@docker-test3 code]# python modify_docker_container_firewall.py test2 -rp 117.121.x.99 this is external ip:117.121.x.99 is has used by container:test1.if you want to delete it,please input correct container:test1 and external ip:117.121.x.99. [root@docker-test3 code]# python modify_docker_container_firewall.py test1 -rp 117.121.x.99 delete external ip:117.121.x.99 is success!删除的时候，还必须制定对应的容器与额外外网ip，否则删除失败，但提供对于的容器与ip
[root@docker-test3 code]# ping -c2 117.121.x.99 PING 117.121.x.99 (117.121.x.99) 56(84) bytes of data. From 117.121.x.3 icmp_seq=1 Destination Host Unreachable From 117.121.x.3 icmp_seq=2 Destination Host Unreachable --- 117.121.x.99 ping statistics --- 2 packets transmitted, 0 received, +2 errors, 100% packet loss, time 1000ms pipe 2可以看到已经删除完成，ping不通了。
目前docker方面的文章，已经完成了：
1、安装docker
2、动态扩容docker容器的空间
3、动态绑定volume
4、docker多主机网络互联
5、docker持久化固定ip
6、docker智能添加与修改防火墙
以后有空给大家分享docker集群与平台方面知识，希望大家多提意见。
转自：http://www.aixchina.net/Document/detail/tid/123369