宜信漏洞管理平台《洞察》阿里云ECS部署
引言
洞察是宜信安全部用来对公司内部系统所出现的安全漏洞进行线上全生命周期管理的漏洞管理平台，并在2018年4月下旬对外开源。
主要由3部分组成：
- 应用系统资产管理
- 漏洞生命周期管理
- 安全知识库管理
洞察使用了Python语言进行开发，利用Flask框架+MySQL+Docker部署实现。
GitHub项目地址：https://github.com/creditease-sec/insight
感谢宜信安全应急响应中心的开源，给企业和安全从业者带来一个很好的漏洞管理系统。
实验环境
- 阿里云ECS
- 系统镜像：CentOS 7.4
- 安装软件版本：MySQL 5.6 、 Docker 1.13.1 、Nginx 1.12.2
实验说明
- 1、本文档的主要步骤参考github上的安装文档的部分，并对按照官方文档执行报错的地方进行了补充说明或更正；
- 2、本文档邮箱部分设置采用默认设置，实际应用安装可参考github 安装说明中 《部署和启动APP》中 2、3、4部分；
- 3、增加了安装nginx，并设置端口转发，可以让其他用户访问所安装的漏洞管理系统；
- 4、欢迎各位参考和批评指正。
第一步： 安装docker、MySQL Client、Git
- 1.1、登陆服务器，执行 "yum -y install docker mariadb git telnet "安装所需软件；
- 1.2、启动docker，并加入开机启动。
过程记录如下：
---安装需要软件
[root@Insight-APP01 opt]# yum -y install docker mariadb git telnet
.......执行过程省略......
---查看安装是否完成
[root@Insight-APP01 opt]# yum -y install docker mariadb git telnet
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Package 2:docker-1.13.1-53.git774336d.el7.centos.x86_64 already installed and latest version
Package 1:mariadb-5.5.56-2.el7.x86_64 already installed and latest version
Package git-1.8.3.1-12.el7_4.x86_64 already installed and latest version
Package 1:telnet-0.17-64.el7.x86_64 already installed and latest version
Nothing to do
---启动docker并加入开启启动项
[root@Insight-APP /]# systemctl enable docker && systemctl start docker
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.
---查看docker版本
[root@Insight-APP /]# docker --version
Docker version 1.13.1, build 774336d/1.13.1
第二步： 拉取MySQL 5.6镜像并启动MySQL
- 2.1、执行"docker pull mysql:5.6"拉取MySQL 5.6镜像;
- 2.2、执行"docker run -d -p 127.0.0.1:6606:3306 --name open_source_mysqldb -e MYSQL_ROOT_PASSWORD=root mysql:5.6"启动MySQL。
过程执行如下：
---拉取MySQL 5.6镜像
[root@Insight-APP /]# docker pull mysql:5.6
Trying to pull repository docker.io/library/mysql ...
5.6: Pulling from docker.io/library/mysql
f2aa67a397c4: Pull complete
1accf44cb7e0: Pull complete
2d830ea9fa68: Pull complete
740584693b89: Pull complete
4d620357ec48: Pull complete
f5cd6ee094d7: Pull complete
09f4550fd7ee: Pull complete
f6cac39ec886: Pull complete
a18dadcb91ad: Pull complete
98882079d267: Pull complete
5b36fb1e0cb2: Pull complete
Digest: sha256:c636cf8b6d07293d7d05446c1b4e91f799472c6eb858cce4c60b965a0c56561e
Status: Downloaded newer image for docker.io/mysql:5.6
---查看执行结果
[root@Insight-APP /]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/mysql 5.6 e09f6de95634 25 hours ago 256 MB
---启动MySQL
[root@Insight-APP /]# docker run -d -p 127.0.0.1:6606:3306 --name open_source_mysqldb -e MYSQL_ROOT_PASSWORD=root mysql:5.6
d45f960073bf0d702e3333b37f9dfc79f6d341e7de11c70b09282974d4fedd76
---查看容器运行情况
[root@Insight-APP /]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d45f960073bf mysql:5.6 "docker-entrypoint..." 6 seconds ago Up 4 seconds 127.0.0.1:6606->3306/tcp open_source_mysqldb
第三步： 创建数据库和账号权限配置
- 3.1、在主机命令窗口，执行"mysql -h 127.0.0.1 -P 6606 -u root -p"登陆数据库；（参考官方文档时，此步骤会报错，原因时没安装MySQL客户端）
- 3.2、登陆数据库后，参考官方稳定，创建数据库、账号及授权。（本部分对比官方文档，增加了127.0.0.1和localhost可以登陆vuldb数据库）
过程执行如下
---登陆数据库
[root@Insight-APP01 /]# mysql -h 127.0.0.1 -P 6606 -u root -p
Enter password:
---创建数据库、用户和授权
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.40 MySQL Community Server (GPL)
Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MySQL [(none)]> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
MySQL [mysql]> CREATE DATABASE IF NOT EXISTS vuldb DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)
MySQL [mysql]> grant all on vuldb.* to vuluser@'%' identified by 'vulpassword';
Query OK, 0 rows affected (0.00 sec)
MySQL [mysql]> grant all on vuldb.* to vuluser@'localhost';
Query OK, 0 rows affected (0.00 sec)
MySQL [mysql]> grant all on vuldb.* to vuluser@'127.0.0.1';
Query OK, 0 rows affected (0.00 sec)
MySQL [mysql]> flush privileges;
Query OK, 0 rows affected (0.00 sec)
MySQL [mysql]> quit;
Bye
- 3.3、验证数据库创建和授权结果，过程执行如下：
---查看容器ID
[root@Insight-APP01 /]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2b671f752009 mysql:5.6 "docker-entrypoint..." 3 minutes ago Up 2 minutes 127.0.0.1:6606->3306/tcp open_source_mysqldb
---登陆到MySQL容器内
[root@Insight-APP01 /]# docker exec -ti 2b671f752009 /bin/bash
---切换环境成功,,查看数据库
root@2b671f752009:/# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.40 MySQL Community Server (GPL)
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| vuldb |
+--------------------+
4 rows in set (0.00 sec)
---退出容器环境到主机下
mysql> quit
Bye
root@2b671f752009:/# exit
exit
第四步： 部署和启动APP
- 4.1、执行命令"git clone https://github.com/creditease-sec/insight.git"，从GitHub上clone源码到/opt目录下；
- 4.2、修改邮箱配置文件，此部分省略，参考官方：安装文档
- 4.3、在/opt/insight/目录下，执行"docker pull daocloud.io/liusheng/vulpm_docker:latest"拉取最新镜像。
过程执行如下：
---在/opt目录下，下载源码
[root@Insight-APP01 /]# cd /opt/
[root@Insight-APP01 opt]# git clone https://github.com/creditease-sec/insight.git
Cloning into 'insight'...
remote: Counting objects: 2885, done.
remote: Compressing objects: 100% (1205/1205), done.
remote: Total 2885 (delta 1668), reused 2865 (delta 1650), pack-reused 0
Receiving objects: 100% (2885/2885), 9.93 MiB | 4.10 MiB/s, done.
Resolving deltas: 100% (1668/1668), done.
---查看下载结果
[root@Insight-APP01 opt]# ll -sht
total 4.0K
4.0K drwxr-xr-x 5 root root 4.0K May 1 15:31 insight
---切换到项目根目录/opt/insight/下，拉取镜像
[root@Insight-APP01 opt]# cd insight/
[root@Insight-APP01 insight]# docker pull daocloud.io/liusheng/vulpm_docker:latest
Trying to pull repository daocloud.io/liusheng/vulpm_docker ...
latest: Pulling from daocloud.io/liusheng/vulpm_docker
8d79cffd937a: Pull complete
32e85905550a: Pull complete
db09ba89643a: Pull complete
a1686d0cb069: Pull complete
ddff347288f6: Pull complete
74b81186cf37: Pull complete
0fd9f7ca9d4f: Pull complete
ff123365e789: Pull complete
7d527ac71904: Pull complete
b3bc258e3f61: Pull complete
2f378852dda8: Pull complete
Digest: sha256:95eda77659472fd70c2799bc9d75f3aeca2bfe0b1e28cce9359191638268c93e
Status: Downloaded newer image for daocloud.io/liusheng/vulpm_docker:latest
---查看拉取结果
[root@Insight-APP01 insight]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/mysql 5.6 e09f6de95634 25 hours ago 256 MB
daocloud.io/liusheng/vulpm_docker latest 46c491c92fac 6 weeks ago 695 MB
- 4.4、执行如下命令，启动SRCPM容器，
docker run -d -p 127.0.0.1:9000:5000 \
--link open_source_mysqldb:db \
--name open_source_srcpm \
--privileged=true \
-v $PWD/srcpm:/opt/webapp/srcpm \
-e DEV_DATABASE_URL='mysql://vuluser:vulpassword@db/vuldb' \
-e SrcPM_CONFIG=development \
-e MAIL_PASSWORD='X6x6x6' \
daocloud.io/liusheng/vulpm_docker:latest \
sh -c 'supervisord -c srcpm/supervisor.conf && supervisorctl -c srcpm/supervisor.conf start all && tail -f srcpm/log/gunicorn.err && tail -f srcpm/log/mail_sender.err'
过程执行如下：
--- 启动srcpm容器
[root@Insight-APP01 insight]# docker run -d -p 127.0.0.1:9000:5000 \
> --link open_source_mysqldb:db \
> --name open_source_srcpm \
> --privileged=true \
> -v $PWD/srcpm:/opt/webapp/srcpm \
> -e DEV_DATABASE_URL='mysql://vuluser:vulpassword@db/vuldb' \
> -e SrcPM_CONFIG=development \
> -e MAIL_PASSWORD='123456' \
> daocloud.io/liusheng/vulpm_docker:latest \
> sh -c 'supervisord -c srcpm/supervisor.conf && supervisorctl -c srcpm/supervisor.conf start all && tail -f srcpm/log/gunicorn.err && tail -f srcpm/log/mail_sender.err'
1004664017b9fd21826a66cd84b1e3520d9b69f53b546613b7c1327b96684d75
---查看启动结果
[root@Insight-APP01 insight]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1004664017b9 daocloud.io/liusheng/vulpm_docker:latest "sh -c 'supervisor..." 9 seconds ago Up 9 seconds 127.0.0.1:9000->5000/tcp open_source_srcpm
2b671f752009 mysql:5.6 "docker-entrypoint..." 13 minutes ago Up 13 minutes 127.0.0.1:6606->3306/tcp open_source_mysqldb
第五步： 初始化数据库
- 5.1、在主机命令行，切换到/opt/insight/srcpm下，执行"mysql -h 127.0.0.1 -P 6606 -u root -p vuldb < vuldb_init.sql"初始化数据库。
过程执行如下：
---找到初始化脚本
[root@Insight-APP01 insight]# cd /opt/insight/srcpm/
[root@Insight-APP01 srcpm]# ll
total 80
drwxr-xr-x 9 root root 4096 May 1 15:38 app
-rw-r--r-- 1 root root 3162 May 1 15:31 config.py
-rw-r--r-- 1 root root 2696 May 1 15:38 config.pyc
drwxr-xr-x 2 root root 4096 May 1 15:31 log
-rw-r--r-- 1 root root 6432 May 1 15:31 mail_sender.py
-rw-r--r-- 1 root root 1250 May 1 15:31 manage.py
-rw-r--r-- 1 root root 1382 May 1 15:38 manage.pyc
-rw-r--r-- 1 root root 169 May 1 15:31 pip.conf
-rw-r--r-- 1 root root 389 May 1 15:31 requirement.txt
-rw-r--r-- 1 root root 10254 May 1 15:31 supervisor.conf
drwxr-xr-x 2 root root 4096 May 1 15:31 tests
drwxr-xr-x 3 root root 4096 May 1 15:31 venv_srcpm
-rw-r--r-- 1 root root 17165 May 1 15:31 vuldb_init.sql
---初始化数据库，输入密码，没有报错初始化完成
[root@Insight-APP01 srcpm]# mysql -h 127.0.0.1 -P 6606 -u root -p vuldb < vuldb_init.sql
Enter password:
---检查数据库初始化结果
[root@Insight-APP01 srcpm]# mysql -h 127.0.0.1 -P 6606 -u root -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.6.40 MySQL Community Server (GPL)
Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MySQL [(none)]> use vuldb
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
MySQL [vuldb]> show tables;
+-----------------+
| Tables_in_vuldb |
+-----------------+
| alembic_version |
| assets |
| categorys |
| comment |
| departs |
| login_users |
| permissions |
| postdrops |
| relationships |
| roles |
| tags |
| users |
| vul_logs |
| vul_reports |
| vul_types |
+-----------------+
15 rows in set (0.00 sec)
第六步： 安装配置Nginx，发布服务到外网环境
- 6.1、执行命令"wget http://127.0.0.1:9000/srcpm/"检查在服务器上，能否正常访问;
- 6.2、执行命令"yum -y install nginx"，安装Nginx;
- 6.3、修改nginx配置文件，主要修改第41和48行；
- 6.4、启动nginx，并加入到开机启动项。
过程执行如下：
---测试在服务器上能否正常访问
[root@Insight-APP01 opt]# wget http://127.0.0.1:9000/srcpm/
--2018-05-01 15:44:22-- http://127.0.0.1:9000/srcpm/
Connecting to 127.0.0.1:9000... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11567 (11K) [text/html]
Saving to: ‘index.html’
100%[=======================================================================================================================================>] 11,567 --.-K/s in 0s
2018-05-01 15:44:22 (266 MB/s) - ‘index.html’ saved [11567/11567]
---安装nginx，并查看版本
[root@Insight-APP01 opt]# yum -y install nginx
........
Complete!
[root@Insight-APP01 opt]# nginx -v
nginx version: nginx/1.12.2
---切换到/etc/nginx目录下，修改配置文件第41和48行
[root@Insight-APP01 nginx]# vim nginx.conf
33 # Load modular configuration files from the /etc/nginx/conf.d directory.
34 # See http://nginx.org/en/docs/ngx_core_module.html#include
35 # for more information.
36 include /etc/nginx/conf.d/*.conf;
37
38 server {
39 listen 80 default_server;
40 listen [::]:80 default_server;
41 server_name 127.0.0.1:9000;
42 root /usr/share/nginx/html;
43
44 # Load configuration files for the default server block.
45 include /etc/nginx/default.d/*.conf;
46
47 location / {
48 proxy_pass http://127.0.0.1:9000;
49 }
50
---启动nginx并加入到开机启动项
[root@Insight-APP01 nginx]# systemctl enable nginx && systemctl start nginx
Created symlink from /etc/systemd/system/multi-user.target.wants/nginx.service to /usr/lib/systemd/system/nginx.service.
---检查Nginx端口监听情况
[root@Insight-APP01 nginx]# netstat -antlp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:6606 0.0.0.0:* LISTEN 1883/docker-proxy-c
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 2575/nginx: master
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1205/sshd
tcp 0 0 127.0.0.1:9000 0.0.0.0:* LISTEN 2300/docker-proxy-c
tcp 0 52 172.31.111.150:22 161.48.212.56:64385 ESTABLISHED 1280/sshd: root@pts
tcp 0 0 172.31.111.150:44956 206.101.168.3:80 ESTABLISHED 1254/AliYunDun
tcp6 0 0 :::80 :::* LISTEN 2575/nginx: master
Nginx编辑截图如下：
至此，部署已全部完成。
第七步： 在外网环境下登陆系统
注意ECS安全组的设置，需要把服务器的80端口对需要访问的用户开放。
- 登陆地址为：http://x.x.x.x/srcpm/
- 初始化用户名：admin@admin
- 密码：11
首页截图如下：
登陆系统后，截图如下：
----完----