使用ConfigMap管理应用配置
拆分环境
| 主机名 | 角色 | ip |
|---|---|---|
| HDSS7-11.host.com | zk1.od.com(Test环境) | 10.4.7.11 |
| HDSS7-12.host.com | zk2.od.com(Prod环境) | 10.4.7.12 |
重配zookeeper
HDSS7-11.host.com上：
复制/opt/zookeeper/conf/zoo.cfg
1 |
tickTime=2000 |
HDSS7-12.host.com上：
复制/opt/zookeeper/conf/zoo.cfg
1 |
tickTime=2000 |
重启zk(删除数据文件)
复制
1 |
[root@hdss7-11 ~]# /opt/zookeeper/bin/zkServer.sh restart && /opt/zookeeper/bin/zkServer.sh status |
准备资源配置清单(dubbo-monitor)
在运维主机HDSS7-200.host.com上：
vi /data/k8s-yaml/dubbo-monitor/configmap.yaml
复制
1 |
apiVersion: v1 |
vi /data/k8s-yaml/dubbo-monitor/deployment.yaml
复制
1 |
kind: Deployment |
应用资源配置清单
在任意一台k8s运算节点执行：
复制
1 |
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-monitor/configmap.yaml |
重新发版，修改dubbo项目的配置文件
修改项目源代码
-
duboo-demo-service
复制dubbo-server/src/main/java/config.properties1
2dubbo.registry=zookeeper://zk1.od.com:2181
dubbo.port=28080 -
dubbo-demo-web
复制dubbo-client/src/main/java/config.properties1dubbo.registry=zookeeper://zk1.od.com:2181
使用Jenkins进行CI
略
修改/应用资源配置清单
k8s的dashboard上，修改deployment使用的容器版本，提交应用
验证configmap的配置
在K8S的dashboard上，修改dubbo-monitor的configmap配置为不同的zk，重启POD，浏览器打开http://dubbo-monitor.od.com 观察效果
交付Apollo至Kubernetes集群
Apollo简介
Apollo（阿波罗）是携程框架部门研发的分布式配置中心，能够集中化管理应用不同环境、不同集群的配置，配置修改后能够实时推送到应用端，并且具备规范的权限、流程治理等特性，适用于微服务配置管理场景。
官方GitHub地址
Apollo官方地址
官方release包
基础架构
简化模型
交付apollo-configservice
准备软件包
在运维主机HDSS7-200.host.com上：
下载官方release包
复制/opt/src
1 |
[root@hdss7-200 src]# ls -l|grep apollo |
执行数据库脚本
在数据库主机HDSS7-11.host.com上：
注意：MySQL版本应为5.6或以上！
- 更新yum源
复制/etc/yum.repos.d/MariaDB.repo
1 |
[mariadb] |
-
导入GPG-KEY
复制1[root@hdss7-11 ~]# rpm --import https://mirrors.ustc.edu.cn/mariadb/yum/RPM-GPG-KEY-MariaDB -
更新数据库版本
复制1[root@hdss7-11 ~]# yum update MariaDB-server -y
数据库脚本地址
复制
1 |
[root@hdss7-11 ~]# mysql -uroot -p |
数据库用户授权
复制
1 |
mysql> grant INSERT,DELETE,UPDATE,SELECT on ApolloConfigDB.* to "apolloconfig"@"10.4.7.%" identified by "123456"; |
修改初始数据
复制
1 |
mysql> update ApolloConfigDB.ServerConfig set ServerConfig.Value="http://config.od.com/eureka" where ServerConfig.Key="eureka.service.url"; |
制作Docker镜像
在运维主机HDSS7-200.host.com上：
-
配置数据库连接串
复制/data/dockerfile/apollo-configservice1[root@hdss7-200 apollo-configservice]# cat config/application-github.properties -
更新startup.sh
复制/data/dockerfile/apollo-configservice/scripts/startup.sh1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63#!/bin/bash
SERVICE_NAME=apollo-configservice
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-config-server
## Adjust server port if necessary
SERVER_PORT=8080
APOLLO_CONFIG_SERVICE_NAME=$(hostname -i)
SERVER_URL="http://${APOLLO_CONFIG_SERVICE_NAME}:${SERVER_PORT}"
## Adjust memory settings if necessary
#export JAVA_OPTS="-Xms6144m -Xmx6144m -Xss256k -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m -XX:NewSize=4096m -XX:MaxNewSize=4096m -XX:SurvivorRatio=8"
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
export JAVA_OPTS="$JAVA_OPTS -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=9 -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Duser.timezone=Asia/Shanghai -Dclient.encoding.override=UTF-8 -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTS="$JAVA_OPTS -Dserver.port=$SERVER_PORT -Dlogging.file=$LOG_DIR/$SERVICE_NAME.log -XX:HeapDumpPath=$LOG_DIR/HeapDumpOnOutOfMemoryError/"
# Find Java
if [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]]; then
javaexe="$JAVA_HOME/bin/java"
elif type -p java > /dev/null 2>&1; then
javaexe=$(type -p java)
elif [[ -x "/usr/bin/java" ]]; then
javaexe="/usr/bin/java"
else
echo "Unable to find Java"
exit 1
fi
if [[ "$javaexe" ]]; then
version=$("$javaexe" -version 2>&1 | awk -F '"' '/version/ {print $2}')
version=$(echo "$version" | awk -F. '{printf("%03d%03d",$1,$2);}')
# now version is of format 009003 (9.3.x)
if [ $version -ge 011000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 010000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 009000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
else
JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="$JAVA_OPTS -Xloggc:$LOG_DIR/gc.log -XX:+PrintGCDetails"
JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M"
fi
fi
printf "$(date) ==== Starting ==== \n"
cd `dirname $0`/..
chmod 755 $SERVICE_NAME".jar"
./$SERVICE_NAME".jar" start
rc=$?;
if [[ $rc != 0 ]];
then
echo "$(date) Failed to start $SERVICE_NAME.jar, return code: $rc"
exit $rc;
fi
tail -f /dev/null -
写Dockerfile
复制/data/dockerfile/apollo-configservice/Dockerfile1
2
3
4
5
6
7
8
9
10
11
12FROM stanleyws/jre8:8u112
ENV VERSION 1.3.0
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo "Asia/Shanghai" > /etc/timezone
ADD apollo-configservice-${VERSION}.jar /apollo-configservice/apollo-configservice.jar
ADD config/ /apollo-configservice/config
ADD scripts/ /apollo-configservice/scripts
CMD ["/apollo-configservice/scripts/startup.sh"] -
制作镜像并推送
复制1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41[root@hdss7-200 apollo-configservice]# docker build . -t harbor.od.com/infra/apollo-configservice:v1.3.0
Sending build context to Docker daemon 61.91 MB
Step 1 : FROM stanleyws/jre8:8u112
---> fa3a085d6ef1
Step 2 : ENV VERSION 1.3.0
---> [Warning] IPv4 forwarding is disabled. Networking will not work.
---> Running in 685d51b5adb4
---> feb4c0289f04
Removing intermediate container 685d51b5adb4
Step 3 : RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo "Asia/Shanghai" > /etc/timezone
---> [Warning] IPv4 forwarding is disabled. Networking will not work.
---> Running in eaa05073feeb
---> a3e3fd61ae35
Removing intermediate container eaa05073feeb
Step 4 : ADD apollo-configservice-${VERSION}.jar /apollo-configservice/apollo-configservice.jar
---> be09a59b83a2
Removing intermediate container ac6b8af3979b
Step 5 : ADD config/ /apollo-configservice/config
---> fb64fc0f3194
Removing intermediate container b73c5315ad20
Step 6 : ADD scripts/ /apollo-configservice/scripts
---> 96ff3d9b9456
Removing intermediate container 67ba203b3101
Step 7 : CMD /apollo-configservice/scripts/startup.sh
---> [Warning] IPv4 forwarding is disabled. Networking will not work.
---> Running in 80bd3f53fefc
---> 551ea2ba8de3
Removing intermediate container 80bd3f53fefc
Successfully built 551ea2ba8de3
[root@hdss7-200 apollo-configservice]# docker push harbor.od.com/infra/apollo-configservice:v1.3.0
The push refers to a repository [harbor.od.com/infra/apollo-configservice]
25efb9a44683: Pushed
b3572bb46247: Pushed
e7994b936025: Pushed
0ff1d078cbc4: Pushed
ebfb473df5c2: Pushed
aae5c057d1b6: Pushed
dee6aef5c2b6: Pushed
a464c54f93a9: Pushed
v1.3.0: digest: sha256:6a8e4fdda58de0dfba9985ebbf91c4d6f46f5274983d2efa8853b03f4e45fa06 size: 1992
解析域名
DNS主机HDSS7-11.host.com上：
复制/var/named/od.com.zone
1 |
mysql 60 IN A 10.4.7.11 |
准备资源配置清单
在运维主机HDSS7-200.host.com上
复制/data/k8s-yaml
1 |
[root@hdss7-200 k8s-yaml]# mkdir /data/k8s-yaml/apollo-configservice && cd /data/k8s-yaml/apollo-configservice |
vi deployment.yaml
复制
1 |
kind: Deployment |
vi svc.yaml
复制
1 |
kind: Service |
vi ingress.yaml
复制
1 |
kind: Ingress |
vi configmap.yaml
复制
1 |
apiVersion: v1 |
应用资源配置清单
在任意一台k8s运算节点执行：
复制
1 |
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-configservice/configmap.yaml |
浏览器访问
http://config.od.com
交付apollo-adminservice
准备软件包
在运维主机HDSS7-200.host.com上：
下载官方release包
复制
1 |
[root@hdss7-200 src]# ls -l|grep apollo |
制作Docker镜像
在运维主机HDSS7-200.host.com上：
-
配置数据库连接串
复制/data/dockerfile/apollo-adminservice1[root@hdss7-200 apollo-adminservice]# cat config/application-github.properties -
更新starup.sh
复制/data/dockerfile/apollo-adminservice/scripts/startup.sh1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64#!/bin/bash
SERVICE_NAME=apollo-adminservice
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-adminservice
## Adjust server port if necessary
SERVER_PORT=8080
APOLLO_ADMIN_SERVICE_NAME=$(hostname -i)
# SERVER_URL="http://localhost:${SERVER_PORT}"
SERVER_URL="http://${APOLLO_ADMIN_SERVICE_NAME}:${SERVER_PORT}"
## Adjust memory settings if necessary
#export JAVA_OPTS="-Xms2560m -Xmx2560m -Xss256k -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m -XX:NewSize=1536m -XX:MaxNewSize=1536m -XX:SurvivorRatio=8"
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
export JAVA_OPTS="$JAVA_OPTS -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=9 -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Duser.timezone=Asia/Shanghai -Dclient.encoding.override=UTF-8 -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTS="$JAVA_OPTS -Dserver.port=$SERVER_PORT -Dlogging.file=$LOG_DIR/$SERVICE_NAME.log -XX:HeapDumpPath=$LOG_DIR/HeapDumpOnOutOfMemoryError/"
# Find Java
if [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]]; then
javaexe="$JAVA_HOME/bin/java"
elif type -p java > /dev/null 2>&1; then
javaexe=$(type -p java)
elif [[ -x "/usr/bin/java" ]]; then
javaexe="/usr/bin/java"
else
echo "Unable to find Java"
exit 1
fi
if [[ "$javaexe" ]]; then
version=$("$javaexe" -version 2>&1 | awk -F '"' '/version/ {print $2}')
version=$(echo "$version" | awk -F. '{printf("%03d%03d",$1,$2);}')
# now version is of format 009003 (9.3.x)
if [ $version -ge 011000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 010000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 009000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
else
JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="$JAVA_OPTS -Xloggc:$LOG_DIR/gc.log -XX:+PrintGCDetails"
JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M"
fi
fi
printf "$(date) ==== Starting ==== \n"
cd `dirname $0`/..
chmod 755 $SERVICE_NAME".jar"
./$SERVICE_NAME".jar" start
rc=$?;
if [[ $rc != 0 ]];
then
echo "$(date) Failed to start $SERVICE_NAME.jar, return code: $rc"
exit $rc;
fi
tail -f /dev/null -
写Dockerfile
复制/data/dockerfile/apollo-adminservice/Dockerfile1
2
3
4
5
6
7
8
9
10
11
12FROM stanleyws/jre8:8u112
ENV VERSION 1.3.0
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo "Asia/Shanghai" > /etc/timezone
ADD apollo-adminservice-${VERSION}.jar /apollo-adminservice/apollo-adminservice.jar
ADD config/ /apollo-adminservice/config
ADD scripts/ /apollo-adminservice/scripts
CMD ["/apollo-adminservice/scripts/startup.sh"] -
制作镜像并推送
复制1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37[root@hdss7-200 apollo-adminservice]# docker build . -t harbor.od.com/infra/apollo-adminservice:v1.3.0
Sending build context to Docker daemon 58.31 MB
Step 1 : FROM stanleyws/jre8:8u112
---> fa3a085d6ef1
Step 2 : ENV VERSION 1.3.0
---> Using cache
---> feb4c0289f04
Step 3 : RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo "Asia/Shanghai" > /etc/timezone
---> Using cache
---> a3e3fd61ae35
Step 4 : ADD apollo-adminservice-${VERSION}.jar /apollo-adminservice/apollo-adminservice.jar
---> 6a1eb9565777
Removing intermediate container 7196df9af6af
Step 5 : ADD config/ /apollo-adminservice/config
---> 9f364b732d46
Removing intermediate container 9b24669c6c78
Step 6 : ADD scripts/ /apollo-adminservice/scripts
---> b7bc5517b0fc
Removing intermediate container f3e34e759148
Step 7 : CMD /apollo-adminservice/scripts/startup.sh
---> [Warning] IPv4 forwarding is disabled. Networking will not work.
---> Running in 18c6597914b4
---> 82145db3ee88
Removing intermediate container 18c6597914b4
Successfully built 82145db3ee88
[root@hdss7-200 apollo-adminservice]# docker push harbor.od.com/infra/apollo-adminservice:v1.3.0
docker push harbor.od.com/infra/apollo-adminservice:v1.3.0
The push refers to a repository [harbor.od.com/infra/apollo-adminservice]
19b1ca6c066d: Pushed
8fa6cde49908: Pushed
0b2c9b9226cc: Pushed
ebfb473df5c2: Mounted from infra/apollo-configservice
aae5c057d1b6: Mounted from infra/apollo-configservice
dee6aef5c2b6: Mounted from infra/apollo-configservice
a464c54f93a9: Mounted from infra/apollo-configservice
v1.3.0: digest: sha256:75367caab9bad3d0d281eb3324451a0734e84b6aa3ee860e38ad758d7166a7d1 size: 1785
准备资源配置清单
在运维主机HDSS7-200.host.com上
复制/data/k8s-yaml
1 |
[root@hdss7-200 k8s-yaml]# mkdir /data/k8s-yaml/apollo-adminservice && cd /data/k8s-yaml/apollo-adminservice |
vi deployment.yaml
复制
1 |
kind: Deployment |
vi configmap.yaml
复制
1 |
apiVersion: v1 |
应用资源配置清单
在任意一台k8s运算节点执行：
复制
1 |
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-adminservice/configmap.yaml |
浏览器访问
http://config.od.com
交付apollo-portal
准备软件包
在运维主机HDSS7-200.host.com上：
下载官方release包
复制
1 |
[root@hdss7-200 src]# ls -l|grep apollo |
执行数据库脚本
在数据库主机HDSS7-11.host.com上：
数据库脚本地址
复制
1 |
[root@hdss7-11 ~]# mysql -uroot -p |
数据库用户授权
复制
1 |
mysql> grant INSERT,DELETE,UPDATE,SELECT on ApolloPortalDB.* to "apolloportal"@"172.7.%" identified by "123456"; |
制作Docker镜像
在运维主机HDSS7-200.host.com上：
-
配置数据库连接串
复制/data/dockerfile/apollo-portal1[root@hdss7-200 apollo-portal]# cat config/application-github.properties -
配置Portal的meta service
复制/data/dockerfile/apollo-portal/config/apollo-env.properties1dev.meta=http://config.od.com -
更新starup.sh
复制/data/dockerfile/apollo-portal/scripts/startup.sh1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64#!/bin/bash
SERVICE_NAME=apollo-portal
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-portal-server
## Adjust server port if necessary
SERVER_PORT=8080
APOLLO_PORTAL_SERVICE_NAME=$(hostname -i)
# SERVER_URL="http://localhost:$SERVER_PORT"
SERVER_URL="http://${APOLLO_PORTAL_SERVICE_NAME}:${SERVER_PORT}"
## Adjust memory settings if necessary
#export JAVA_OPTS="-Xms2560m -Xmx2560m -Xss256k -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m -XX:NewSize=1536m -XX:MaxNewSize=1536m -XX:SurvivorRatio=8"
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
export JAVA_OPTS="$JAVA_OPTS -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=9 -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Duser.timezone=Asia/Shanghai -Dclient.encoding.override=UTF-8 -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTS="$JAVA_OPTS -Dserver.port=$SERVER_PORT -Dlogging.file=$LOG_DIR/$SERVICE_NAME.log -XX:HeapDumpPath=$LOG_DIR/HeapDumpOnOutOfMemoryError/"
# Find Java
if [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]]; then
javaexe="$JAVA_HOME/bin/java"
elif type -p java > /dev/null 2>&1; then
javaexe=$(type -p java)
elif [[ -x "/usr/bin/java" ]]; then
javaexe="/usr/bin/java"
else
echo "Unable to find Java"
exit 1
fi
if [[ "$javaexe" ]]; then
version=$("$javaexe" -version 2>&1 | awk -F '"' '/version/ {print $2}')
version=$(echo "$version" | awk -F. '{printf("%03d%03d",$1,$2);}')
# now version is of format 009003 (9.3.x)
if [ $version -ge 011000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 010000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 009000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
else
JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="$JAVA_OPTS -Xloggc:$LOG_DIR/gc.log -XX:+PrintGCDetails"
JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M"
fi
fi
printf "$(date) ==== Starting ==== \n"
cd `dirname $0`/..
chmod 755 $SERVICE_NAME".jar"
./$SERVICE_NAME".jar" start
rc=$?;
if [[ $rc != 0 ]];
then
echo "$(date) Failed to start $SERVICE_NAME.jar, return code: $rc"
exit $rc;
fi
tail -f /dev/null -
写Dockerfile
复制/data/dockerfile/apollo-portal/Dockerfile
1 |
FROM stanleyws/jre8:8u112 |
- 制作镜像并推送
复制
1 |
[root@hdss7-200 apollo-portal]# docker build . -t harbor.od.com/infra/apollo-portal:v1.3.0 |
解析域名
DNS主机HDSS7-11.host.com上：
复制/var/named/od.com.zone
1 |
portal 60 IN A 10.4.7.10 |
准备资源配置清单
在运维主机HDSS7-200.host.com上
复制/data/k8s-yaml
1 |
[root@hdss7-200 k8s-yaml]# mkdir /data/k8s-yaml/apollo-portal && cd /data/k8s-yaml/apollo-portal |
vi deployment.yaml
复制
1 |
kind: Deployment |
vi svc.yaml
复制
1 |
kind: Service |
vi ingress.yaml
复制
1 |
kind: Ingress |
vi configmap.yaml
复制
1 |
apiVersion: v1 |
应用资源配置清单
在任意一台k8s运算节点执行：
复制
1 |
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/configmap.yaml |
浏览器访问
http://portal.od.com
- 用户名：apollo
- 密码： admin
实战dubbo微服务接入Apollo配置中心
改造dubbo-demo-service项目
使用IDE拉取项目（这里使用git bash作为范例）
复制
1 |
$ git clone [email protected]/stanleywang/dubbo-demo-service.git |
切到apollo分支
复制
1 |
$ git checkout -b apollo |
修改pom.xml
- 加入apollo客户端jar包的依赖
复制dubbo-server/pom.xml
1 |
|
- 修改resource段
复制dubbo-server/pom.xml
1 |
|
增加resources目录
复制/d/workspace/dubbo-demo-service/dubbo-server/src/main
1 |
$ mkdir -pv resources/META-INF |
修改config.properties文件
复制/d/workspace/dubbo-demo-service/dubbo-server/src/main/resources/config.properties
1 |
dubbo.registry=${dubbo.registry} |
修改srping-config.xml文件
- beans段新增属性
复制/d/workspace/dubbo-demo-service/dubbo-server/src/main/resources/spring-config.xml
1 |
xmlns:apollo="http://www.ctrip.com/schema/apollo" |
- xsi:schemaLocation段内新增属性
复制/d/workspace/dubbo-demo-service/dubbo-server/src/main/resources/spring-config.xml
1 |
http://www.ctrip.com/schema/apollo http://www.ctrip.com/schema/apollo.xsd |
- 新增配置项
复制/d/workspace/dubbo-demo-service/dubbo-server/src/main/resources/spring-config.xml
1 |
|
- 删除配置项（注释）
复制/d/workspace/dubbo-demo-service/dubbo-server/src/main/resources/spring-config.xml
1 |
|
增加app.properties文件
复制/d/workspace/dubbo-demo-service/dubbo-server/src/main/resources/META-INF/app.properties
1 |
app.id=dubbo-demo-service |
提交git中心仓库（gitee）
复制
1 |
$ git push origin apollo |
配置apollo-portal
创建项目
-
部门
样例部门1（TEST1）
-
应用id
dubbo-demo-service
-
应用名称
dubbo服务提供者
-
应用负责人
apollo|apollo
-
项目管理员
apollo|apollo
提交
进入配置页面
新增配置项1
-
Key
dubbo.registry
-
Value
zookeeper://zk1.od.com:2181
-
选择集群
DEV
提交
新增配置项2
-
Key
dubbo.port
-
Value
20880
-
选择集群
DEV
提交
发布配置
点击发布，配置生效
使用jenkins进行CI
略（注意记录镜像的tag）
上线新构建的项目
准备资源配置清单
运维主机HDSS7-200.host.com上：
复制/data/k8s-yaml/dubbo-demo-service/deployment.yaml
1 |
kind: Deployment |
注意：增加了env段配置
注意：docker镜像新版的tag
应用资源配置清单
在任意一台k8s运算节点上执行：
复制
1 |
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-demo-service/deployment.yaml |
观察项目运行情况
http://dubbo-monitor.od.com
改造dubbo-demo-web
略
配置apollo-portal
创建项目
-
部门
样例部门1（TEST1）
-
应用id
dubbo-demo-web
-
应用名称
dubbo服务消费者
-
应用负责人
apollo|apollo
-
项目管理员
apollo|apollo
提交
进入配置页面
新增配置项1
-
Key
dubbo.registry
-
Value
zookeeper://zk1.od.com:2181
-
选择集群
DEV
提交
发布配置
点击发布，配置生效
使用jenkins进行CI
略（注意记录镜像的tag）
上线新构建的项目
准备资源配置清单
运维主机HDSS7-200.host.com上：
复制/data/k8s-yaml/dubbo-demo-consumer/deployment.yaml
1 |
kind: Deployment |
注意：增加了env段配置
注意：docker镜像新版的tag
应用资源配置清单
在任意一台k8s运算节点上执行：
复制
1 |
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-demo-web/deployment.yaml |
通过Apollo配置中心动态维护项目的配置
以dubbo-demo-service项目为例，不用修改代码
- 在http://portal.od.com 里修改dubbo.port配置项
- 重启dubbo-demo-service项目
- 配置生效
实战维护多套dubbo微服务环境
生产实践
- 迭代新需求/修复BUG（编码->提GIT）
- 测试环境发版，测试（应用通过编译打包发布至TEST命名空间）
- 测试通过，上线（应用镜像直接发布至PROD命名空间）
系统架构
- 物理架构
| 主机名 | 角色 | ip |
|---|---|---|
| HDSS7-11.host.com | zk-test(测试环境Test) | 10.4.7.11 |
| HDSS7-12.host.com | zk-prod(生产环境Prod) | 10.4.7.12 |
| HDSS7-21.host.com | kubernetes运算节点 | 10.4.7.21 |
| HDSS7-22.host.com | kubernetes运算节点 | 10.4.7.22 |
| HDSS7-200.host.com | 运维主机，harbor仓库 | 10.4.7.200 |
- K8S内系统架构
| 环境 | 命名空间 | 应用 |
|---|---|---|
| 测试环境（TEST） | test | apollo-config，apollo-admin |
| 测试环境（TEST） | test | dubbo-demo-service，dubbo-demo-web |
| 生产环境（PROD） | prod | apollo-config，apollo-admin |
| 生产环境（PROD） | prod | dubbo-demo-service，dubbo-demo-web |
| ops环境（infra） | infra | jenkins，dubbo-monitor，apollo-portal |
修改/添加域名解析
DNS主机HDSS7-11.host.com上：
复制/var/named/od.com.zone
1 |
zk-test 60 IN A 10.4.7.11 |
Apollo的k8s应用配置
- 删除app命名空间内应用，创建test命名空间，创建prod命名空间
- 删除infra命名空间内apollo-configservice，apollo-adminservice应用
- 数据库内删除ApolloConfigDB，创建ApolloConfigTestDB，创建ApolloConfigProdDB
复制
1 |
mysql> drop database ApolloConfigDB; |
- 准备apollo-config，apollo-admin的资源配置清单（各2套）
注：apollo-config/apollo-admin的configmap配置要点
- Test环境
复制
1 |
application-github.properties: | |
- Prod环境
复制
1 |
application-github.properties: | |
- 依次应用，分别发布在test和prod命名空间
- 修改apollo-portal的configmap并重启portal
复制
1 |
apollo-env.properties: | |
Apollo的portal配置
管理员工具
删除应用、集群、AppNamespace，将已配置应用删除
系统参数
-
Key
apollo.portal.envs
-
Value
TEST,PROD
查询
- Value
TEST,PROD
保存
新建dubbo-demo-service和dubbo-demo-web项目
在TEST/PROD环境分别增加配置项并发布
发布dubbo微服务
- 准备dubbo-demo-service和dubbo-demo-web的资源配置清单（各2套）
- 依次应用，分别发布至app-test和app-prod命名空间
- 使用dubbo-monitor查验
互联网公司技术部的日常
- 产品经理整理需求，需求评审，出产品原型
- 开发同学夜以继日的开发，提测
- 测试同学使用Jenkins持续集成，并发布至测试环境
- 验证功能，通过->待上线or打回->修改代码
- 提交发版申请，运维同学将测试后的包发往生产环境
- 无尽的BUG修复（笑cry）