高可用之——Consul+Consul-template实现HTTP动态负载均衡

转载请注明出处：https://blog.csdn.net/l1028386804/article/details/100568343

一、安装Nginx

Nginx的安装和配置可参考博文《Nginx+Tomcat+Memcached负载均衡集群服务搭建》

二、安装Consul

1.下载Consul

wget https://releases.hashicorp.com/consul/1.6.0/consul_1.6.0_linux_amd64.zip

2.解压Consul

unzip consul_1.6.0_linux_amd64.zip

解压后的Consul是一个可执行文件consul
3.移动可执行文件consul

mkdir -p /usr/local/consul/bin
mv consul /usr/local/consul/bin

4.配置系统环境变量
编辑/etc/profile文件，如下所示。

sudo vim /etc/profile
JAVA_HOME=/usr/local/jdk1.8.0_212
CLASS_PATH=.:$JAVA_HOME/lib
CONSUL_HOME=/usr/local/consul
PATH=$JAVA_HOME/bin:$CONSUL_HOME/bin:$PATH
export JAVA_HOME CONSUL_HOME PATH

输入如下命令是系统环境变量生效。

source /etc/profile

三、启动Consul

在命令行输入如下命令启动Consul。

consul agent -server -bootstrap-expect 1 -data-dir /tmp/consul -bind 0.0.0.0 -client 0.0.0.0 -ui

启动成功后命令行输出如下信息。

BootstrapExpect is set to 1; this is the same as Bootstrap mode.
bootstrap = true: do not enable unless necessary
==> Starting Consul agent...
           Version: 'v1.6.0'
           Node ID: 'dbce6fb7-3b4b-f412-01a7-1303079f6fb5'
         Node name: 'binghe100'
        Datacenter: 'dc1' (Segment: '')
            Server: true (Bootstrap: true)
       Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, gRPC: -1, DNS: 8600)
      Cluster Addr: 192.168.175.100 (LAN: 8301, WAN: 8302)
           Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false, Auto-Encrypt-TLS: false

==> Log data will now stream in as it occurs:

    2019/09/05 16:32:18 [INFO]  raft: Initial configuration (index=1): [{Suffrage:Voter ID:dbce6fb7-3b4b-f412-01a7-1303079f6fb5 Address:192.168.175.100:8300}]
    2019/09/05 16:32:18 [INFO] serf: EventMemberJoin: binghe100.dc1 192.168.175.100
    2019/09/05 16:32:18 [INFO] serf: EventMemberJoin: binghe100 192.168.175.100
    2019/09/05 16:32:18 [INFO] agent: Started DNS server 0.0.0.0:8600 (udp)
    2019/09/05 16:32:18 [INFO]  raft: Node at 192.168.175.100:8300 [Follower] entering Follower state (Leader: "")
    2019/09/05 16:32:18 [WARN] serf: Failed to re-join any previously known node
    2019/09/05 16:32:18 [WARN] serf: Failed to re-join any previously known node
    2019/09/05 16:32:18 [INFO] consul: Adding LAN server binghe100 (Addr: tcp/192.168.175.100:8300) (DC: dc1)
    2019/09/05 16:32:18 [INFO] consul: Handled member-join event for server "binghe100.dc1" in area "wan"
    2019/09/05 16:32:18 [INFO] agent: Started DNS server 0.0.0.0:8600 (tcp)
    2019/09/05 16:32:18 [INFO] agent: Started HTTP server on [::]:8500 (tcp)
    2019/09/05 16:32:18 [INFO] agent: started state syncer
==> Consul agent running!
    2019/09/05 16:32:25 [WARN]  raft: Heartbeat timeout from "" reached, starting election
    2019/09/05 16:32:25 [INFO]  raft: Node at 192.168.175.100:8300 [Candidate] entering Candidate state in term 6
    2019/09/05 16:32:25 [INFO]  raft: Election won. Tally: 1
    2019/09/05 16:32:25 [INFO]  raft: Node at 192.168.175.100:8300 [Leader] entering Leader state
    2019/09/05 16:32:25 [INFO] consul: cluster leadership acquired
    2019/09/05 16:32:25 [INFO] consul: New leader elected: binghe100
    2019/09/05 16:32:25 [INFO] agent: Synced node info

在浏览器中输入链接http://192.168.175.100:8500访问Consul的UI界面，如下图所示。

四、安装Consul-template

1.下载Consul-template
可以到链接https://releases.hashicorp.com/consul-template/ 下载合适的版本，本人下载的是consul-template_0.21.2版本。

wget https://releases.hashicorp.com/consul-template/0.21.2/consul-template_0.21.2_linux_amd64.tgz

2.解压Consul-template

tar -zxvf consul-template-0.21.2.tar.gz

解压出的是一个consul-template可执行文件。
3.移动可执行文件consul-template

mkdir -p /usr/local/consul-template/bin
mv consul-template /usr/local/consul-template/bin

4.配置系统环境变量
编辑/etc/profile文件，如下所示。

sudo vim /etc/profile
JAVA_HOME=/usr/local/jdk1.8.0_212
CLASS_PATH=.:$JAVA_HOME/lib
CONSUL_HOME=/usr/local/consul
CONSUL_TEMPLATE_HOME=/usr/local/consul-template
PATH=$JAVA_HOME/bin:$CONSUL_HOME/bin:$CONSUL_TEMPLATE_HOME/bin:$PATH
export JAVA_HOME CONSUL_HOME CONSUL_TEMPLATE_HOME PATH

输入如下命令是系统环境变量生效。

source /etc/profile

五、编写配置模板

在Consule-template所在的主机上创建配置模板文件binghe_tomcat.ctmpl，如下所示。

mkdir -p /usr/local/nginx_template
vim /usr/local/nginx_template/binghe_tomcat.ctmpl

内容如下：

upstream binghe_tomcat{
	server 127.0.0.1:11111; #占位Server，必须有一个Server，不然无法启动
	{{range service "test.binghe_tomcat@dc1"}}
		server {{.Address}}:{{.Port}} weight=1;
	{{end}}
}

service指定格式为:标签.服务@数据中心，然后通过循环输出Address和Port，生成Nginx upstream。

六、包含配置文件

在Nginx的nginx.conf文件中，包含binghe_tomcat文件，如下所示。

include domains/binghe_tomcat;

七、编写Nginx重启脚本

vim /usr/local/nginx-1.17.2/nginx_restart.sh
#!/bin/bash
ps -fe | grep nginx | grep -v grep
if [ $? -eq 0 ]
then 
  sudo /usr/local/nginx-1.17.2/sbin/nginx
  echo "nginx start"
else
  sudo /usr/local/nginx-1.17.2/sbin/nginx -s reload
  echo "nginx reload"
fi

chmod a+x /usr/local/nginx-1.17.2/nginx_restart.sh

八、启动Consul-template

首先创建目录，如下所示

/usr/local/nginx-1.17.2/conf/domains

接下来启动Consul-template

consul-template -consul-addr 192.168.175.100:8500 -template /usr/local/nginx_template/binghe_tomcat.ctmpl:/usr/local/nginx-1.17.2/conf/domains/binghe_tomcat:"/usr/local/nginx-1.17.2/nginx_restart.sh"

使用consul指定Consul服务器客户端通信地址，template格式为“配置模板:目标配置文件:脚本”，通过配置模板更新目标配置文件，然后调用脚本重启Nginx。

注意：启动consul-template之前，需要保证/usr/local/nginx-1.17.2/conf/domains/目录下没有binghe_tomcat文件，否则启动不成功。

启动Consul-template之后，会在目录/usr/local/nginx-1.17.2/conf/domains/下自动生成binghe_tomcat文件，如下所示。

-bash-4.1$ ll /usr/local/nginx-1.17.2/conf/domains/
total 4
-rw-r--r--. 1 hadoop hadoop 154 Sep  5 21:48 binghe_tomcat

查看文件的内容如下：

-bash-4.1$ cat /usr/local/nginx-1.17.2/conf/domains/binghe_tomcat 
upstream binghe_tomcat{
        server 127.0.0.1:11111; #占位Server，必须有一个Server，不然无法启动
}

九、编写SpringBoot程序

创建Maven项目mykit-ha，并编辑pom.xml文件，如下所示。

	UTF-8
	false
	1.7
	1.1.10
	3.4.6
	1.0.0-SNAPSHOT
	1.1.2
	2.6.6
	1.16.10
	1.3.7


	org.springframework.boot
	spring-boot-starter-parent
	1.5.14.RELEASE


	
		org.springframework.boot
		spring-boot-starter-test
	
	
		org.springframework.boot
		spring-boot-starter-web
		
			
				org.springframework.boot
				spring-boot-starter-tomcat
			
		
	
	
		org.springframework.boot
		spring-boot-starter-undertow
	
	
		org.springframework.boot
		spring-boot-configuration-processor
		true
	
	
		com.orbitz.consul
		consul-client
		${consul.version}
	


	
		
			org.apache.maven.plugins
			maven-compiler-plugin
			3.1
			
				${jdk.version}
				${jdk.version}
				${project.build.sourceEncoding}
			
		
		
			org.apache.maven.plugins
			maven-jar-plugin
			2.4
		
		
			org.apache.maven.plugins
			maven-eclipse-plugin
			
				true
				true
				2.0
			
		
		
			org.apache.maven.plugins
			maven-jar-plugin
			
				target/classes/
				
					
						io.mykit.binghe.ha.MykitHACoreApplication
						
						false
						true
						lib/
					
					
						.
					
				
			
		
		
			org.apache.maven.plugins
			maven-dependency-plugin
			
				
					copy-dependencies
					package
					
						copy-dependencies
					
					
						jar
						jar
						
							${project.build.directory}/lib
						
					
				
			
		
	
	
		
		
			src/main/resources
			${project.build.directory}/classes
			
				**/*
			
			true
		
	

接下来，创建Java类io.mykit.binghe.ha.MykitHACoreApplication，如下所示。

package io.mykit.binghe.ha;
import com.google.common.net.HostAndPort;
import com.orbitz.consul.AgentClient;
import com.orbitz.consul.Consul;
import com.orbitz.consul.model.agent.ImmutableRegistration;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

/**
 * @author binghe
 * @version 1.0.0
 * @description 启动类
 */
@SpringBootApplication
public class MykitHACoreApplication {
    private static final String SERVICE = "binghe_tomcat";
    private static final String TAG = "test";
    public static void main(String[] args){
        SpringApplication.run(MykitHACoreApplication.class);
        //服务注册
        Consul consul = Consul.builder().withHostAndPort(HostAndPort.fromString("192.168.175.100:8500")).build();
        final AgentClient agentClient = consul.agentClient();
        String addrsss = "192.168.175.100";
        int port = 8090;
        final String serviceId = addrsss + ":" + port;
        ImmutableRegistration.Builder builder = ImmutableRegistration.builder();
        builder.id(serviceId).name(SERVICE).address(addrsss).port(port).addTags(TAG);
        agentClient.register(builder.build());

        //JVM停止时移除服务
        Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
            @Override
            public void run() {
                agentClient.deregister(serviceId);
            }
        }));
    }
}

十、打包程序、上传、运行

（1）使用Maven将项目打包成mykit-ha-1.0.0-SNAPSHOT.jar，并将mykit-ha-1.0.0-SNAPSHOT.jar文件和对应的lib目录上传到服务器的/home/hadoop/ha目录下。
（2）运行Jar文件，如下所示。

java -jar /home/hadoop/ha/mykit-ha-1.0.0-SNAPSHOT.jar 

运行成功输出如下信息。

2019-09-05 21:48:50.222  INFO 2395 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
2019-09-05 21:48:50.411  INFO 2395 --- [           main] b.c.e.u.UndertowEmbeddedServletContainer : Undertow started on port(s) 8080 (http)
2019-09-05 21:48:50.428  INFO 2395 --- [           main] i.m.binghe.ha.MykitHACoreApplication     : Started MykitHACoreApplication in 7.064 seconds (JVM running for 8.692)

（3）再次查看binghe_tomcat文件内容如下：

-bash-4.1$ cat /usr/local/nginx-1.17.2/conf/domains/binghe_tomcat 
upstream binghe_tomcat{
        server 127.0.0.1:11111; #占位Server，必须有一个Server，不然无法启动
        server 192.168.175.100:8090 weight=1;
}

发现Consul动态写入了如下所示的一行代码。

server 192.168.175.100:8090 weight=1;

（4）再次在浏览器中输入链接http://192.168.175.100:8500访问Consul的UI界面，如下图所示。

发现多了一个名称为binghe_test的服务分组。