Kubernetes部署SpringBoot连接外部数据库使用svc模式

这篇文章主要讲解Kubernetes部署SpringBoot的过程，其中主要的难点是用svc名称动态获取数据库IP。网上有一部分文档有说，但是不进行讲解，我在最初部署的时候测试不成功，后来发现主要是Kubernetes内部dns配置的问题。以下文章都是基于centos7二进制安装kubernetes1.12添加证书配置这篇文章搭建的kubernetes的基础上完成的，其中比较重要的是coredns，没有配置这部，网上其他文章无法成功，因为没办法解析域名，下面开始我的流程步骤

一 编写SpringBoot工程

创建过程参见IDEA创建SpringBoot工程我就不描述了，我就把需要注意的代码列出来。

1 编写pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0modelVersion>

    <groupId>com.examplegroupId>
    <artifactId>demoartifactId>
    <version>0.0.1-SNAPSHOTversion>
    <packaging>jarpackaging>

    <name>demoname>
    <description>Demo project for Spring Bootdescription>

    <parent>
        <groupId>org.springframework.bootgroupId>
        <artifactId>spring-boot-starter-parentartifactId>
        <version>2.0.4.RELEASEversion>
        <relativePath/> 
    parent>

    <properties>
        <project.build.sourceEncoding>UTF-8project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8project.reporting.outputEncoding>
        <java.version>1.8java.version>
        <spark.version>2.3.0spark.version>
        <spring-cloud.version>Edgware.RELEASEspring-cloud.version>
        <ojdbc6.version>11.2.0.1.0ojdbc6.version>
    properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.bootgroupId>
            <artifactId>spring-boot-starterartifactId>
        dependency>

        <dependency>
            <groupId>org.springframework.bootgroupId>
            <artifactId>spring-boot-starter-webartifactId>
        dependency>

        <dependency>
            <groupId>com.alibabagroupId>
            <artifactId>druidartifactId>
            <version>1.1.10version>
        dependency>
        <dependency>
            <groupId>mysqlgroupId>
            <artifactId>mysql-connector-javaartifactId>
            <version>8.0.12version>
        dependency>

        <dependency>
            <groupId>com.google.code.gsongroupId>
            <artifactId>gsonartifactId>
            <version>2.8.2version>
        dependency>

        <dependency>
            <groupId>org.springframework.bootgroupId>
            <artifactId>spring-boot-starter-jdbcartifactId>
            <exclusions>
                <exclusion>
                    <groupId>org.apache.tomcatgroupId>
                    <artifactId>tomcat-jdbcartifactId>
                exclusion>
            exclusions>
        dependency>
    dependencies>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloudgroupId>
                <artifactId>spring-cloud-dependenciesartifactId>
                <version>${spring-cloud.version}version>
                <type>pomtype>
                <scope>importscope>
            dependency>
        dependencies>
    dependencyManagement>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.bootgroupId>
                <artifactId>spring-boot-maven-pluginartifactId>
            plugin>
        plugins>
    build>


project>

2 编写datasource

package com.example.demo.datasource;

import com.alibaba.druid.pool.DruidDataSource;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.env.Environment;
import org.springframework.jdbc.core.JdbcTemplate;

import javax.sql.DataSource;

/**
 * Created by arnold.zhu on 6/13/2017.
 */
@Configuration
public class MysqlDataSource {

    private Logger logger = LoggerFactory.getLogger(MysqlDataSource.class);

    @Autowired
    private Environment env;

    @Value("${spring.datasource.mysql.url}")
    private String dbUrl;

    @Value("${spring.datasource.mysql.username}")
    private String username;

    @Value("${spring.datasource.mysql.password}")
    private String password;

    @Value("${spring.datasource.mysql.driver-class-name}")
    private String driverClassName;

    @Value("${spring.datasource.mysql.initialSize}")
    private int initialSize;

    @Value("${spring.datasource.mysql.minIdle}")
    private int minIdle;

    @Value("${spring.datasource.mysql.maxActive}")
    private int maxActive;

    @Value("${spring.datasource.mysql.maxWait}")
    private int maxWait;

    @Value("${spring.datasource.mysql.timeBetweenEvictionRunsMillis}")
    private int timeBetweenEvictionRunsMillis;

    @Value("${spring.datasource.mysql.minEvictableIdleTimeMillis}")
    private int minEvictableIdleTimeMillis;

    @Value("${spring.datasource.mysql.validationQuery}")
    private String validationQuery;

    @Value("${spring.datasource.mysql.testWhileIdle}")
    private boolean testWhileIdle;

    @Value("${spring.datasource.mysql.testOnBorrow}")
    private boolean testOnBorrow;

    @Value("${spring.datasource.mysql.testOnReturn}")
    private boolean testOnReturn;

    @Value("${spring.datasource.mysql.filters}")
    private String filters;

    @Value("${spring.datasource.mysql.logSlowSql}")
    private String logSlowSql;

    @Value("${spring.datasource.mysql.dbType}")
    private String dbType;

    @Bean(name = "mysqlJdbcDataSource")
    @Qualifier("mysqlJdbcDataSource")
    public DataSource dataSource() {
        DruidDataSource datasource = new DruidDataSource();
        System.out.println("-----------------------------------");
        System.out.println(env.getProperty("MYSQL_SERVICE_HOST"));
        System.out.println("-----------------------------------");
        datasource.setUrl(dbUrl.replaceAll("$\\{MYSQL_SERVICE_HOST\\}",env.getProperty("MYSQL_SERVICE_HOST")));
        datasource.setUsername(username);
        datasource.setPassword(password);
        datasource.setDriverClassName(driverClassName);
        datasource.setInitialSize(initialSize);
        datasource.setMinIdle(minIdle);
        datasource.setMaxActive(maxActive);
        datasource.setMaxWait(maxWait);
        datasource.setTimeBetweenEvictionRunsMillis(timeBetweenEvictionRunsMillis);
        datasource.setMinEvictableIdleTimeMillis(minEvictableIdleTimeMillis);
        datasource.setValidationQuery(validationQuery);
        datasource.setTestWhileIdle(testWhileIdle);
        datasource.setTestOnBorrow(testOnBorrow);
        datasource.setTestOnReturn(testOnReturn);
        datasource.setDbType(dbType);
        //此功能不支持hive
//        try {
//            datasource.setFilters(filters);
//        } catch (SQLException e) {
//            logger.error("druid configuration initialization filter", e);
//        }
        return datasource;
    }

    @Bean(name = "mysqlJdbcTemplate")
    public JdbcTemplate mysqlJdbcTemplate(@Qualifier("mysqlJdbcDataSource") DataSource dataSource) {
        return new JdbcTemplate(dataSource);
    }

}

3 编写测试Dao与Impl

package com.example.demo.dao;

public interface MysqlDao {

    public String test();
}

package com.example.demo.dao.impl;

import com.example.demo.dao.MysqlDao;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Repository;

import java.util.Map;

@Repository
public class MysqlDaoImpl implements MysqlDao {

    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Override
    public String test() {
        String sql = "select HOST from db limit 1";
        Map map = jdbcTemplate.queryForMap(sql);
        return map.get("HOST").toString();
    }

}

4 编写Controller

package com.example.demo.controller;

import com.example.demo.dao.MysqlDao;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class HelloController {

    @Autowired
    private MysqlDao mysqlDao;

    @RequestMapping(value = {"/hello"},method= {RequestMethod.GET,RequestMethod.POST})
    public String hello() {
        return mysqlDao.test();
    }
}

5 编写SpringBoot配置文件application.yml

server:
  port: 8080
spring:
  application:
    name: service-hadoop
  datasource:
    mysql:
      type: com.mysql.jdbc.Driver
      url: jdbc:mysql://${MYSQL_SERVICE_HOST}:3306/mysql
      driver-class-name: com.mysql.jdbc.Driver
      username: root
      password: root
      initialSize: 5
      minIdle: 5
      maxActive: 20
      maxWait: 600000
      timeBetweenEvictionRunsMillis: 60000
      minEvictableIdleTimeMillis: 300000
      validationQuery: SELECT 1
      testWhileIdle: true
      testOnBorrow: false
      testOnReturn: false
      filters: stat,wall,log4j #负载监控功能 此功能不支持hive
      logSlowSql: true
      dbType: mysql

注：
1. 在application.yml文件中${MYSQL_SERVICE_HOST}与MysqlDataSource.java中的dbUrl.replaceAll(“$\{MYSQL_SERVICE_HOST\}”,env.getProperty(“MYSQL_SERVICE_HOST”))对应
2. ${MYSQL_SERVICE_HOST}这个的数值是从Kubernetes传入的环境变量，所以用户名与密码也可以采用这种方式进行部署，只需要按照需求修改MysqlDataSource.java即可

datasource.setUsername(username);
datasource.setPassword(password);
改为
datasource.setUsername(env.getProperty("MYSQL_SERVICE_USERNAME"))
datasource.setPassword(env.getProperty("MYSQL_SERVICE_PASSWORD"))
此处的变量名需要与Kubernetes配置文件中的变量名对应

二 编写Kubernetes配置文件

1 mysql服务配置文件

由于mysql服务是外部数据库独立于Kubernetes之外，所以需要将mysql服务引入到Kubernetes之中，利用的是Kubernetes的Endpoints技术。SpringBoot连接数据库时，其实是通过Dns解析Service服务，无论是外部数据库或者是Kubernetes启动的Mysql服务，都是通过Service层进行隔离，两者并没有区别，下面是配置文件。

1.1 mysql-endpoint.yaml

apiVersion: v1
kind: Endpoints
metadata:
  name: mysqljdbc
subsets:
  - addresses:
    - ip: 192.168.200.223
    ports:
    - port: 3306
      protocol: TCP

1.2 mysql-svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: mysqljdbc
spec:
  selector:
    app: mysql
  ports:
  - port: 3306
    targetPort: 3306
    protocol: TCP

2 SpringBoot配置文件

2.1 springboot-rc.yml

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: springboot-demo
  labels:
    app: springboot-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: springboot-demo
  template:
    metadata:
      labels:
        app: springboot-demo
    spec:
      containers:
        - name: springboot-demo
          image: springboot-demo
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
          env:
            - name: MYSQL_SERVICE_HOST
              value: 'mysqljdbc'
            - name: MYSQL_SERVICE_PORT
              value: '3306'

MYSQL_SERVICE_HOST和MYSQL_SERVICE_PORT就是环境变量，与SpringBoot中的配置文件进行对应，只能接收字符串，mysqljdbc其实指代的是机器名，然后通过coredns进行Service名称解析

2.2 springboot-svc.yml

apiVersion: v1
kind: Service
metadata:
  name: springboot-demo
spec:
  type: NodePort
  ports:
    - name: springboot-svc
      port: 8080
      nodePort: 30000
  selector:
    app: springboot-demo

三 启动与验证

1 启动服务

kubectl create -f mysql-endpoint.yaml
kubectl create -f mysql-svc.yaml
kubectl create -f springboot-rc.yml
kubectl create -f springboot-svc.yml

2 验证服务

kubectl get pods

kubectl get svc

访问地址http://192.168.200.223:30000/hello

注：

1. 其实mysql的ip可以写死，譬如我mysql地址是192.168.200.223，在Kubernetes是可以访问到的，写这个ip其实可以访问。
2. mysqljdbc服务会映射出一个ClusterIP，写这个ip也可以访问
3. 写服务名的好处就是配置无需根据你的数据库进行更新，只需要修改以下配置文件就可以了
4. 写ip的方式比较好调通，写服务名的方式主要是由于dns配置问题，可能会出现各类问题假如有问题可以参考一下我的安装文档
5. 假如服务不正常可以通过如下指令进入虚拟手动排查问题

kubectl get pods

kubectl exec -it springboot-demo-76ddcb9f67-stdmj /bin/bash