记录thredds集群踩的坑

1、官方镜像的坑

首先，官方镜像unidata/thredds-docker在我的集群上部署有问题，部署完后可以显示网页，但是修改配置文件catalog.xml无法生效，故制作了自己的镜像：thredds-xyp。
先将Dockerfile和setenv.sh文件放在这：

[root@gst178243 thredds-xyp]# cat Dockerfile
FROM centos:7

RUN yum update -y && yum -y install wget

WORKDIR /usr

#install JDK
RUN mkdir /opt/jdk && wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" -P /opt/jdk http://yilong-common.oss-cn-shanghai.aliyuncs.com/lib/jdk-8u211-linux-x64.tar.gz
RUN tar -zxvf /opt/jdk/jdk-8u211-linux-x64.tar.gz -C /opt/jdk && rm -rf /opt/jdk/jdk-8u211-linux-x64.tar.gz

#install tomcat
RUN mkdir /opt/tomcat && wget -P /opt/tomcat https://mirrors.tuna.tsinghua.edu.cn/apache/tomcat/tomcat-8/v8.5.65/bin/apache-tomcat-8.5.65.tar.gz 
RUN tar -zxvf /opt/tomcat/apache-tomcat-8.5.65.tar.gz -C /opt/tomcat && rm -rf /opt/tomcat/apache-tomcat-8.5.65.tar.gz

#set env
ENV JAVA_HOME /opt/jdk/jdk1.8.0_211
ENV JRE_HOME $JAVA_HOME/jre
ENV CLASSPATH $JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
ENV CATALINA_HOME /opt/tomcat/apache-tomcat-8.5.65
ENV PATH $PATH:$JAVA_HOME/bin:$CATALINA_HOME/bin

#install thredds
RUN yum install -y net-tools
RUN wget -P /opt/tds https://artifacts.unidata.ucar.edu/content/repositories/unidata-releases/edu/ucar/tds/4.6.15/tds-4.6.15.war
RUN mv /opt/tds/tds-4.6.15.war /opt/tomcat/apache-tomcat-8.5.65/webapps/thredds.war
COPY ./setenv.sh /opt/tomcat/apache-tomcat-8.5.65/bin/
WORKDIR /opt/tomcat/apache-tomcat-8.5.65
ENTRYPOINT ./bin/startup.sh && tail -f /dev/null

[root@gst178243 thredds-xyp]# cat setenv.sh
#!/bin/sh
#
# ENVARS for Tomcat
#
export CATALINA_HOME="/opt/tomcat/apache-tomcat-8.5.65"

export CATALINA_BASE="/opt/tomcat/apache-tomcat-8.5.65"

export JAVA_HOME="/opt/jdk/jdk1.8.0_211"

# TDS specific ENVARS
#
# Define where the TDS content directory will live
#   THIS IS CRITICAL and there is NO DEFAULT - the
#   TDS will not start without this.
#
CONTENT_ROOT=-Dtds.content.root.path=/opt/tomcat/apache-tomcat-8.5.65/content

# set java prefs related variables (used by the wms service, for example)
JAVA_PREFS_ROOTS="-Djava.util.prefs.systemRoot=$CATALINA_HOME/content/thredds/javaUtilPrefs \
                  -Djava.util.prefs.userRoot=$CATALINA_HOME/content/thredds/javaUtilPrefs"

#
# Some commonly used JAVA_OPTS settings:
#
NORMAL="-d64 -Xmx4096m -Xms512m -server -ea"
HEAP_DUMP="-XX:+HeapDumpOnOutOfMemoryError"
HEADLESS="-Djava.awt.headless=true"

#
# Standard setup.
#
JAVA_OPTS="$CONTENT_ROOT $NORMAL $MAX_PERM_GEN $HEAP_DUMP $HEADLESS $JAVA_PREFS_ROOTS"

export JAVA_OPTS

2、k8s的坑写在ymal文件里

在安装前确保gluster volume的目录是空的，安装完后修改配置文件要进入到容器里面去修改

[root@gst178243 thredds_docker]# cat thredds-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: thredds
spec:
  selector:
    matchLabels:
      app: thredds
  replicas: 1
  template:
    metadata:
      labels:
        app: thredds
    spec:
      containers:
        - name: thredds
          image: thredds-xyp
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8443
            - containerPort: 8080   # tomcat的端口
          volumeMounts:
            - mountPath: /opt/tomcat/apache-tomcat-8.5.65/content/thredds    # 这里写thredds安装目录
              name: thredds
              subPath: data	# 一定要添加这一行，表示安装在gluster volume的子目录data，要是没有这一行就会出错
      volumes:
        - name: thredds
          persistentVolumeClaim:
            claimName: thredds-pv-claim

[root@gst178243 thredds_docker]# cat thredds-storage.yaml
kind: PersistentVolume
apiVersion: v1
metadata:
  name: thredds-pv-volume
  labels:
    type: local
    app: thredds
spec:
  storageClassName: manual
  capacity:
    storage: 1Ti
  accessModes:
    - ReadWriteMany
  glusterfs:
    endpoints: glusterfs-cluster
    # Gluster volume name, preceded by /
	    path: k8s-thredds	# 这里填你给threads创建的gluster volume的名称
    readOnly: false
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: thredds-pv-claim
  labels:
    app: thredds
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Ti   # 因为放的数据量比较大，要分配大一点的存储

[root@gst178243 thredds_docker]# cat thredds-service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    app: thredds
  name: thredds
  namespace: default
spec:
  selector:
    app: thredds
  externalTrafficPolicy: Cluster
  ports:
  - nodePort: 30005		# 映射到容器外地址的端口
    port: 8080		# 容器里面的端口
    protocol: TCP
    targetPort: 8080
  type: NodePort

3、glusterfs的坑

glusterfs 要按照项目创建volume，另外，部署的应用不要直接部署在volume的根目录，添加一个subPath。目前这个项目创建分布式卷（Distribute）就好了。创建卷的目录最好是空的卷，创建完后mount到一个目录，在thredds部署好后要添加或删除数据从这个mount的目录去增加、修改，它会自动将数据分布到三台服务器。