基于ambari 2.6.2.0的docker镜像安装大数据服务组件
之前有使用docker搭建了一套Apache Hadoop版本的大数据平台，参考我的码云地址：https://gitee.com/elbertmalone/HadoopHACluster，整个编写脚本和搭建过程花了很多时间，且灵活性不好。Ambari在大数据集群部署方面有得天独厚的优势，但是集群操作系统安装准备工作以及基础包的安装还是需要花费很多的时间。为了节省大数据集群的部署时间接下来我们用Docker容器化的方案部署Ambari。
费话少说，放码出来。让我们开始吧！
Ambari的架构
从Ambari的架构主要有两个组件：Ambari Server和Ambari Agent。
Ambari架构图
编写CentOS7支持Systemd镜像Dockfile
FROM centos:7
MAINTAINER elbert.lau
# 2, Setting up systemd
ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME [ "/sys/fs/cgroup" ]
ENTRYPOINT ["/usr/sbin/init"]
RUN yum install -y systemd* && yum clean all
构建ambari-systemd镜像
#!/bin/bash
docker rmi local/ambari-systemd:2.6.2.0-155
docker build --rm -t local/ambari-systemd:2.6.2.0-155 .编写Ambari Repos镜像Dockerfile
FROM local/ambari-systemd:2.6.2.0-155
MAINTAINER elbert.lau
RUN yum -y install httpd; yum clean all; systemctl enable httpd.service
ADD http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.0/ambari-2.6.2.0-centos7.tar.gz /var/www/html
ADD http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/HDP-2.6.5.0-centos7-rpm.tar.gz /var/www/html
ADD http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz /var/www/html/HDP-UTILS
ADD http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.5.0/HDP-GPL-2.6.5.0-centos7-gpl.tar.gz /var/www/html
EXPOSE 80构建ambari-repos镜像
#!/bin/bash
docker rmi local/ambari-repos:2.6.2.0-155
docker build --rm -t local/ambari-repos:2.6.2.0-155 .
编写Ambari Base镜像Dockerfile
FROM local/ambari-systemd:2.6.2.0-155
MAINTAINER elbert.lau
# 1, epel-release install
RUN yum install epel-release -y && yum clean all
RUN yum update -y && yum clean all
RUN yum install -y yum-utils yum-plugin-ovl tar git curl bind-utils unzip wget && yum clean all
# 2, Setup sshd
RUN yum install -y openssh-server openssh-clients && yum clean all
RUN systemctl enable sshd
# 3, kerberos client
RUN yum install -y krb5-workstation && yum clean all
# 4, initscripts (service wrapper for servicectl) is need othewise the Ambari is unable to start postgresql
RUN yum install -y initscripts && yum clean all
RUN curl -o /usr/bin/jq http://stedolan.github.io/jq/download/linux64/jq && chmod +x /usr/bin/jq
ENV JDK_ARTIFACT jdk-8u112-linux-x64.tar.gz
ENV JDK_VERSION jdk1.8.0_112
# 5, install Ambari specified 1.8 jdk
RUN mkdir /usr/jdk64 && cd /usr/jdk64 && wget http://public-repo-1.hortonworks.com/ARTIFACTS/$JDK_ARTIFACT \
&& tar -xf $JDK_ARTIFACT && rm -f $JDK_ARTIFACT
ENV JAVA_HOME /usr/jdk64/$JDK_VERSION
ENV PATH $PATH:$JAVA_HOME/bin
# 6, jce
ADD http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-8.zip $JAVA_HOME/jre/lib/security/
RUN cd $JAVA_HOME/jre/lib/security && unzip jce_policy-8.zip \
&& rm -f jce_policy-8.zip && mv UnlimitedJCEPolicyJDK8/*jar . && rm -rf UnlimitedJCEPolicyJDK8
ADD etc/yum.conf /etc/yum.conf
# 7, for ERROR: SSLError: Failed to connect. Please check openssl library versions.
RUN sed -i 's/verify=platform_default/verify=disable/' /etc/python/cert-verification.cfg构建ambari-base镜像
#!/bin/bash
docker rmi local/ambari-base:2.6.2.0-155
docker build --rm -t local/ambari-base:2.6.2.0-155 .
编写Ambari Server镜像Dockerfile
FROM local/ambari-base:2.6.2.0-155
MAINTAINER elbert.lau
# 1, add ambari repo
ADD http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.0/ambari.repo /etc/yum.repos.d/ambari.repo
ADD http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/hdp.repo /etc/yum.repos.d/hdp.repo
ADD http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.5.0/hdp.gpl.repo /etc/yum.repos.d/hdp.gpl.repo
# 2, initscripts (service wrapper for servicectl) is need othewise the Ambari is unable to start postgresql
RUN yum -y install ambari-server; yum clean all
# 3, add psql connectors to ambari-server so it can be downloaded by services (e.g.: Ranger)
ADD https://jdbc.postgresql.org/download/postgresql-42.2.5.jar /var/lib/ambari-server/resources/postgres-jdbc-driver.jar
RUN echo DefaultEnvironment=\"JAVA_HOME=$JAVA_HOME\" >> /etc/systemd/system.conf
# 4, ambari server setup
RUN ambari-server setup --silent --java-home $JAVA_HOME; systemctl enable ambari-server.service
EXPOSE 8080构建ambari-server镜像
#!/bin/bash
docker rmi local/ambari-server:2.6.2.0-155
docker build --rm -t local/ambari-server:2.6.2.0-155 .
编写Ambari Agent镜像Dockerfile
FROM local/ambari-base:2.6.2.0-155
MAINTAINER elbert.lau
# 1, add ambari repo
ADD http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.0/ambari.repo /etc/yum.repos.d/ambari.repo
ADD http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/hdp.repo /etc/yum.repos.d/hdp.repo
ADD http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.5.0/hdp.gpl.repo /etc/yum.repos.d/hdp.gpl.repo
# 2, install ambari agent
RUN yum -y install ambari-agent; yum clean all
RUN echo DefaultEnvironment=\"JAVA_HOME=$JAVA_HOME\" >> /etc/systemd/system.conf
构建ambari-agent镜像
#!/bin/bash
docker rmi local/ambari-agent:2.6.2.0-155
docker build --rm -t local/ambari-agent:2.6.2.0-155 .
参考资料
【1】https://github.com/sequenceiq/docker-ambari