以下内容均来自个人笔记并重新梳理,如有错误欢迎指正!如果对您有帮助,烦请点赞、关注、转发、订阅专栏!欢迎扫码关注个人公众号,不定期推送热点文章!
公众号原文链接:MySQL 源码构建 Docker 镜像(基于 ARM 64 架构)
背景介绍
近期,笔者正推进公司 MySQL 适配 ARM 64 架构工作,由于一直使用 Docker Hub 上的官方镜像,所以第一时间在 Hub 上检索,却发现官方只为 MySQL 8.0 以上版本提供 ARM 64 镜像。
为避免 MySQL 版本变动带来的研发改造成本,笔者最终决定针对 MySQL 5.7.40 版本自行构建 ARM 64 镜像,以下为完整过程。
过程回顾
1、获取 Dockerfile
使用 dfimage 工具获取 MySQL 5.7.40 官方镜像原始的 Dockerfile。
alias dfimage="docker run -v /var/run/docker.sock:/var/run/docker.sock --rm alpine/dfimage"
dfimage -sV=1.36 mysql:5.7.40 > /root/mysql_rebuild/Dockerfile
这里需要说明的是,该 Dockerfile 中是通过 yum 安装 mysql 服务,经过实测其指定的 yum 源也未提供 5.7.40 版本的 ARM 64 rpm文件,因此还需要使用源码自行编译生成 rpm,并修改 Dockerfile 中的安装逻辑。
附 yum 源地址:https://repo.mysql.com/yum
2、编译源码生成 rpm
2.1、在 ARM 环境安装依赖
yum install -y cmake time libaio-devel ncurses-devel numactl-devel openssl-devel zlib-devel \
cyrus-sasl-devel openldap-devel perl-Env gcc gcc-c++ make rpm-build autoconf perl-JSON
2.2、下载源码 rpm 并安装
wget https://repo.mysql.com/yum/mysql-5.7-community/docker/el/7/SRPMS/mysql-community-minimal-5.7.40-1.el7.src.rpm
rpm -ivh mysql-community-minimal-5.7.40-1.el7.src.rpm
2.3、修改源码配置
cd /root/rpmbuild/SOURCES/
tar -xzvf mysql-5.7.40.tar.gz && rm -rf mysql-5.7.40.tar.gz
vim mysq1-5.7.40/sql/mysqld.cc,添加 #include
sed -i "s#-O3#-O1#g" mysql-5.7.40/cmake/build_configurations/compiler_options.cmake
tar -czvf mysql-5.7.40.tar.gz mysql-5.7.40 --remove-files
2.4、编译源码生成 rpm
rpmbuild -bb /root/rpmbuild/SPECS/mysql.spec
生成的 rpm 文件位于 /root/rpmbuild/RPMS/aarch64 目录下
3、构建镜像
cd /root/mysql_rebuild && tree
目录结构如下,部分文件的详细内容见附录:
├── Dockerfile
├── aarch64
│ ├── mysql-community-minimal-debuginfo-5.7.40-1.el7.aarch64.rpm
│ └── mysql-community-server-minimal-5.7.40-1.el7.aarch64.rpm
├── docker-entrypoint.sh
├── gosu
├── gosu.asc
├── my.cnf
└── mysql-shell-8.0.12-1.el7.aarch64.rpm
docker build --pull --platform=linux/arm64 -t mysql:5.7.40-armtest -f Dockerfile . --no-cache
附录
1、Dockerfile
FROM oraclelinux:7-slim
CMD ["/bin/bash"]
RUN set -eux; groupadd --system --gid 999 mysql; useradd --system --uid 999 --gid 999 --home-dir /var/lib/mysql --no-create-home mysql
ENV GOSU_VERSION=1.14
# 此处提前将 gosu 原文件下载好,缩短构建时间
# wget -O gosu.asc https://github.com/tianon/gosu/releases/download/1.14/gosu-arm64.asc
# wget -O gosu https://github.com/tianon/gosu/releases/download/1.14/gosu-arm64
COPY gosu.asc gosu /usr/local/bin
RUN set -eux; export GNUPGHOME="$(mktemp -d)"; gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4; gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc; chmod +x /usr/local/bin/gosu; gosu --version; gosu nobody true; yum install -y --setopt=skip_missing_names_on_install=False oracle-epel-release-el7; yum install -y --setopt=skip_missing_names_on_install=False bzip2 gzip openssl xz zstd; yum clean all
ENV MYSQL_MAJOR=5.7
ENV MYSQL_VERSION=5.7.40-1.el7
# 此处提前将 rpm 原文件下载好,缩短构建时间
COPY aarch64 mysql-shell-8.0.12-1.el7.aarch64.rpm /tmp
COPY my.cnf /etc/my.cnf
RUN set -eux; yum install -y libaio numactl; cd /tmp && rpm -ivh --force mysql-community-minimal-debuginfo-5.7.40-1.el7.aarch64.rpm mysql-community-server-minimal-5.7.40-1.el7.aarch64.rpm; yum clean all; mkdir -p /etc/mysql/conf.d; mkdir -p /etc/mysql/mysql.conf.d; find /etc/my.cnf /etc/mysql/ -name '*.cnf' -print0 | xargs -0 grep -lZE '^(bind-address|log)' | xargs -rt -0 sed -Ei 's/^(bind-address|log)/#&/'; mkdir -p /var/lib/mysql /var/run/mysqld; chown mysql:mysql /var/lib/mysql /var/run/mysqld; chmod 1777 /var/lib/mysql /var/run/mysqld; mkdir /docker-entrypoint-initdb.d; mysqld --version; mysql --version
ENV MYSQL_SHELL_VERSION=8.0.31-1.el7
RUN set -eux; rpm -ivh /tmp/mysql-shell-8.0.12-1.el7.aarch64.rpm; rm -rf /tmp;mkdir /tmp; chmod -R 777 /tmp; yum clean all; mysqlsh --version
# /var/lib/mysql 需要添加引号,否则运行时报错 mount destination [/var/lib/mysql] not absolute: unknown
VOLUME ["/var/lib/mysql"]
COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
RUN ln -s /usr/local/bin/docker-entrypoint.sh /entrypoint.sh # backwards compat
ENTRYPOINT ["docker-entrypoint.sh"]
EXPOSE 3306 33060
CMD ["mysqld"]
2、docker-entrypoint.sh
说明1:EOF 块代码的缩进做了调整,避免报错 warning: here-document at line 161 delimited by end-of-file (wanted `EOF')。
说明2:在原始内容基础上单独添加了 391 行,以解决Kylin V10兼容性问题,有空单独出一篇文讲一下其中的坑。
#!/bin/bash
set -eo pipefail
shopt -s nullglob
# logging functions
mysql_log() {
local type="$1"; shift
# accept argument string or stdin
local text="$*"; if [ "$#" -eq 0 ]; then text="$(cat)"; fi
local dt; dt="$(date --rfc-3339=seconds)"
printf '%s [%s] [Entrypoint]: %s\n' "$dt" "$type" "$text"
}
mysql_note() {
mysql_log Note "$@"
}
mysql_warn() {
mysql_log Warn "$@" >&2
}
mysql_error() {
mysql_log ERROR "$@" >&2
exit 1
}
# usage: file_env VAR [DEFAULT]
# ie: file_env 'XYZ_DB_PASSWORD' 'example'
# (will allow for "$XYZ_DB_PASSWORD_FILE" to fill in the value of
# "$XYZ_DB_PASSWORD" from a file, especially for Docker's secrets feature)
file_env() {
local var="$1"
local fileVar="${var}_FILE"
local def="${2:-}"
if [ "${!var:-}" ] && [ "${!fileVar:-}" ]; then
mysql_error "Both $var and $fileVar are set (but are exclusive)"
fi
local val="$def"
if [ "${!var:-}" ]; then
val="${!var}"
elif [ "${!fileVar:-}" ]; then
val="$(< "${!fileVar}")"
fi
export "$var"="$val"
unset "$fileVar"
}
# check to see if this file is being run or sourced from another script
_is_sourced() {
# https://unix.stackexchange.com/a/215279
[ "${#FUNCNAME[@]}" -ge 2 ] \
&& [ "${FUNCNAME[0]}" = '_is_sourced' ] \
&& [ "${FUNCNAME[1]}" = 'source' ]
}
# usage: docker_process_init_files [file [file [...]]]
# ie: docker_process_init_files /always-initdb.d/*
# process initializer files, based on file extensions
docker_process_init_files() {
# mysql here for backwards compatibility "${mysql[@]}"
mysql=( docker_process_sql )
echo
local f
for f; do
case "$f" in
*.sh)
# https://github.com/docker-library/postgres/issues/450#issuecomment-393167936
# https://github.com/docker-library/postgres/pull/452
if [ -x "$f" ]; then
mysql_note "$0: running $f"
"$f"
else
mysql_note "$0: sourcing $f"
. "$f"
fi
;;
*.sql) mysql_note "$0: running $f"; docker_process_sql < "$f"; echo ;;
*.sql.bz2) mysql_note "$0: running $f"; bunzip2 -c "$f" | docker_process_sql; echo ;;
*.sql.gz) mysql_note "$0: running $f"; gunzip -c "$f" | docker_process_sql; echo ;;
*.sql.xz) mysql_note "$0: running $f"; xzcat "$f" | docker_process_sql; echo ;;
*.sql.zst) mysql_note "$0: running $f"; zstd -dc "$f" | docker_process_sql; echo ;;
*) mysql_warn "$0: ignoring $f" ;;
esac
echo
done
}
# arguments necessary to run "mysqld --verbose --help" successfully (used for testing configuration validity and for extracting default/configured values)
_verboseHelpArgs=(
--verbose --help
--log-bin-index="$(mktemp -u)" # https://github.com/docker-library/mysql/issues/136
)
mysql_check_config() {
local toRun=( "$@" "${_verboseHelpArgs[@]}" ) errors
if ! errors="$("${toRun[@]}" 2>&1 >/dev/null)"; then
mysql_error $'mysqld failed while attempting to check config\n\tcommand was: '"${toRun[*]}"$'\n\t'"$errors"
fi
}
# Fetch value from server config
# We use mysqld --verbose --help instead of my_print_defaults because the
# latter only show values present in config files, and not server defaults
mysql_get_config() {
local conf="$1"; shift
"$@" "${_verboseHelpArgs[@]}" 2>/dev/null \
| awk -v conf="$conf" '$1 == conf && /^[^ \t]/ { sub(/^[^ \t]+[ \t]+/, ""); print; exit }'
# match "datadir /some/path with/spaces in/it here" but not "--xyz=abc\n datadir (xyz)"
}
# Ensure that the package default socket can also be used
# since rpm packages are compiled with a different socket location
# and "mysqlsh --mysql" doesn't read the [client] config
# related to https://github.com/docker-library/mysql/issues/829
mysql_socket_fix() {
local defaultSocket
defaultSocket="$(mysql_get_config 'socket' mysqld --no-defaults)"
if [ "$defaultSocket" != "$SOCKET" ]; then
ln -sfTv "$SOCKET" "$defaultSocket" || :
fi
}
# Do a temporary startup of the MySQL server, for init purposes
docker_temp_server_start() {
if [ "${MYSQL_MAJOR}" = '5.7' ]; then
"$@" --skip-networking --default-time-zone=SYSTEM --socket="${SOCKET}" &
mysql_note "Waiting for server startup"
local i
for i in {30..0}; do
# only use the root password if the database has already been initialized
# so that it won't try to fill in a password file when it hasn't been set yet
extraArgs=()
if [ -z "$DATABASE_ALREADY_EXISTS" ]; then
extraArgs+=( '--dont-use-mysql-root-password' )
fi
if docker_process_sql "${extraArgs[@]}" --database=mysql <<<'SELECT 1' &> /dev/null; then
break
fi
sleep 1
done
if [ "$i" = 0 ]; then
mysql_error "Unable to start server."
fi
else
# For 5.7+ the server is ready for use as soon as startup command unblocks
if ! "$@" --daemonize --skip-networking --default-time-zone=SYSTEM --socket="${SOCKET}"; then
mysql_error "Unable to start server."
fi
fi
}
# Stop the server. When using a local socket file mysqladmin will block until
# the shutdown is complete.
docker_temp_server_stop() {
if ! mysqladmin --defaults-extra-file=<( _mysql_passfile ) shutdown -uroot --socket="${SOCKET}"; then
mysql_error "Unable to shut down server."
fi
}
# Verify that the minimally required password settings are set for new databases.
docker_verify_minimum_env() {
if [ -z "$MYSQL_ROOT_PASSWORD" -a -z "$MYSQL_ALLOW_EMPTY_PASSWORD" -a -z "$MYSQL_RANDOM_ROOT_PASSWORD" ]; then
mysql_error </dev/null && ulimit -n
mysql_check_config "$@"
# Load various environment variables
docker_setup_env "$@"
docker_create_db_directories "$@"
# If container is started as root user, restart as dedicated mysql user
if [ "$(id -u)" = "0" ]; then
mysql_note "Switching to dedicated user 'mysql'"
exec gosu mysql "$BASH_SOURCE" "$@"
fi
# there's no database, so it needs to be initialized
if [ -z "$DATABASE_ALREADY_EXISTS" ]; then
docker_verify_minimum_env
# check dir permissions to reduce likelihood of half-initialized database
ls /docker-entrypoint-initdb.d/ > /dev/null
docker_init_database_dir "$@"
mysql_note "Starting temporary server"
docker_temp_server_start "$@"
mysql_note "Temporary server started."
mysql_socket_fix
docker_setup_db
docker_process_init_files /docker-entrypoint-initdb.d/*
mysql_expire_root_user
mysql_note "Stopping temporary server"
docker_temp_server_stop
mysql_note "Temporary server stopped"
echo
mysql_note "MySQL init process done. Ready for start up."
echo
else
mysql_socket_fix
fi
fi
exec "$@"
}
# If we are sourced from elsewhere, don't perform any further actions
if ! _is_sourced; then
_main "$@"
fi
3、my.cnf
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html
[mysqld]
#
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
# innodb_buffer_pool_size = 128M
#
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
#
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M
skip-host-cache
skip-name-resolve
datadir=/var/lib/mysql
socket=/var/run/mysqld/mysqld.sock
secure-file-priv=/var/lib/mysql-files
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
#log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
[client]
socket=/var/run/mysqld/mysqld.sock
!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/mysql.conf.d/