腾讯云数据库TDSQL-单机安装体验&安装脚本分析

目录

一、机器配置

二、部署规划

三、修改tdsql_hosts文件

四、修改ansible变量group_vars/all文件

五、安装ansible

六、安装part1_site.yml

七、安装part1_site.yml

八、安装hdfs

九、安装kafka

十、安装consumer

十一、安装后的资源使用截图

十二、创建分布式实例

总结

---

前言

      根据腾讯云官方要求，TDSQL安装至少需要三台机器。本人想要学习TDSQL但是没有足够的机器资源，经过仔细分析研究TDSQL安装过程，尝试发现单台机器也可以成功安装TDSQL，现将本人单台机器安装TDSQL的过程以及安装过程中遇到的问题在此分享。具体安装过程可参考腾讯官方文档，本文仅分享与官方文档不一致的步骤。

一、机器配置

本次尝试在win10笔记本电脑安装vmware虚拟机，虚拟机安装centos操作系统，centos安装tdsql

win10笔记本电脑配置：

处理器：Intel(R) Core(TM) i5-8265U CPU @ 1.60GHz   1.80 GHz

内存：8.00 GB (7.78 GB 可用)

虚拟机配置：处理器：2颗CPU 内存：6G 硬盘：80G

虚拟机操作系统：centos 7.6

二、部署规划

	192.168.92.132
zk	Y
scheduler	Y
oss	Y
chitu	Y
monitor（采集监控）	Y
db	Y
proxy	Y
hdfs	Y
kafka	Y
consumer	Y

三、修改tdsql_hosts文件

# 填入所有机器的地址（包括主控机）
# 用于环境检测和初始化配置
# 同一个标签组内的ip地址不要重复，一个ip一行
# 序号从1递增，比如只有2台机器，那么序号保留tdsql_mac1 tdsql_mac2
# 后续如果有扩容，新扩容的机器也需要在这里补上
[tdsql_allmacforcheck]
tdsql_mac1 ansible_ssh_host=192.168.92.132



# zookeeper机器ip地址
# 数量只能是1、3、5， 正式环境建议3台或者5台
# 如果是使用自己已有的zk集群，这里同样要填写正确的zk地址
[tdsql_zk]
tdsql_zk1 ansible_ssh_host=192.168.92.132



# scheduler机器的ip地址，正式环境建议2个
[tdsql_scheduler]
tdsql_scheduler1 ansible_ssh_host=192.168.92.132


# oss机器的ip地址， 正式环境建议2个
[tdsql_oss]
tdsql_oss1 ansible_ssh_host=192.168.92.132


# 赤兔机器的ip地址， 正式环境建议2个
[tdsql_chitu]
tdsql_chitu1 ansible_ssh_host=192.168.92.132


# 监控采集模块的IP地址，正式环境建议2个
[tdsql_monitor]
tdsql_monitor1 ansible_ssh_host=192.168.92.132


# db机器地址
# 有多少台就填多少个ip地址
# 注意tdsql_db序号逐个递增，不要跳跃
[tdsql_db]
tdsql_db1 ansible_ssh_host=192.168.92.132


# proxy机器地址
[tdsql_proxy]
tdsql_proxy1 ansible_ssh_host=192.168.92.132


# hdfs机器地址
# 机器数量只能是1台或者3台
# 如果单节点部署，则只保留tdsql_hdfs1，其他删除
# 正式环境建议高可用部署，初始安装数量只能3台
[tdsql_hdfs]
tdsql_hdfs1 ansible_ssh_host=192.168.92.132



# lvs机器地址，数量固定2台
[tdsql_lvs]
tdsql_lvs1 ansible_ssh_host=192.168.92.132


# kafka机器地址，数量固定3台
[tdsql_kafka]
tdsql_kafka1 ansible_ssh_host=192.168.92.132



# 多源同步消费者服务ip地址，数量固定1台
[tdsql_consumer]
tdsql_consumer1 ansible_ssh_host=192.168.92.132


# es机器地址
[tdsql_es]
tdsql_es1 ansible_ssh_host=192.168.92.132


[tdsql_mc]
tdsql_mc1 ansible_ssh_host=192.168.92.132





[tdsql_newdb]
tdsql_newdb1 ansible_ssh_host=192.168.92.132


 
[tdsql_ansible_test]
tdsql_ansible_test1 ansible_ssh_host=192.168.92.132

四、修改ansible变量group_vars/all文件

---
# scheduler,oss机器网卡
tdsql_sche_netif: ens33

# 操作系统账号tdsql的明文密码
# 如果有规划要部署两个集群做DCN同步, 则这两个集群的tdsql密码要一致
tdsql_os_pass: a+complex+password

# tdsql在zk上的根路径, 保持默认不允许修改
tdsql_zk_rootdir: /tdsqlzk

# zk机器的域名配置, 会写入各配置文件, 并将域名配置到/etc/hosts中
# 正式环境必须用机房或者地区的关键字, 有意义的关键字来命名
# 如果部署多套TDSQL集群, 则名字需要唯一
# 例如: 深圳机房zk的域名可以定义为tdsql_sz_zk
tdsql_zk_domain_name: tdsql_test_zk

# zk端口配置, 保持默认不要改,如果是自建的zk, 则和已有zk端口保持一致
tdsql_zk_clientport: 2118
tdsql_zk_serverport1: 2338
tdsql_zk_serverport2: 2558


# 赤兔监控库配置, 赤兔初始化完成后需要将监控库信息在这里更新
tdsql_metadb_ip: 192.168.92.132
tdsql_metadb_port: 15001
tdsql_metadb_ip_bak: 192.168.92.132
tdsql_metadb_port_bak: 15001
tdsql_metadb_user: hanlon
tdsql_metadb_password: 123456


# hdfs机器的ssh端口
tdsql_hdfs_ssh: 22

# hdfs数据目录, 正式环境要求mount挂载比较大的数据盘
tdsql_hdfs_datadir: /data2/hdfs,/data3/hdfs,/data4/hdfs


# kafka日志目录，正式环境要求mount挂载比较大的数据盘
tdsql_kafka_logdir: /data2/kafka,/data3/kafka,/data4/kafka

# 多源同步消费服务的机器网卡
tdsql_consumer_netif: ens33


# es7配置
tdsql_es7_mem: 4
tdsql_es7_base_path: /data1/es
tdsql_helper_cluster_name: tdsql


# 一致性读MC机器的网卡, 需要安装MC时配置
tdsql_mc_netif: ens33


update_tdsqlinstall_packet: mysqlagent

五、安装ansible

在tdsql_install 目录下执行
source scripts/environment_set
sh scripts/install_ansible.sh
验证ansible安装是否成功
ansible --version

六、安装part1_site.yml

ansible-playbook -i tdsql_hosts playbooks/tdsql_part1_site.yml

该过程安装不是很顺利，首次安装遭遇如下报错提示：

 根据报错信息检查发现：/usr/local/php/etc/php-fpm.d目录不存在，

报错环节：TASK[tdsql_chitu:start_php]

进一步分析tdsql_part1_site.yml安装脚本，理清安装过程如下：

1.tdsql_part1_site.yml调用roles/tdsql_chitu/tasks/main.yml

- name: install chitu server
  hosts: tdsql_chitu
  remote_user: root
  gather_facts: false
  roles:
    - tdsql_chitu
  tags: chitu

2.roles/tdsql_chitu/tasks/main.yml包含了报错信息中执行的脚本

######## install php ########
- name: copy the tdsql_php file to the dst host
  synchronize:
    src: "{{ playbook_dir }}/../../group_files/tdsql_php/"
    dest: /data/tools/

- name: copy install_php.sh
  synchronize:
    src: "../files/shell_scripts/install_php.sh"
    dest: /data/tools/

- name: copy the php conf
  synchronize:
    src: "../templates/{{ item }}"
    dest: /data/tools/
  with_items: ['www.conf', 'php.ini', 'php-fpm.conf']

- name: compile the php
  shell: "/bin/bash /data/tools/install_php.sh"

- name: add the x privi
  shell: "chmod +x -R  /usr/local/php && chown -R nginx:nginx /usr/local/php"

- name: add the nginx privi
  shell: "chown -R nginx:nginx /data/website"

- name: start php
  shell: "/usr/local/php/sbin/php-fpm -y /usr/local/php/etc/php-fpm.conf -c /usr/local/php/lib/php.ini"
  become: true
  become_user: nginx
######## install php ########

该脚本主要完成了以下几个工作：

（1）拷贝php安装包到/data/tools/目录

（2）拷贝install_php.sh到/data/tools/目录

（3）拷贝php conf文件到/data/tools/目录

（4）执行install_php.sh脚本

（5）添加执行权限，修改归属用户和组

（6）start php

其中start php即为报错信息中执行的脚本，根据报错信息启动php需要在/usr/local/php/etc/php-fpm.d目录找配置文件，但是以上脚本将配置文件拷贝到了/data/tools目录，然后执行了install_php.sh脚本，所以有可能改变配置文件的脚本只可能是install_php.sh脚本

3.查看install_php.sh脚本

#!/usr/bin/env bash

pkill php-fpm

# 如果php-fpm存在则不编译
#test -e /usr/local/php/sbin/php-fpm && exit 0

# 安装PHP
cd /data/tools
test -e php || tar xf php.tar.gz
cd php

export PATH=/usr/bin:${PATH}

# x86
uname -i | grep x86_64 && ./configure --prefix=/usr/local/php \
--enable-fpm \
--with-fpm-user=nginx \
--with-fpm-group=nginx \
--with-libxml-dir \
--with-gd \
--with-jpeg-dir \
--with-png-dir \
--with-zlib-dir \
--enable-gd-jis-conv \
--enable-mysqlnd \
--enable-mbstring \
--with-mysqli=mysqlnd \
--with-pdo-mysql=mysqlnd \
--enable-sockets \
--enable-soap \
--with-curl \
--with-openssl

# arm
uname -i | grep aarch64 && ./configure --build=arm --prefix=/usr/local/php \
--enable-fpm \
--with-fpm-user=nginx \
--with-fpm-group=nginx \
--with-libxml-dir \
--with-gd \
--with-jpeg-dir \
--with-png-dir \
--with-zlib-dir \
--enable-gd-native-ttf \
--enable-gd-jis-conv \
--enable-mysqlnd \
--enable-mbstring \
--with-mysql=mysqlnd \
--with-mysqli=mysqlnd \
--with-pdo-mysql=mysqlnd \
--enable-sockets \
--enable-soap \
--with-curl=/usr/local/curl \
--with-mcrypt \
--with-openssl

make -j 4 && make install

cd /data/tools
/bin/cp -a php.ini  /usr/local/php/lib/php.ini
/bin/cp -a php-fpm.conf  /usr/local/php/etc/php-fpm.conf
/bin/cp -a www.conf  /usr/local/php/etc/php-fpm.d/www.conf

mkdir -p /data/website/logs

#启动php-fpm
#/usr/local/php/sbin/php-fpm -y /usr/local/php/etc/php-fpm.conf -c /usr/local/php/lib/php.ini
You have new mail in /var/spool/mail/root

该脚本第7行即完成了/usr/local/php/etc/php-fpm.d目录中配置文件的拷贝

经尝试重新执行install_php.sh脚本后/usr/local/php/etc/php-fpm.d目录中产生了www.conf配置文件

再次尝试执行安装part1_site.yml发现之前的报错已经消失，但是产生了新的报错信息：

fatal: [tdsql_chitu1]: FAILED! => {"changed": true, "cmd": "/usr/local/php/sbin/php-fpm -y /usr/local/php/etc/php-fpm.conf -c /usr/local/php/lib/php.ini", "delta": "0:00:00.130908", "end": "2022-05-07 23:42:32.457796", "msg": "non-zero return code", "rc": 70, "start": "2022-05-07 23:42:32.326888", "stderr": "[07-May-2022 23:42:32] NOTICE: [pool www] 'user' directive is ignored when FPM is not running as root\n[07-May-2022 23:42:32] NOTICE: [pool www] 'group' directive is ignored when FPM is not running as root\n[07-May-2022 23:42:32] ERROR: unable to bind listening socket for address '127.0.0.1:9000': Address already in use (98)\n[07-May-2022 23:42:32] ERROR: FPM initialization failed", "stderr_lines": ["[07-May-2022 23:42:32] NOTICE: [pool www] 'user' directive is ignored when FPM is not running as root", "[07-May-2022 23:42:32] NOTICE: [pool www] 'group' directive is ignored when FPM is not running as root", "[07-May-2022 23:42:32] ERROR: unable to bind listening socket for address '127.0.0.1:9000': Address already in use (98)", "[07-May-2022 23:42:32] ERROR: FPM initialization failed"], "stdout": "", "stdout_lines": []}

分析报错信息发现报错原因是127.0.0.1:9000这个端口服务已经存在

使用netstat命令查看该端口服务启动的进程

[root@tdsql01 php-fpm.d]# netstat -anp | grep 9000
tcp        0      0 127.0.0.1:9000          0.0.0.0:*               LISTEN      70994/php-fpm: mast 
You have new mail in /var/spool/mail/root
[root@tdsql01 php-fpm.d]# ps -ef|grep 70994
root     70994 70820  0 23:48 ?        00:00:00 php-fpm: master process (/usr/local/php/etc/php-fpm.conf)
nginx    71006 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71007 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71008 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71009 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71010 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71011 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71012 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71013 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71014 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71015 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71016 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71017 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71018 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71019 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71020 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71021 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71022 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71023 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71024 70994  0 23:48 ?        00:00:00 php-fpm: pool www
nginx    71025 70994  0 23:48 ?        00:00:00 php-fpm: pool www
root     73778  1170  0 23:58 pts/8    00:00:00 grep --color=auto 70994
[root@tdsql01 php-fpm.d]# ps -ef|grep 70820
root     70820     1  0 23:48 ?        00:00:00 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
root     70836 70820  0 23:48 ?        00:00:00 nginx: master process /usr/local/nginx/sbin/nginx -g daemon off; -c /usr/local/nginx/conf/nginx.conf
root     70994 70820  0 23:48 ?        00:00:00 php-fpm: master process (/usr/local/php/etc/php-fpm.conf)
root     73796  1170  0 23:58 pts/8    00:00:00 grep --color=auto 70820

查看/etc/supervisord.conf配置文件，最后两行包含以下内容

[include]
files = /etc/supervisord.d/*.conf

查看查看/etc/supervisord.d目录下的文件，包含supervisord_php.conf脚本，内容如下：

; TDSQL supervisor configure file for php-fpm


[program:php-fpm]
command=/usr/local/php/sbin/php-fpm -y /usr/local/php/etc/php-fpm.conf --nodaemonize -c /usr/local/php/lib/php.ini
user=root
autostart=true
autorestart=true
startsecs=3
startretries=100
stdout_logfile=/var/log/php-fpm_stdout.log
stdout_logfile_maxbytes=1MB
stdout_logfile_backups=2
stderr_logfile=/var/log/php-fpm_stderr.log
stderr_logfile_maxbytes=1MB
stderr_logfile_backups=2

该脚本包含了启动php-fpm的脚本

Supervisor 是一个用 Python 写的进程管理工具，可以很方便的用来在 UNIX-like 系统（不支持 Windows）下启动、重启（自动重启程序）、关闭进程。根据以上内容分析，tdsql安装脚本在/etc/supervisord.d目录放置了启动php-fpm的脚本，用于自动启动php-fpm进程，再次安装报错是因为php-fpm进程被Supervisor进程自动拉起。删除掉该目录下的文件后再次尝试执行安装part1_site.yml，未报错，安装成功。安装成功后显示信息如下：

2022-05-08 00:17:42,588 p=34880 u=root n=ansible | tdsql_chitu1               : ok=39   changed=26   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
2022-05-08 00:17:42,588 p=34880 u=root n=ansible | tdsql_db1                  : ok=9    changed=9    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
2022-05-08 00:17:42,588 p=34880 u=root n=ansible | tdsql_mac1                 : ok=61   changed=41   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
2022-05-08 00:17:42,588 p=34880 u=root n=ansible | tdsql_oss1                 : ok=15   changed=12   unreachable=0    failed=0    skipped=1    rescued=0    ignored=0   
2022-05-08 00:17:42,590 p=34880 u=root n=ansible | tdsql_proxy1               : ok=9    changed=8    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
2022-05-08 00:17:42,590 p=34880 u=root n=ansible | tdsql_scheduler1           : ok=30   changed=29   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
2022-05-08 00:17:42,590 p=34880 u=root n=ansible | tdsql_zk1                  : ok=13   changed=8    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0  

之后按照腾讯官方提供的安装步骤执行，未产生新的报错。

七、安装part1_site.yml

安装成功显示信息如下：

2022-05-08 00:36:52,749 p=23280 u=root n=ansible | tdsql_chitu1               : ok=18   changed=15   unreachable=0    failed=0    skipped=4    rescued=0    ignored=0   
2022-05-08 00:36:52,749 p=23280 u=root n=ansible | tdsql_mac1                 : ok=46   changed=32   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
2022-05-08 00:36:52,749 p=23280 u=root n=ansible | tdsql_monitor1             : ok=13   changed=13   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
2022-05-08 00:36:52,750 p=23280 u=root n=ansible | tdsql_oss1                 : ok=15   changed=10   unreachable=0    failed=0    skipped=1    rescued=0    ignored=0   
2022-05-08 00:36:52,750 p=23280 u=root n=ansible | tdsql_scheduler1           : ok=38   changed=35   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0 

八、安装hdfs

安装成功显示信息如下：

2022-05-08 21:45:19,518 p=42548 u=root n=ansible | tdsql_hdfs1                : ok=12   changed=12   unreachable=0    failed=0    skipped=1    rescued=0    ignored=0   
2022-05-08 21:45:19,519 p=42548 u=root n=ansible | tdsql_mac1                 : ok=58   changed=42   unreachable=0    failed=0    skipped=3    rescued=0    ignored=0   

九、安装kafka

安装报错信息如下：

2022-05-08 21:52:01,006 p=67751 u=root n=ansible | fatal: [tdsql_kafka1]: UNREACHABLE! => {"changed": true, "msg": "All items completed", "results": [{"ansible_loop_var": "item", "changed":
 true, "failed": false, "invocation": {"module_args": {"after": null, "attributes": null, "backup": false, "before": null, "content": null, "delimiter": null, "dest": "/data/application/kaf
ka/config/server.properties", "directory_mode": null, "encoding": "utf-8", "follow": false, "force": null, "group": null, "mode": null, "owner": null, "path": "/data/application/kafka/confi
g/server.properties", "regexp": "wait_for_change_brokerid", "remote_src": null, "replace": "1", "selevel": null, "serole": null, "setype": null, "seuser": null, "src": null, "unsafe_writes"
: null, "validate": null}}, "item": 1, "msg": "1 replacements made"}, {"ansible_loop_var": "item", "item": 2, "msg": "Failed to connect to the host via ssh: ssh: Could not resolve hostname 
tdsql_kafka2: Name or service not known", "unreachable": true}, {"ansible_loop_var": "item", "item": 3, "msg": "Failed to connect to the host via ssh: ssh: Could not resolve hostname tdsql_
kafka3: Name or service not known", "unreachable": true}], "warnings": ["The value 1 (type int) in a string field was converted to u'1' (type string). If this does not look like what you ex
pect, quote the entire value to ensure it does not change."]}
2022-05-08 21:52:01,006 p=67751 u=root n=ansible | NO MORE HOSTS LEFT ***********************************************************************************************************************
***************************************************
2022-05-08 21:52:01,007 p=67751 u=root n=ansible | PLAY RECAP *******************************************************************************************************************************
***************************************************
2022-05-08 21:52:01,007 p=67751 u=root n=ansible | tdsql_kafka1               : ok=51   changed=36   unreachable=1    failed=0    skipped=0    rescued=0    ignored=0 

错误信息提取关键内容：Could not resolve hostname tdsql_kafka2

分析原因：安装脚本要求kafka必须三台机器，本人只有一台机器，所以需要修改kafka安装脚本

查看kafka安装脚本playbooks/tdsql_kafka.yml，内容如下：

---
- name: install kafka beginning
  hosts: tdsql_kafka
  remote_user: root
  gather_facts: false
  roles:
    - tdsql_beginning

- name: install kafka
  hosts: tdsql_kafka
  remote_user: root
  gather_facts: true
  roles:
    - tdsql_kafka

查看roles/tdsql_kafka/tasks/main.yml包含以下关键内容

- name: set brokerid for kafka
  replace:
    dest: /data/application/kafka/config/server.properties
    regexp: 'wait_for_change_brokerid'
    replace: "{{ item }}"
  run_once: true
  delegate_to: tdsql_kafka{{ item }}
  with_items: [1, 2, 3]

这里指定了kafka安装主机需要三台，修改为1台，修改后的内容如下：

- name: set brokerid for kafka
  replace:
    dest: /data/application/kafka/config/server.properties
    regexp: 'wait_for_change_brokerid'
    replace: "{{ item }}"
  run_once: true
  delegate_to: tdsql_kafka{{ item }}
  with_items: [1]

再次执行安装脚本，提示安装成功

tdsql_kafka1               : ok=57   changed=40   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

根据安装文档查看kafka启动信息，发现/data/application/kafka/logs目录下没有server.log文件

[root@tdsql01 tdsql_install]# cd /data/application/kafka/logs
[root@tdsql01 logs]# ls
kafkaServer-gc.log  kafkaServer.out
[root@tdsql01 logs]# more kafkaServer-gc.log
OpenJDK 64-Bit Server VM (25.242-b1) for linux-amd64 JRE (1.8.0_242-b1), built on Feb 19 2020 08:17:47 by "root" with gcc 4.4.7 20120313 (Red Hat 4.4.7-23)
Memory: 4k page, physical 6925020k(2979164k free), swap 0k(0k free)
CommandLine flags: -XX:+ExplicitGCInvokesConcurrent -XX:InitialHeapSize=6442450944 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ManagementServer -XX:MaxGCPauseMillis=20 -XX:MaxHeapSize=644245
0944 -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseG1GC 
You have new mail in /var/spool/mail/root
[root@tdsql01 logs]# more kafkaServer.out
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000680000000, 6442450944, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 6442450944 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /data/application/kafka/bin/hs_err_pid22721.log
[root@tdsql01 logs]# free -g
              total        used        free      shared  buff/cache   available
Mem:              6           2           2           0           1           3
Swap:             0           0           0

查看目录下的kafkaServer.out文件发现报错：内存不足

进一步分析kafka安装脚本，尝试修改内存限制

查看roles/tdsql_kafka/tasks/main.yml包含以下kafka启动脚本

- name: start kafka service
  shell: "source /etc/profile ; cd /data/application/kafka/bin ; ./kafka-server-start.sh -daemon ../config/server.properties; sleep 5"
  become: true
  become_user: tdsql

查看/data/application/kafka/bin/kafka-server-start.sh包含以下内存要求

if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
    export KAFKA_HEAP_OPTS="-Xmx6G -Xms6G"
fi

因为本机仅剩2G内存，这里修改JVM最大可用内存Xmx 和 初始内存Xms为1G

if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
    export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
fi

再次执行启动kafka的脚本，并查看日志，发现生成了server.log

[root@tdsql01 bin]# ./kafka-server-start.sh -daemon ../config/server.properties
[root@tdsql01 bin]# cd ../logs
[root@tdsql01 logs]# ls
controller.log  kafka-authorizer.log  kafka-request.log  kafkaServer-gc.log  kafkaServer.out  log-cleaner.log  server.log  state-change.log

查看server.log包含启动kafka的关键信息

 至此kafka安装成功

十、安装consumer

安装成功后显示信息及进程信息如下：

tdsql_consumer1            : ok=56   changed=37   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

You have new mail in /var/spool/mail/root
[root@tdsql01 tdsql_install]# ps -ef | grep consumer
root     28982     1  0 23:34 ?        00:00:00 ./binlogconsumermgn --zklist tdsql_test_zk1:2118 --zkrootpath /tdsqlzk --kafkazklist tdsql_test_zk1:2118 --kafkazkrootpath /kafka --dev ens33
root     28984 28982  0 23:34 ?        00:00:00 ./binlogconsumermgn --zklist tdsql_test_zk1:2118 --zkrootpath /tdsqlzk --kafkazklist tdsql_test_zk1:2118 --kafkazkrootpath /kafka --dev ens33
root     32007 61018  0 23:35 pts/1    00:00:00 grep --color=auto consumer

十一、安装后的资源使用截图

安装后的机型规格截图：

设备资源截图：

十二、创建分布式实例

因电脑资源有限，创建分布式实例CPU，内存，磁盘均需手动修改设置为最小

 容灾设置选择一主零备

其他设置选择默认即可，创建成功截图：

 

总结

1.实验证明tdsql可以单机安装，希望对于没有足够的机器资源但是想学习tdsql的小伙伴可以有所帮助，经本人测试除了tdsql必须的核心组件zk，scheduler，oss，chitu，monitor，db,proxy可以成功安装，非核心组件hdfs，kafka,consumer单机也可以成功安装，可以学习使用tdsql的绝大部分功能。
2.首次安装未成功产生启动php-frm的配置文件，具体原因后续有时间打算再做进一步尝试研究。

3.tdsql安装会在/etc/supervisord.d目录放置nginx，php，zookeeper的启动脚本，再次安装需要把这三个文件清理掉，否则安装会报错，提示端口已被使用。

4.kafka安装需要修改roles/tdsql_kafka/tasks/main.yml中数量，官方脚本限定kafka必须是3台，需要修改为1台。

5.kafka安装需要修改/data/application/kafka/bin/kafka-server-start.sh脚本中kafka启动的JVM内存设置，本次测试修改为1G，可以正常启动kafka

6.LVS负载均衡组件是否安装不影响tdsql的学习使用，故本次测试未安装LVS

7.es数据检索及分析工具是否安装不影响tdsql的学习使用，故本次测试未安装LVS