Clickhouse 性能测试数据集之TSBS

 TSBS概述：

Time Series Benchmark Suite (TSBS) 是一个使用go语言编写的，用于生成数据集的程序，TSBS可以对各种时序数据库的
读写性能进行基准测试。TSBS具有可扩展性， 可以对包括各种使用场景（例如devops，IoT，财务等）进行各种数据查询，对
数据库并进行基准测试。

支持测试各种时序数据库：
Clickhouse
MongoDB
InfluxDB
CrateDB
ElasticSearch
Cassandra
TimescaleDB
一些非常用的：
Akumuli 
SiriDB 
VictoriaMetrics 
OpenTSDB
Graphite
Splunk


使用场景：
截止目前TSBS支持IOT和devops场景。

devops:


IoT:

TSBS测试什么：


安装：

TSBS使用：

测试场景：
MongoDB
Elasticsearch
Clickhouse

环境设置：

1.下载go语言的安装包：
# wget https://golang.google.cn/dl/go1.9.6.linux-amd64.tar.gz
2.简单设置环境变量：

# cat /etc/profile.d/golang.sh 
#!/bin/bash
export PATH=$PATH:/usr/local/go/bin
export GOPATH=/root/go

# source  /etc/profile.d/golang.sh

# go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/root/go"
GOPRIVATE=""
GOPROXY="https://goproxy.cn,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/root/go/src/github.com/timescale/tsbs/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build865047752=/tmp/go-build -gno-record-gcc-switches"



设置国内的代理：

检查环境变量：
[root@hadoop scripts]# go version
go version go1.14.6 linux/amd64
[root@hadoop scripts]# echo $GOPATH
/root/go

安装编译：


[root@hadoop tsbs]# git clone https://github.com/go-yaml/yaml.git
Cloning into 'yaml'...
remote: Enumerating objects: 7, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 1643 (delta 1), reused 4 (delta 1), pack-reused 1636
Receiving objects: 100% (1643/1643), 1.52 MiB | 18.00 KiB/s, done.
Resolving deltas: 100% (1065/1065), done.
[root@hadoop tsbs]# mv yaml/ yaml.v2
[root@hadoop tsbs]# cd yaml.v2/
[root@hadoop yaml.v2]# go install
[root@hadoop yaml.v2]# go get github.com/timescale/tsbs
go: downloading github.com/timescale/tsbs v0.0.0-20200713202717-afd34446a5d9
go: github.com/timescale/tsbs upgrade => v0.0.0-20200713202717-afd34446a5d9

[root@hadoop ~]# cd /root/go/src/github.com/timescale/tsbs/cmd/
[root@hadoop cmd]# cd tsbs_generate_data && go install
go: downloading github.com/spf13/viper v1.4.0
go: downloading github.com/spf13/pflag v1.0.3
go: downloading github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8
go: downloading github.com/transceptor-technology/go-qpack v0.0.0-20190116123619-49a14b216a45
go: downloading github.com/google/flatbuffers v1.11.0
go: downloading github.com/filipecosta90/hdrhistogram v0.0.0-20191025144016-6360d1757d33
go: downloading golang.org/x/time v0.0.0-20190308202827-9d24e82272b4
go: downloading github.com/fsnotify/fsnotify v1.4.7
go: downloading github.com/mitchellh/mapstructure v1.1.2
go: downloading github.com/spf13/afero v1.1.2
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/magiconair/properties v1.8.0
go: downloading github.com/spf13/jwalterweatherman v1.0.0
go: downloading github.com/pelletier/go-toml v1.2.0
go: downloading github.com/spf13/cast v1.3.0
go: downloading golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456
go: downloading golang.org/x/text v0.3.2

[root@hadoop cmd]# go install ./...
go: downloading github.com/gocql/gocql v0.0.0-20190810123941-df4b9cc33030
go: downloading github.com/jackc/pgconn v1.1.0
go: downloading github.com/valyala/fasthttp v1.4.0
go: downloading github.com/shirou/gopsutil v2.18.12+incompatible
go: downloading github.com/pkg/errors v0.8.1
go: downloading github.com/jackc/pgx/v4 v4.1.1
go: downloading github.com/lib/pq v1.2.0
go: downloading github.com/SiriDB/go-siridb-connector v0.0.0-20190110105621-86b34c44c921
go: downloading github.com/kshvakov/clickhouse v1.3.11
go: downloading github.com/jmoiron/sqlx v1.2.0
go: downloading golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7
go: downloading github.com/jackc/pgproto3 v1.1.0
go: downloading github.com/jackc/pgpassfile v1.0.0
go: downloading github.com/jackc/pgio v1.0.0
go: downloading github.com/jackc/chunkreader v1.0.0
go: downloading github.com/jackc/pgtype v1.0.1
go: downloading golang.org/x/crypto v0.0.0-20190911031432-227b76d455e7
go: downloading github.com/jackc/puddle v1.0.0
go: downloading github.com/golang/snappy v0.0.0-20170215233205-553a64147049
go: downloading github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed
go: downloading github.com/jackc/pgproto3/v2 v2.0.0
go: downloading github.com/valyala/bytebufferpool v1.0.0
go: downloading github.com/klauspost/compress v1.4.0
go: downloading github.com/jackc/chunkreader/v2 v2.0.0
go: downloading gopkg.in/inf.v0 v0.9.1
go: downloading github.com/klauspost/cpuid v0.0.0-20180405133222-e7e905edc00e

[root@hadoop cmd]# cd /root/go/src/github.com/timescale/tsbs/scripts

简写的步骤：

# Fetch TSBS and its dependencies
$ go get github.com/timescale/tsbs
$ cd $GOPATH/src/github.com/timescale/tsbs/cmd
$ go get ./...

# Install desired binaries. At a minimum this includes tsbs_generate_data,
# tsbs_generate_queries, one tsbs_load_* binary, and one tsbs_run_queries_*
# binary:
$ cd $GOPATH/src/github.com/timescale/tsbs/cmd
$ cd tsbs_generate_data && go install
$ cd ../tsbs_generate_queries && go install
$ cd ../tsbs_load_timescaledb && go install
$ cd ../tsbs_run_queries_timescaledb && go install

# Optionally, install all binaries:
$ cd $GOPATH/src/github.com/timescale/tsbs/cmd
$ go install ./...

 

 

TSBS：
https://github.com/timescale/tsbs
https://github.com/influxdata/influxdb-comparisons

参考：

https://www.altinity.com/blog/clickhouse-for-time-series
https://www.altinity.com/blog/clickhouse-timeseries-scalability-benchmarks
https://www.altinity.com/blog/clickhouse-continues-to-crush-time-series

参考：

https://www.influxdata.com/blog/influxdb-markedly-elasticsearch-in-time-series-data-metrics-benchmark/