filecoin benchmarks v25

1，filecoin benchmarks v25

• rust安装

# curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# rustc --version
rustc 1.42.0 (b8cedc004 2020-03-09)

2，设置环境变量

• 使用终端代理

# export ALL_PROXY=socks5://127.0.0.1:1080
# git config --global http.proxy 'socks5://127.0.0.1:1080'
# export GO111MODULE=on
# export GOPROXY=https://goproxy.cn

3，下载源代码编译安装

在这里插入代码片

# git clone --branch testnet/3 https://github.com/filecoin-project/lotus.git

# git show
commit fd78f30204abd4b142419415c0e7f8b756fd1823 (HEAD -> testnet/3, origin/testnet/3)
Merge: 3bbbefd3 2384c673
Author: Whyrusleeping 
Date:   Tue Apr 21 15:23:49 2020 -0700

    Merge pull request #1568 from filecoin-project/feat/fix-reward-redeem

    fix reward redeem command to work again

# env RUSTFLAGS="-C target-cpu=native -g" FFI_BUILD_FROM_SOURCE=1 make clean deps bench

• 报错

rm -rf  build/.filecoin-install build/.update-modules  lotus lotus-storage-miner lotus-seal-worker lotus-shed lotus-seed pond townhall fountain chainwatch bench stats health
make -C extern/filecoin-ffi/ clean
make[1]: 进入目录“/root/lotus/extern/filecoin-ffi”
make[1]: *** 没有规则可制作目标“clean”。 停止。
make[1]: 离开目录“/root/lotus/extern/filecoin-ffi”
Makefile:180: recipe for target 'clean' failed
make: [clean] Error 2 (已忽略）
git submodule update --init --recursive
子模组 'extern/filecoin-ffi'（https://github.com/filecoin-project/filecoin-ffi.git）未对路径 'extern/filecoin-ffi' 注册
子模组 'extern/serialization-vectors'（https://github.com/filecoin-project/serialization-vectors）未对路径 'extern/serialization-vectors' 注册
正克隆到 '/root/lotus/extern/filecoin-ffi'...
正克隆到 '/root/lotus/extern/serialization-vectors'...
fatal: unable to access 'https://github.com/filecoin-project/serialization-vectors/': gnutls_handshake() failed: The TLS connection was non-properly terminated.
fatal: 无法克隆 'https://github.com/filecoin-project/serialization-vectors' 到子模组路径 '/root/lotus/extern/serialization-vectors'
克隆 'extern/serialization-vectors' 失败。按计划重试
正克隆到 '/root/lotus/extern/serialization-vectors'...
fatal: unable to access 'https://github.com/filecoin-project/serialization-vectors/': gnutls_handshake() failed: The TLS connection was non-properly terminated.
fatal: 无法克隆 'https://github.com/filecoin-project/serialization-vectors' 到子模组路径 '/root/lotus/extern/serialization-vectors'
第二次尝试克隆 'extern/serialization-vectors' 失败，退出
Makefile:39: recipe for target 'build/.update-modules' failed
make: *** [build/.update-modules] Error 1

• 设置环境变量

# export RUSTFLAGS="-C target-cpu=native -g"
# export FFI_BUILD_FROM_SOURCE=1

• 编译

make deps bench

# export FIL_PROOFS_MAXIMIZE_CACHING=1
# export RUST_LOG=info

4，设置环境变量，编译安装bench

export LOTUS_STORAGE_PATH=/storage/lotuswork/lotusstorage
export LOTUS_PATH=/storage/lotuswork/lotus
export WORKER_PATH=/storage/lotuswork/lotusworker

export FIL_PROOFS_PARAMETER_CACHE=/storage/filecoin-proof-parameters-v25

export TMPDIR=/storage/lotuswork/tmpdir

#bench
export RUSTFLAGS="-C target-cpu=native -g"
export FFI_BUILD_FROM_SOURCE=1
export FIL_PROOFS_MAXIMIZE_CACHING=1
export RUST_LOG=info

# cd lotus/
# git show
commit fd78f30204abd4b142419415c0e7f8b756fd1823 (HEAD -> testnet/3, origin/testnet/3)
Merge: 3bbbefd3 2384c673
Author: Whyrusleeping 
Date:   Tue Apr 21 15:23:49 2020 -0700

# git pull
# git show
commit a5d482bbc6565b4aaed3d68476248a13765009d8 (HEAD -> testnet/3, origin/testnet/3)
Merge: c4ffc2d9 e5aa67cc
Author: Łukasz Magiera 
Date:   Thu Apr 23 14:12:20 2020 +0200

# make clean deps bench

5，扇区选择

5.1 ，512MiB

# ./bench sealing --storage-dir=/storage/benchtmp --sector-size=512MiB

thread '' panicked at 'assertion failed: `(left == right)`
  left: `16777216`,
 right: `1073741824`: Cache is only available for 32GiB sectors', src/lib.rs:1:1

• 512MiB扇区需要注销#export FIL_PROOFS_MAXIMIZE_CACHING=1

results (v25) (536870912)
seal: addPiece: 6.431150925s (79.6 MiB/s)
seal: preCommit phase 1: 3m41.825094799s (2.31 MiB/s)
seal: preCommit phase 2: 15.639589781s (32.7 MiB/s)
seal: commit phase 1: 35.370985ms (14.1 GiB/s)
seal: commit phase 2: 1m59.496811517s (4.28 MiB/s)
seal: verify: 13.04201ms
unseal: 150.463µs  (3.25 TiB/s)

generate candidates: 64.817µs (7.53 TiB/s)
compute winnnig post proof (cold): 3.968097853s
compute winnnig post proof (hot): 3.918159352s
verify winnnig post proof (cold): 55.520937ms
verify winnnig post proof (hot): 29.877532ms

compute window post proof (cold): 1.673591195s
compute window post proof (hot): 1.627889423s
verify window post proof (cold): 22.017174ms
verify window post proof (hot): 11.483745ms

• 不使用显卡

# ./bench sealing --storage-dir=/storage/benchtmp --sector-size=512MiB --no-gpu

results (v25) (536870912)
seal: addPiece: 6.216612597s (82.4 MiB/s)
seal: preCommit phase 1: 3m41.521653028s (2.31 MiB/s)
seal: preCommit phase 2: 15.806485523s (32.4 MiB/s)
seal: commit phase 1: 43.850921ms (11.4 GiB/s)
seal: commit phase 2: 39.432575238s (13 MiB/s)
seal: verify: 14.539835ms
unseal: 271.023µs  (1.8 TiB/s)

generate candidates: 45.849µs (10.6 TiB/s)
compute winnnig post proof (cold): 4.475717057s
compute winnnig post proof (hot): 4.382428114s
verify winnnig post proof (cold): 61.266451ms
verify winnnig post proof (hot): 42.3675ms

compute window post proof (cold): 1.509256316s
compute window post proof (hot): 1.50099384s
verify window post proof (cold): 18.665974ms
verify window post proof (hot): 15.794574ms

5.2 ，32GiB

# ./bench sealing --storage-dir=/storage/benchtmp --sector-size=32GiB

# nohup ./bench sealing --storage-dir=/storage/benchtmp --sector-size=32GiB 2>&1 >> /storage/bench.log &

• 报错，需要内存211.25GB

[278332.825071] Out of memory: Killed process 11700 (bench) total-vm:221517396kB, anon-rss:130300340kB, file-rss:0kB, shmem-rss:0kB
[278345.286194] oom_reaper: reaped process 11700 (bench), now anon-rss:88kB, file-rss:0kB, shmem-rss:0kB

参考：

1. benchmarks github
2. Rust 安装
3. git socks5 代理