京东 fastreid windows 10 环境配置详细过程

提示：文章写完后，目录可以自动生成，如何生成可参考右边的帮助文档

---

前言

在有些场景中，目标检测无法解决重复识别的问题。为了解决去重，引入Reid 的网络模型。万丈高楼平地起，先搭个环境吧。

---

一、fastreid 下载

1.打开github，在搜索框输入fastreid。或者直接点击https://github.com/JDAI-CV/fast-reid进入fast-reid的仓库。

2.在Code 下拉列表中选择Download ZIP，下载源码文件。

3.将源码 文件解压至合适的地方。

二、环境配置

1.在开始菜单Anaconda3 中选择Anaconda Prompt(Anaconda3) 。

2.使用命令创建fastreid 的环境

conda create -n fastreid python=3.7

3.使用命令激活fastreid 环境

conda activate fastreid

4.安装cuda cudnn。

conda install cudatoolkit=10.2 cudnn=7.6.5

5.安装pytorch 。进入pytorch 官网，查找对应cuda 9.2版本的pytorch 安装命令

conda install pytorch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1 cudatoolkit=10.2 -c pytorch

6.检测cuda 、torch 是否安装正确

python
>>> import torch
>>> print(torch.cuda.is_available())
True
>>> exit()

//最后测试结果为True 表示安装正确

7.安装reqiurements.txt。

pip install -r requirements.txt

8.安装faiss-cpu

pip install faiss-cpu

9.安装其他包

pip install Cython yacs sklearn termcolor tabulate gdown

10 编译

//官方使用命令
cd fastreid/evaluation/rank_cylib; make all 

// 使用mingw32-make 替换make
cd fastreid/evaluation/rank_cylib; mingw32-make all 
python3 setup.py build_ext --inplace
mingw32-make: *** [Makefile:2: all] Error 9009

//改用python setup.py install 解决
python setup.py install

---

三、测试环境

训练测试

python3 tools/train_net.py --config-file ./configs/Market1501/bagtricks_R50.yml MODEL.DEVICE "cuda:0"

[06/29 16:05:48 fastreid.utils.events]:  eta: 0:39:52  epoch/iter: 24/4999  total_loss: 1.391  loss_cls: 1.335  loss_triplet: 0.04697  time: 0.1240  data_time: 0.0008  lr: 3.50e-04  max_mem: 4775M
[06/29 16:05:54 fastreid.utils.events]:  eta: 0:39:46  epoch/iter: 24/5049  total_loss: 1.38  loss_cls: 1.331  loss_triplet: 0.04439  time: 0.1240  data_time: 0.0007  lr: 3.50e-04  max_mem: 4775M
[06/29 16:06:13 fastreid.utils.events]:  eta: 0:39:28  epoch/iter: 25/5199  total_loss: 1.385  loss_cls: 1.334  loss_triplet: 0.04699  time: 0.1240  data_time: 0.0008  lr: 3.50e-04  max_mem: 4775M
[06/29 16:06:19 fastreid.utils.events]:  eta: 0:39:21  epoch/iter: 25/5251  total_loss: 1.381  loss_cls: 1.33  loss_triplet: 0.04889  time: 0.1240  data_time: 0.0007  lr: 3.50e-04  max_mem: 4775M
[06/29 16:06:38 fastreid.utils.events]:  eta: 0:39:03  epoch/iter: 26/5399  total_loss: 1.375  loss_cls: 1.316  loss_triplet: 0.04676  time: 0.1240  data_time: 0.0009  lr: 3.50e-04  max_mem: 4775M
[06/29 16:06:44 fastreid.utils.events]:  eta: 0:38:57  epoch/iter: 26/5453  total_loss: 1.368  loss_cls: 1.314  loss_triplet: 0.04802  time: 0.1241  data_time: 0.0007  lr: 3.50e-04  max_mem: 4775M
[06/29 16:07:02 fastreid.utils.events]:  eta: 0:38:39  epoch/iter: 27/5599  total_loss: 1.368  loss_cls: 1.317  loss_triplet: 0.05085  time: 0.1241  data_time: 0.0008  lr: 3.50e-04  max_mem: 4775M
[06/29 16:07:09 fastreid.utils.events]:  eta: 0:38:32  epoch/iter: 27/5655  total_loss: 1.37  loss_cls: 1.32  loss_triplet: 0.05181  time: 0.1241  data_time: 0.0007  lr: 3.50e-04  max_mem: 4775M
[06/29 16:07:27 fastreid.utils.events]:  eta: 0:38:14  epoch/iter: 28/5799  total_loss: 1.373  loss_cls: 1.318  loss_triplet: 0.0477  time: 0.1241  data_time: 0.0007  lr: 3.50e-04  max_mem: 4775M
[06/29 16:07:35 fastreid.utils.events]:  eta: 0:38:08  epoch/iter: 28/5857  total_loss: 1.373  loss_cls: 1.312  loss_triplet: 0.0514  time: 0.1241  data_time: 0.0009  lr: 3.50e-04  max_mem: 4775M
[06/29 16:07:52 fastreid.utils.events]:  eta: 0:37:49  epoch/iter: 29/5999  total_loss: 1.367  loss_cls: 1.304  loss_triplet: 0.05396  time: 0.1241  data_time: 0.0006  lr: 3.50e-04  max_mem: 4775M
[06/29 16:08:00 fastreid.engine.defaults]: Prepare testing set
E:\Anaconda3\envs\fastreid\lib\site-packages\torchvision\transforms\transforms.py:288: UserWarning: Argument interpolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum.
  "Argument interpolation should be of type InterpolationMode instead of int. "
[06/29 16:08:00 fastreid.data.datasets.bases]: => Loaded Market1501 in csv format:
| subset   | # ids   | # images   | # cameras   |
|:---------|:--------|:-----------|:------------|
| query    | 750     | 3368       | 6           |
| gallery  | 751     | 15913      | 6           |
[06/29 16:08:00 fastreid.evaluation.evaluator]: Start inference on 19281 images
[06/29 16:08:09 fastreid.evaluation.evaluator]: Inference done 11/151. 0.1032 s / batch. ETA=0:00:15
[06/29 16:08:32 fastreid.evaluation.evaluator]: Total inference time: 0:00:23.524918 (0.161130 s / batch per device, on 1 devices)
[06/29 16:08:32 fastreid.evaluation.evaluator]: Total inference pure compute time: 0:00:16 (0.110007 s / batch per device, on 1 devices)
[06/29 16:09:56 fastreid.engine.defaults]: Evaluation results for Market1501 in csv format:
[06/29 16:09:56 fastreid.evaluation.testing]: Evaluation results in csv format:
| Dataset    | Rank-1   | Rank-5   | Rank-10   | mAP   | mINP   | metric   |
|:-----------|:---------|:---------|:----------|:------|:-------|:---------|
| Market1501 | 79.99    | 92.40    | 95.25     | 56.51 | 16.58  | 68.25    |
[06/29 16:09:56 fastreid.utils.checkpoint]: Saving checkpoint to logs/market1501/bagtricks_R50\model_best.pth
[06/29 16:09:56 fastreid.utils.checkpoint]: Saving checkpoint to logs/market1501/bagtricks_R50\model_0029.pth
[06/29 16:09:57 fastreid.utils.events]:  eta: 0:37:42  epoch/iter: 29/6059  total_loss: 1.352  loss_cls: 1.304  loss_triplet: 0.04775  time: 0.1241  data_time: 0.0009  lr: 3.50e-04  max_mem: 4775M
[06/29 16:10:13 fastreid.utils.events]:  eta: 0:37:22  epoch/iter: 30/6199  total_loss: 1.365  loss_cls: 1.313  loss_triplet: 0.04689  time: 0.1240  data_time: 0.0010  lr: 3.50e-04  max_mem: 4775M
[06/29 16:10:21 fastreid.utils.events]:  eta: 0:37:13  epoch/iter: 30/6261  total_loss: 1.372  loss_cls: 1.316  loss_triplet: 0.05124  time: 0.1239  data_time: 0.0015  lr: 3.50e-04  max_mem: 4775M

四、问题汇总

1.pytorch 和 torchvision 安装成cpu 版本。

cpuonly            pytorch/noarch::cpuonly-1.0-0
  freetype           anaconda/pkgs/main/win-64::freetype-2.10.4-hd328e21_0
  jpeg               anaconda/pkgs/main/win-64::jpeg-9e-h2bbff1b_0
  libpng             anaconda/pkgs/main/win-64::libpng-1.6.37-h2a8f88b_0
  libtiff            anaconda/pkgs/main/win-64::libtiff-4.2.0-he0120a3_1
  libuv              anaconda/pkgs/main/win-64::libuv-1.40.0-he774522_0
  libwebp            anaconda/pkgs/main/win-64::libwebp-1.2.2-h2bbff1b_0
  lz4-c              anaconda/pkgs/main/win-64::lz4-c-1.9.3-h2bbff1b_1
  ninja              anaconda/pkgs/main/win-64::ninja-1.10.2-haa95532_5
  ninja-base         anaconda/pkgs/main/win-64::ninja-base-1.10.2-h6d14046_5
  pillow             anaconda/pkgs/main/win-64::pillow-9.0.1-py37hdc2b20a_0
  pytorch            pytorch/win-64::pytorch-1.7.0-py3.7_cpu_0
  tk                 anaconda/pkgs/main/win-64::tk-8.6.12-h2bbff1b_0
  torchaudio         pytorch/win-64::torchaudio-0.7.0-py37
  torchvision        pytorch/win-64::torchvision-0.8.0-py37_cpu
  xz                 anaconda/pkgs/main/win-64::xz-5.2.5-h8cc25b3_1
  zstd               anaconda/pkgs/main/win-64::zstd-1.5.2-h19a0ad4_0

原因：是因为某些镜像源导致的。删除该镜像源再用pip安装，最后解决方案是采用更高的版本的pytorch。

config --show-sources  //显示所有源

(fastreid) C:\Users\Administrator>conda config --show-sources
==> C:\Users\Administrator\.condarc <==
ssl_verify: True
channels:
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
  - defaults
show_channel_urls: True

//删除镜像源
(fastreid) C:\Users\Administrator>conda config --remove channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
(fastreid) C:\Users\Administrator>conda config --remove channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/

3.‘make’ 不是内部或外部命令，也不是可运行的程序
或批处理文件。
参考博客https://blog.csdn.net/qq_20163075/article/details/119944298 安装即可。

4. *** [Makefile:2: all] Error 9009

5.ImportError: TensorBoard logging requires TensorBoard version 1.15 or above
提示说tensorboard的版本太低，提高下版本
查看tensorboard的信息

(fastreid) D:\liufq\reid\fast-reid-1.3.0>pip show tensorboard
Name: tensorboard
Version: 1.14.0
Summary: TensorBoard lets you watch Tensors Flow
Home-page: https://github.com/tensorflow/tensorboard
Author: Google Inc.
Author-email: packages@tensorflow.org
License: Apache 2.0
Location: e:\anaconda3\envs\fastreid\lib\site-packages
Requires: protobuf, werkzeug, wheel, numpy, grpcio, markdown, six, setuptools, absl-py
Required-by:

将tensorboard 版本改成1.15.0后解决。

6.ValueError: Buffer dtype mismatch, expected ‘long’ but got ‘long long’

Traceback (most recent call last):
  File ".\fastreid\engine\train_loop.py", line 147, in train
    self.after_epoch()
  File ".\fastreid\engine\train_loop.py", line 181, in after_epoch
    h.after_epoch()
  File ".\fastreid\engine\hooks.py", line 371, in after_epoch
    self._do_eval()
  File ".\fastreid\engine\hooks.py", line 345, in _do_eval
    results = self._func()
  File ".\fastreid\engine\defaults.py", line 305, in test_and_save_results
    self._last_eval_results = self.test(self.cfg, self.model)
  File ".\fastreid\engine\defaults.py", line 441, in test
    results_i = inference_on_dataset(model, data_loader, evaluator, flip_test=cfg.TEST.FLIP.ENABLED)
  File ".\fastreid\evaluation\evaluator.py", line 156, in inference_on_dataset
    results = evaluator.evaluate()
  File ".\fastreid\evaluation\reid_evaluation.py", line 105, in evaluate
    cmc, all_AP, all_INP = evaluate_rank(dist, query_pids, gallery_pids, query_camids, gallery_camids)
  File ".\fastreid\evaluation\rank.py", line 200, in evaluate_rank
    return evaluate_cy(distmat, q_pids, g_pids, q_camids, g_camids, max_rank, use_metric_cuhk03)
  File "rank_cy.pyx", line 20, in rank_cy.evaluate_cy
  File "rank_cy.pyx", line 28, in rank_cy.evaluate_cy
ValueError: Buffer dtype mismatch, expected 'long' but got 'long long'

在windows 10 下 np.int_t 替换成np.int64_t，如果不行就替换成np.int32_t。测试发现仍然报错。

    #q_pids = np.asarray(q_pids, dtype=np.int64)
    q_pids = np.asarray(q_pids,dtype=np.int32)
    #g_pids = np.asarray(g_pids, dtype=np.int64)
	g_pids = np.asarray(g_pids, dtype=np.int32)
    #q_camids = np.asarray(q_camids, dtype=np.int64)
	q_camids = np.asarray(q_camids, dtype=np.int32)
    #g_camids = np.asarray(g_camids, dtype=np.int64)
	g_camids = np.asarray(g_camids, dtype=np.int32)

最后的解决办法为

在rank.py文件中，把184行的Ture改成False,不使用cython编译的东西