SlowFast复现

SlowFast复现

  • 目录
    • 参考
    • 环境准备
    • 配置slowfast环境
    • 代码修改
    • 测试
    • 有可能出现的错误

目录

参考

论文
github地址
参考博客:
【SlowFast复现】SlowFast Networks for Video Recognition复现代码 使用自己的视频进行demo检测
可以对着这个大佬的b站视频复现,但是可以先看一下我的文章,因为官方对环境的要求有了变化

环境准备

windows下复现问题较多
租用网站的GPU:极链AI云平台
学生认证可以白嫖100云币券

  • 创建实例
    选择1个GPU就够用了
    SlowFast复现_第1张图片
  • 选择镜像
    pytorch尽量选择最高版本的,上面两个博客都是基于pytorch1.4(python3.7),现在github中的配置文件已经要求python≥3.8
    SlowFast复现_第2张图片
  • 安装Xshell
    SlowFast复现_第3张图片

SlowFast复现_第4张图片
SlowFast复现_第5张图片

SlowFast复现_第6张图片

配置slowfast环境

官方配置要求
SlowFast复现_第7张图片
我的配置过程
在xshell执行git clone将源代码下载至home目录下

cd /home/
git clone https://github.com/facebookresearch/SlowFast.git
# 创建虚拟环境
conda create -n slowfast python=3.8
conda activate slowfast

# 安装cuda11.1的pytorch1.8和torchvision0.9.0
pip install torch==1.8.0+cu111 torchvision==0.9.0+cu111 torchaudio==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html
# 安装fvcore
pip install 'git+https://github.com/facebookresearch/fvcore'
# 安装simplejson
pip install simplejson
# 安装PyAv
conda install av -c conda-forge
# 安装iopath
pip install -U iopath
# 安装psutil
pip install psutil
# 安装opencv-python pip install opencv-python
#我在这没有安装opencv 因为后面执行setup.py会给我们安好,建议不要pip安装

# 安装tensorboard
pip install tensorboard
# 安装cython
pip install cython
# 安装detectron2,这里与官网安装方式不同
python -m pip install detectron2==0.6 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.8/index.html

# 导入路径,路径如下图所示,修改成你自己的路径即可
export PYTHONPATH=/home/slowfast/slowfast:$PYTHONPATH
# 进入SlowFast目录
cd slowfast
# 编译,可以干点别的事等着,非常慢
python setup.py build develop

代码修改

  • 在/slowfast/demo/AVA目录下新建ava.json,文件内容如下
{"bend/bow (at the waist)": 0, "crawl": 1, "crouch/kneel": 2, "dance": 3, "fall down": 4, "get up": 5, "jump/leap": 6, "lie/sleep": 7, "martial art": 8, "run/jog": 9, "sit": 10, "stand": 11, "swim": 12, "walk": 13, "answer phone": 14, "brush teeth": 15, "carry/hold (an object)": 16, "catch (an object)": 17, "chop": 18, "climb (e.g., a mountain)": 19, "clink glass": 20, "close (e.g., a door, a box)": 21, "cook": 22, "cut": 23, "dig": 24, "dress/put on clothing": 25, "drink": 26, "drive (e.g., a car, a truck)": 27, "eat": 28, "enter": 29, "exit": 30, "extract": 31, "fishing": 32, "hit (an object)": 33, "kick (an object)": 34, "lift/pick up": 35, "listen (e.g., to music)": 36, "open (e.g., a window, a car door)": 37, "paint": 38, "play board game": 39, "play musical instrument": 40, "play with pets": 41, "point to (an object)": 42, "press": 43, "pull (an object)": 44, "push (an object)": 45, "put down": 46, "read": 47, "ride (e.g., a bike, a car, a horse)": 48, "row boat": 49, "sail boat": 50, "shoot": 51, "shovel": 52, "smoke": 53, "stir": 54, "take a photo": 55, "text on/look at a cellphone": 56, "throw": 57, "touch (an object)": 58, "turn (e.g., a screwdriver)": 59, "watch (e.g., TV)": 60, "work on a computer": 61, "write": 62, "fight/hit (a person)": 63, "give/serve (an object) to (a person)": 64, "grab (a person)": 65, "hand clap": 66, "hand shake": 67, "hand wave": 68, "hug (a person)": 69, "kick (a person)": 70, "kiss (a person)": 71, "lift (a person)": 72, "listen to (a person)": 73, "play with kids": 74, "push (another person)": 75, "sing to (e.g., self, a person, a group)": 76, "take (an object) from (a person)": 77, "talk to (e.g., self, a person, a group)": 78, "watch (a person)": 79}

  • 修改
    /slowfast/demo/AVA/SLOWFAST_32x2_R101_50_50.yaml,内容改为如下:
TRAIN:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 16
  EVAL_PERIOD: 1
  CHECKPOINT_PERIOD: 1
  AUTO_RESUME: True
  CHECKPOINT_FILE_PATH: "/media/bao/新加卷1/sunqiang/SlowFast/configs/AVA/c2/SLOWFAST_32x2_R101_50_50.pkl"  #path to pretrain model
  CHECKPOINT_TYPE: pytorch
DATA:
  NUM_FRAMES: 32
  SAMPLING_RATE: 2
  TRAIN_JITTER_SCALES: [256, 320]
  TRAIN_CROP_SIZE: 224
  TEST_CROP_SIZE: 256
  INPUT_CHANNEL_NUM: [3, 3]
DETECTION:
  ENABLE: True
  ALIGNED: False
AVA:
  BGR: False
  DETECTION_SCORE_THRESH: 0.8
  TEST_PREDICT_BOX_LISTS: ["person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"]
SLOWFAST:
  ALPHA: 4
  BETA_INV: 8
  FUSION_CONV_CHANNEL_RATIO: 2
  FUSION_KERNEL_SZ: 5
RESNET:
  ZERO_INIT_FINAL_BN: True
  WIDTH_PER_GROUP: 64
  NUM_GROUPS: 1
  DEPTH: 101
  TRANS_FUNC: bottleneck_transform
  STRIDE_1X1: False
  NUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]
  SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]]
  SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]]
NONLOCAL:
  LOCATION: [[[], []], [[], []], [[6, 13, 20], []], [[], []]]
  GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]
  INSTANTIATION: dot_product
  POOL: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]]
BN:
  USE_PRECISE_STATS: False
  NUM_BATCHES_PRECISE: 200
SOLVER:
  MOMENTUM: 0.9
  WEIGHT_DECAY: 1e-7
  OPTIMIZING_METHOD: sgd
MODEL:
  NUM_CLASSES: 80
  ARCH: slowfast
  MODEL_NAME: SlowFast
  LOSS_FUNC: bce
  DROPOUT_RATE: 0.5
  HEAD_ACT: sigmoid
TEST:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 8
DATA_LOADER:
  NUM_WORKERS: 2
  PIN_MEMORY: True

NUM_GPUS: 1
NUM_SHARDS: 1
RNG_SEED: 0
OUTPUT_DIR: .

#TENSORBOARD:
#  MODEL_VIS:
#    TOPK: 2
DEMO:
  ENABLE: True
  LABEL_FILE_PATH: "/media/bao/新加卷1/sunqiang/SlowFast/demo/AVA/ava.json"
  INPUT_VIDEO: "/home/slowfast/Vinput/1.mp4"
  OUTPUT_FILE: "/home/slowfast/Vinput/2.mp4"

  DETECTRON2_CFG: "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml"
  DETECTRON2_WEIGHTS: https://dl.fbaipublicfiles.com/detectron2/COCO-Detection/faster_rcnn_R_50_FPN_3x/137849458/model_final_280758.pkl

  • 下载预训练权重文件
    下载页面
    SlowFast复现_第8张图片
    我先下载在本地,然后利用了xftp传输,包括后面用的测试视频也是直接这样传的
    SlowFast复现_第9张图片
    下载模型SLOWFAST_32x2_R101_50_50.pkl 到/home/slowfast/configs/AVA/c2目录下,修改参数
TRAIN:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 16
  EVAL_PERIOD: 1
  CHECKPOINT_PERIOD: 1
  AUTO_RESUME: True
  CHECKPOINT_FILE_PATH: '/home/slowfast/configs/AVA/c2/SLOWFAST_32x2_R101_50_50.pkl'  #path to pretrain model
  CHECKPOINT_TYPE: pytorch

测试

代码运行

cd /home/slowfast/
python tools/run_net.py --cfg demo/AVA/SLOWFAST_32x2_R101_50_50.yaml

结果(随便在网上找的一段视频)
SlowFast复现_第10张图片

有可能出现的错误

我在slowfast的配置部分写的安装命令已经是和我的环境适配了

python -m pip install detectron2==0.6 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.8/index.html

我最开始因为因为detectron2的版本问题报错:not such file or directory
和我的环境不一样的话,可以去该链接下找到合适的版本:
detectron2

SlowFast复现_第11张图片

你可能感兴趣的:(行为识别,论文复现,深度学习,计算机视觉)