用pcdet跑deecamp20_liadr3-1
具体工程地址： deecamp20-3-1-lidar_det_pcdet
目前尝试过的内容：
在不满足速度的前提下，有：
- 使用PV-RCNN(CVPR20)做两阶段的实验
- 在openpcdet上采用gt增广的second
- 在det3D上采用second
满足速度要求的前提下： - 增加gt数据增广（没调参）
- 修改voxel-size，融合vfe特征提取(没调参)
- 增加SA-SSD（CVPR20）的两项附加任务（中西预测和分割）(没调参)
简介
deecamp20赛题3-1，128线下的自动驾驶场景的纯点云目标检测。环境：
实验对比
- 官方Baseline
| model | 备注 | FPS | MAP |
|---|---|---|---|
| second（16epoch） | Det3D版本， 采用multi-head和apex加速 |
6.67 | Evaluation official:car AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 bev AP:70.63 3d AP:68.38 aos AP:0.00 car AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:79.27 3d AP:79.14 aos AP:0.00 truck AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 bev AP:37.04 3d AP:25.94 aos AP:0.00 truck AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:51.19 3d AP:41.73 aos AP:0.00 tricar AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 bev AP:42.09 3d AP:31.61 aos AP:0.00 tricar AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:52.19 3d AP:52.07 aos AP:0.00 cyclist AP(Average Precision)@0.50, 0.50, 0.50: bbox AP:0.00 bev AP:61.12 3d AP:60.53 aos AP:0.00 cyclist AP(Average Precision)@0.50, 0.25, 0.25: bbox AP:0.00 bev AP:61.59 3d AP:61.58 aos AP:0.00 pedestrian AP(Average Precision)@0.50, 0.50, 0.50: bbox AP:0.00 bev AP:38.22 3d AP:36.45 aos AP:0.00 pedestrian AP(Average Precision)@0.50, 0.25, 0.25: bbox AP:0.00 bev AP:50.67 3d AP:50.63 aos AP:0.00 |
官方给出的base在FPS也是需要改进才能做到>10FPS的。
- 没有考虑速度的情形下的实验
| model | 备注 | FPS | MAP |
|---|---|---|---|
| second(16epoch) | openpcdet版本, 没有multihead，无apex加速, 采用gt_数据增广 |
5 | Car [email protected], 0.70, 0.70: bev AP:58.5172 3d AP:19.5790 aos AP:0.00 Car [email protected], 0.50, 0.50: bev AP:67.5123 3d AP:62.4208 aos AP:0.00 Truck [email protected], 0.50, 0.50: bev AP:12.6986 3d AP:9.3443 aos AP:0.00 Truck [email protected], 0.25, 0.25: bev AP:23.7338 3d AP:20.7437 aos AP:0.00 Tricar [email protected], 0.50, 0.50: bev AP:16.6359 3d AP:8.7095 aos AP:0.00 Tricar [email protected], 0.25, 0.25: bev AP:20.7536 3d AP:19.3297 aos AP:0.00 Cyclist [email protected], 0.70, 0.70: bev AP:7.4312 3d AP:2.2727 aos AP:0.00 Cyclist [email protected], 0.50, 0.50: bev AP:34.3507 3d AP:23.6472 aos AP:0.00 Pedestrian [email protected], 0.50, 0.50: bev AP:9.0909 3d AP:9.0909 aos AP:0.00 Pedestrian [email protected], 0.25, 0.25: bev AP:10.2772 3d AP:10.2405 aos AP:0.00 |
| PV-RCNN1(6epoch) | open-pcdet，two-stage，无apex加速， 采用gt_数据增广 |
3.37 | Evaluation official: car AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 bev AP:77.19 3d AP:69.01 aos AP:0.00 car AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:78.82 3d AP:78.46 aos AP:0.00 truck AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 bev AP:56.95 3d AP:46.56 aos AP:0.00 truck AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:64.10 3d AP:58.87 aos AP:0.00 tricar AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 bev AP:31.15 3d AP:21.79 aos AP:0.00 tricar AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:38.87 3d AP:35.25 aos AP:0.00 cyclist AP(Average Precision)@0.50, 0.50, 0.50: bbox AP:0.00 bev AP:61.22 3d AP:55.64 aos AP:0.00 cyclist AP(Average Precision)@0.50, 0.25, 0.25: bbox AP:0.00 bev AP:67.46 3d AP:66.95 aos AP:0.00 pedestrian AP(Average Precision)@0.50, 0.50, 0.50: bbox AP:0.00 bev AP:35.08 3d AP:30.14 aos AP:0.00 pedestrian AP(Average Precision)@0.50, 0.25, 0.25: bbox AP:0.00 bev AP:44.98 3d AP:41.03 aos AP:0.00 |
以上是在没有考虑速度下个人做的一些实验，其中PVRCNN是目前在KITTI上目标检测效果最好的文章，发表在的CVPR20，同时也获得了waymo CVPR20挑战赛的两个冠军，这个实验同时是参考了PV-RCNN文章中的waymo实验。但是非常的慢，达不到10FPS的要求，后续可以考虑apex加速等工作。但是two-stage的方法可能在这里不是很适用。可以看出在没有调参的基础上，在car和在truck上表现比较好。
- 考虑实用性在FPS>10以上的测试
这部分实验主要通过修改官方给出的base_line，首先在满足FPS>10的基础上，做一定的参数调整和小的细节修改。
| model | 备注 | FPS | MAP |
|---|---|---|---|
| second(30epoch) | det3D版本, 无gt_数据增广 设置[0.16, 0.16, 0.1], max_points_in_voxel=10, |
10.7 | Evaluation official: car AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 bev AP:68.53 3d AP:58.96 aos AP:0.00 car AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:70.86 3d AP:70.74 aos AP:0.00 truck AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 bev AP:46.35 3d AP:35.10 aos AP:0.00 truck AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:60.20 3d AP:51.22 aos AP:0.00 tricar AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 bev AP:37.10 3d AP:27.48 aos AP:0.00 tricar AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:49.20 3d AP:48.59 aos AP:0.00 cyclist AP(Average Precision)@0.50, 0.50, 0.50: bbox AP:0.00 bev AP:51.28 3d AP:50.27 aos AP:0.00 cyclist AP(Average Precision)@0.50, 0.25, 0.25: bbox AP:0.00 bev AP:60.13 3d AP:60.08 aos AP:0.00 pedestrian AP(Average Precision)@0.50, 0.50, 0.50: bbox AP:0.00 bev AP:25.82 3d AP:23.94 aos AP:0.00 pedestrian AP(Average Precision)@0.50, 0.25, 0.25: bbox AP:0.00 bev AP:39.67 3d AP:39.41 aos AP:0.00 |
| second(30epoch) | det3D版本, 设置[0.16, 0.16, 0.1], max_points_in_voxel=10, 增加gt数据增广 dict(Car=40), dict(Truck=45), dict(Tricar=45), dict(Cyclist=45), dict(Pedestrian=45) |
10.7 | Evaluation official: car AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 bev AP:57.69 3d AP:46.94 aos AP:0.00 car AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:69.17 3d AP:68.61 aos AP:0.00 truck AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 bev AP:31.42 3d AP:19.37 aos AP:0.00 truck AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:52.18 3d AP:44.65 aos AP:0.00 tricar AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 bev AP:22.14 3d AP:10.91 aos AP:0.00 tricar AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:38.36 3d AP:33.31 aos AP:0.00 cyclist AP(Average Precision)@0.50, 0.50, 0.50: bbox AP:0.00 bev AP:45.78 3d AP:43.43 aos AP:0.00 cyclist AP(Average Precision)@0.50, 0.25, 0.25: bbox AP:0.00 bev AP:57.38 3d AP:56.97 aos AP:0.00 pedestrian AP(Average Precision)@0.50, 0.50, 0.50: bbox AP:0.00 bev AP:28.24 3d AP:23.63 aos AP:0.00 pedestrian AP(Average Precision)@0.50, 0.25, 0.25: bbox AP:0.00 bev AP:40.77 3d AP:40.33 aos AP:0.00 |
| second(30epoch) | det3D版本, voxel_size=[0.12, 0.12, 0.2], 修改backbone使得z轴下采样为8倍 max_points_in_voxel=7, 增加gt数据增广 dict(Car=40), dict(Truck=45), dict(Tricar=45), dict(Cyclist=45), dict(Pedestrian=45) |
10.7 | Evaluation official: car AP(Average Precision)@0.70, 0.70, 0.70 bbox AP:0.00 bev AP:56.97 3d AP:45.90 aos AP:0.00 car AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:68.97 3d AP:68.24 aos AP:0.00 truck AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 3d AP:17.61 aos AP:0.00 truck AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:50.59 3d AP:42.87 aos AP:0.00 tricar AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 bev AP:19.91 3d AP:9.14 aos AP:0.00 tricar AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:37.78 3d AP:32.89 aos AP:0.00 cyclist AP(Average Precision)@0.50, 0.50, 0.50: bbox AP:0.00 bev AP:51.05 3d AP:43.75 aos AP:0.00 cyclist AP(Average Precision)@0.50, 0.25, 0.25: bbox AP:0.00 bev AP:64.20 3d AP:63.36 aos AP:0.00 pedestrian AP(Average Precision)@0.50, 0.50, 0.50: bbox AP:0.00 bev AP:32.24 3d AP:25.50 aos AP:0.00 pedestrian AP(Average Precision)@0.50, 0.25, 0.25: bbox AP:0.00 bev AP:44.06 3d AP:43.64 aos AP:0.00 |
| second(30epoch) | det3D版本, voxel_size=[0.30, 0.30, 0.2], max_points_in_voxel=30, 增加gt数据增广 dict(Car=20), dict(Truck=15), dict(Tricar=15), dict(Cyclist=15), dict(Pedestrian=15) 增加VFE模块（16*32） |
12 | Evaluation official: car AP(Average Precision)70, 0.70, 0.70: bbox AP:0.00 bev AP:47.00 3d AP:36.51 aos AP:0.00 car AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:66.31 3d AP:58.74 aos AP:0.00 truck AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 bev AP:24.46 3d AP:12.94 aos AP:0.00 truck AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:46.98 3d AP:43.84 aos AP:0.00 tricar AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:0.00 bev AP:16.55 3d AP:12.30 aos AP:0.00 tricar AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:0.00 bev AP:29.58 3d AP:24.20 aos AP:0.00 cyclist AP(Average Precision)@0.50, 0.50, 0.50: bbox AP:0.00 bev AP:42.81 3d AP:35.55 aos AP:0.00 cyclist AP(Average Precision)@0.50, 0.25, 0.25: bbox AP:0.00 bev AP:54.78 3d AP:54.31 aos AP:0.00 pedestrian AP(Average Precision)@0.50, 0.50, 0.50: bbox AP:0.00 bev AP:13.37 3d AP:12.05 aos AP:0.00 pedestrian AP(Average Precision)@0.50, 0.25, 0.25: bbox AP:0.00 bev AP:19.18 3d AP:18.84 aos AP:0.00 |
- FPS>10在网络上的改进
| model | 备注 | FPS | MAP |
|---|---|---|---|
| second+分割+中心预测(30epoch) | det3D版本, 无gt_数据增广 设置[0.16, 0.16, 0.1], max_points_in_voxel=10, 加SA-SSD的分割和中心预测 |
10.7 |
在华为云上部署
需要的文件（环境、数据、infos、db_datasets） 下载
Dataset Preparation
在华为云上的文件路径安排如下：
其中baseline如下：
DeepCamp_Lidar数据准备如下：
── DeepCamp_Lidar
│──labels_filter
│──test_filter
│──test_video_filter
│── train_val_filter
│── dee_infos_val.pkl
│── dee_infos_trainval.pkl
│── dee_infos_train.pkl
│── dee_dbinfos_train.pkl
│── dee_dbinfos_test.pkl
│── dee_dbinfos_test.pkl
│── dee_infos_test_video1.pkl
│── dee_infos_test_video2.pkl
│── gt_database
pvrcnn
pvrcnn
├── OpenLidarPerceptron
训练
pv-rcnn
预训练模型，针对car训练了一个模型 car_ckpt，针对所有物体训练了一个模型 all_ckpt .
单卡 训练模型：
python train.py --cfg_file ./cfgs/second_dee_car.yaml --batch_size 2
多卡训练：
python -m torch.distributed.launch --nproc_per_node=2 train.py --launcher pytorch --cfg_file ./cfgs/second_deecamp.yaml --batch_size 4
det3D
单卡训练
python3.6 tools/train.py ./examples/second/configs/deepcamp_all_vfev3_spmiddlefhd_rpn1_mghead_syncbn.py --work_dir ./res
多卡
python -m torch.distributed.launch --nproc_per_node=2 train.py --launcher pytorch ../examples/second/configs/deepcamp_all_vfev3_spmiddlefhd_rpn1_mghead_syncbn.py --work_dir ./ress
测试
pvrcnn
python test.py --cfg_file ../cfgs/second_deecamp.yaml --batch_size 4 --ckpt /your/path//checkpoint_epoch_20.pth
在个人服务器上部署
- 0.0.