版本信息
| 版本号 | 修订日期 | 修订人 |
|---|---|---|
| v0.1 | 2018/10/17 | 汪阳 |
donkeycar
工作原理
如下图所示，donkey car的工作原理很简单，通过行为克隆技术来实现自动驾驶。想了解详细的信息，请访问donkeycar官网。
donkeycar 原理
1. 克隆donkey代码到高性能GPU实例 (google colab GPU or aws EC2 (g2.4xlarge))
这里以google colab 提供的免费GPU实例为例
!git clone -b master https://github.com/wywincl/donkey.git donkey
Cloning into 'donkey'...
remote: Enumerating objects: 7, done.�[K
remote: Counting objects: 100% (7/7), done.�[K
remote: Compressing objects: 100% (7/7), done.�[K
remote: Total 6206 (delta 0), reused 0 (delta 0), pack-reused 6199�[K
Receiving objects: 100% (6206/6206), 30.79 MiB | 18.72 MiB/s, done.
Resolving deltas: 100% (3579/3579), done.
2. 源码安装donkey
!pip3 install -e donkey
Obtaining file:///content/donkey
Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from donkeycar==2.5.1) (1.14.6)
Requirement already satisfied: pillow in /usr/local/lib/python3.6/dist-packages (from donkeycar==2.5.1) (4.0.0)
Requirement already satisfied: docopt in /usr/local/lib/python3.6/dist-packages (from donkeycar==2.5.1) (0.6.2)
Requirement already satisfied: tornado==4.5.3 in /usr/local/lib/python3.6/dist-packages (from donkeycar==2.5.1) (4.5.3)
Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from donkeycar==2.5.1) (2.18.4)
Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from donkeycar==2.5.1) (2.8.0)
Requirement already satisfied: python-socketio in /usr/local/lib/python3.6/dist-packages (from donkeycar==2.5.1) (2.0.0)
Requirement already satisfied: flask in /usr/local/lib/python3.6/dist-packages (from donkeycar==2.5.1) (1.0.2)
Requirement already satisfied: eventlet in /usr/local/lib/python3.6/dist-packages (from donkeycar==2.5.1) (0.24.1)
Requirement already satisfied: moviepy in /usr/local/lib/python3.6/dist-packages (from donkeycar==2.5.1) (0.2.3.5)
Requirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (from donkeycar==2.5.1) (0.22.0)
Requirement already satisfied: olefile in /usr/local/lib/python3.6/dist-packages (from pillow->donkeycar==2.5.1) (0.46)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->donkeycar==2.5.1) (1.22)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->donkeycar==2.5.1) (2.6)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->donkeycar==2.5.1) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->donkeycar==2.5.1) (2018.10.15)
Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from h5py->donkeycar==2.5.1) (1.11.0)
Requirement already satisfied: python-engineio>=2.2.0 in /usr/local/lib/python3.6/dist-packages (from python-socketio->donkeycar==2.5.1) (2.3.2)
Requirement already satisfied: Jinja2>=2.10 in /usr/local/lib/python3.6/dist-packages (from flask->donkeycar==2.5.1) (2.10)
Requirement already satisfied: click>=5.1 in /usr/local/lib/python3.6/dist-packages (from flask->donkeycar==2.5.1) (7.0)
Requirement already satisfied: Werkzeug>=0.14 in /usr/local/lib/python3.6/dist-packages (from flask->donkeycar==2.5.1) (0.14.1)
Requirement already satisfied: itsdangerous>=0.24 in /usr/local/lib/python3.6/dist-packages (from flask->donkeycar==2.5.1) (0.24)
Requirement already satisfied: greenlet>=0.3 in /usr/local/lib/python3.6/dist-packages (from eventlet->donkeycar==2.5.1) (0.4.15)
Requirement already satisfied: monotonic>=1.4 in /usr/local/lib/python3.6/dist-packages (from eventlet->donkeycar==2.5.1) (1.5)
Requirement already satisfied: dnspython>=1.15.0 in /usr/local/lib/python3.6/dist-packages (from eventlet->donkeycar==2.5.1) (1.15.0)
Requirement already satisfied: imageio<3.0,>=2.1.2 in /usr/local/lib/python3.6/dist-packages (from moviepy->donkeycar==2.5.1) (2.4.1)
Requirement already satisfied: tqdm<5.0,>=4.11.2 in /usr/local/lib/python3.6/dist-packages (from moviepy->donkeycar==2.5.1) (4.27.0)
Requirement already satisfied: decorator<5.0,>=4.0.2 in /usr/local/lib/python3.6/dist-packages (from moviepy->donkeycar==2.5.1) (4.3.0)
Requirement already satisfied: pytz>=2011k in /usr/local/lib/python3.6/dist-packages (from pandas->donkeycar==2.5.1) (2018.5)
Requirement already satisfied: python-dateutil>=2 in /usr/local/lib/python3.6/dist-packages (from pandas->donkeycar==2.5.1) (2.5.3)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.6/dist-packages (from Jinja2>=2.10->flask->donkeycar==2.5.1) (1.0)
Installing collected packages: donkeycar
Found existing installation: donkeycar 2.5.1
Uninstalling donkeycar-2.5.1:
Successfully uninstalled donkeycar-2.5.1
Running setup.py develop for donkeycar
Successfully installed donkeycar
3. 创建donkey 自动驾驶小车工程d2
!donkey createcar --template donkey2 ~/d2
using donkey v2.5.1 ...
Creating car folder: /root/d2
making dir /root/d2
Creating data & model folders.
making dir /root/d2/models
making dir /root/d2/data
making dir /root/d2/logs
Car app already exists. Delete it and rerun createcar to replace.
Car config already exists. Delete it and rerun createcar to replace.
Donkey setup complete.
4. 进入d2工程，并获取帮助信息
!cd d2 && python3 manage.py
using donkey v2.5.1 ...
Usage:
manage.py (drive) [--model=] [--js] [--chaos]
manage.py (train) [--tub=] (--model=) [--base_model=] [--no_cache]
5. 上传数据到计算实例
将无人驾驶小车手工采集到的数据压缩并上传到计算实例（google colab GPU or aws EC2 (g2.4xlarge)）中，
然后解压缩到指定目录中
!unzip -o d2/data/training_tubs_26.6.2018.zip -d d2/data/
6. 训练模型
采集到的多轮次数据，首先进行清洗，丢弃不良数据，然后作为训练数据，开始训练模型，并保存模型文件
!cd d2 && python3 manage.py train --tub="data/tub_16/, data/tub_17" --model=models/mypilot.h5
using donkey v2.5.1 ...
loading config file: /content/d2/config.py
config loaded
tub_names data/tub_16/
train: 2624, validation: 657
steps_per_epoch 20
Epoch 1/100
2018-10-17 04:43:11.714513: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:964] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-10-17 04:43:11.715134: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1411] Found device 0 with properties:
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:00:04.0
totalMemory: 11.17GiB freeMemory: 11.10GiB
2018-10-17 04:43:11.715206: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1490] Adding visible gpu devices: 0
2018-10-17 04:43:12.192129: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-10-17 04:43:12.192207: I tensorflow/core/common_runtime/gpu/gpu_device.cc:977] 0
2018-10-17 04:43:12.192278: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990] 0: N
2018-10-17 04:43:12.192551: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:42] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.
2018-10-17 04:43:12.192614: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1103] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10758 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
19/20 [===========================>..] - ETA: 0s - loss: 2.8320 - angle_out_loss: 3.1369 - throttle_out_loss: 0.8777
Epoch 00001: val_loss improved from inf to 1.23087, saving model to models/mypilot.h5
20/20 [==============================] - 11s 532ms/step - loss: 2.7686 - angle_out_loss: 3.0668 - throttle_out_loss: 0.8459 - val_loss: 1.2309 - val_angle_out_loss: 1.3650 - val_throttle_out_loss: 0.2409
Epoch 2/100
19/20 [===========================>..] - ETA: 0s - loss: 1.4160 - angle_out_loss: 1.5702 - throttle_out_loss: 0.2761
Epoch 00002: val_loss did not improve from 1.23087
20/20 [==============================] - 5s 249ms/step - loss: 1.4174 - angle_out_loss: 1.5718 - throttle_out_loss: 0.2783 - val_loss: 1.2457 - val_angle_out_loss: 1.3812 - val_throttle_out_loss: 0.2536
Epoch 3/100
19/20 [===========================>..] - ETA: 0s - loss: 1.2386 - angle_out_loss: 1.3733 - throttle_out_loss: 0.2647
Epoch 00003: val_loss improved from 1.23087 to 1.06440, saving model to models/mypilot.h5
20/20 [==============================] - 5s 263ms/step - loss: 1.2361 - angle_out_loss: 1.3705 - throttle_out_loss: 0.2656 - val_loss: 1.0644 - val_angle_out_loss: 1.1799 - val_throttle_out_loss: 0.2526
Epoch 4/100
19/20 [===========================>..] - ETA: 0s - loss: 1.1434 - angle_out_loss: 1.2673 - throttle_out_loss: 0.2816
Epoch 00004: val_loss improved from 1.06440 to 1.04143, saving model to models/mypilot.h5
20/20 [==============================] - 5s 262ms/step - loss: 1.1412 - angle_out_loss: 1.2649 - throttle_out_loss: 0.2798 - val_loss: 1.0414 - val_angle_out_loss: 1.1544 - val_throttle_out_loss: 0.2453
Epoch 5/100
19/20 [===========================>..] - ETA: 0s - loss: 0.9802 - angle_out_loss: 1.0860 - throttle_out_loss: 0.2863
Epoch 00005: val_loss improved from 1.04143 to 0.98422, saving model to models/mypilot.h5
20/20 [==============================] - 5s 266ms/step - loss: 0.9714 - angle_out_loss: 1.0762 - throttle_out_loss: 0.2850 - val_loss: 0.9842 - val_angle_out_loss: 1.0908 - val_throttle_out_loss: 0.2452
Epoch 6/100
19/20 [===========================>..] - ETA: 0s - loss: 0.9049 - angle_out_loss: 1.0024 - throttle_out_loss: 0.2720
Epoch 00006: val_loss improved from 0.98422 to 0.97521, saving model to models/mypilot.h5
20/20 [==============================] - 5s 264ms/step - loss: 0.9020 - angle_out_loss: 0.9993 - throttle_out_loss: 0.2719 - val_loss: 0.9752 - val_angle_out_loss: 1.0809 - val_throttle_out_loss: 0.2425
Epoch 7/100
19/20 [===========================>..] - ETA: 0s - loss: 0.8303 - angle_out_loss: 0.9198 - throttle_out_loss: 0.2505
Epoch 00007: val_loss improved from 0.97521 to 0.95358, saving model to models/mypilot.h5
20/20 [==============================] - 5s 263ms/step - loss: 0.8286 - angle_out_loss: 0.9179 - throttle_out_loss: 0.2509 - val_loss: 0.9536 - val_angle_out_loss: 1.0569 - val_throttle_out_loss: 0.2408
Epoch 8/100
19/20 [===========================>..] - ETA: 0s - loss: 0.6880 - angle_out_loss: 0.7617 - throttle_out_loss: 0.2461
Epoch 00008: val_loss improved from 0.95358 to 0.85174, saving model to models/mypilot.h5
20/20 [==============================] - 5s 260ms/step - loss: 0.6867 - angle_out_loss: 0.7602 - throttle_out_loss: 0.2459 - val_loss: 0.8517 - val_angle_out_loss: 0.9437 - val_throttle_out_loss: 0.2440
Epoch 9/100
19/20 [===========================>..] - ETA: 0s - loss: 0.6130 - angle_out_loss: 0.6784 - throttle_out_loss: 0.2462
Epoch 00009: val_loss did not improve from 0.85174
20/20 [==============================] - 5s 258ms/step - loss: 0.6079 - angle_out_loss: 0.6728 - throttle_out_loss: 0.2462 - val_loss: 0.9572 - val_angle_out_loss: 1.0609 - val_throttle_out_loss: 0.2418
Epoch 10/100
19/20 [===========================>..] - ETA: 0s - loss: 0.5615 - angle_out_loss: 0.6212 - throttle_out_loss: 0.2436
Epoch 00010: val_loss did not improve from 0.85174
20/20 [==============================] - 5s 255ms/step - loss: 0.5630 - angle_out_loss: 0.6229 - throttle_out_loss: 0.2435 - val_loss: 0.9913 - val_angle_out_loss: 1.0988 - val_throttle_out_loss: 0.2400
Epoch 11/100
19/20 [===========================>..] - ETA: 0s - loss: 0.5203 - angle_out_loss: 0.5754 - throttle_out_loss: 0.2447
Epoch 00011: val_loss improved from 0.85174 to 0.82425, saving model to models/mypilot.h5
20/20 [==============================] - 5s 265ms/step - loss: 0.5158 - angle_out_loss: 0.5703 - throttle_out_loss: 0.2451 - val_loss: 0.8242 - val_angle_out_loss: 0.9132 - val_throttle_out_loss: 0.2378
Epoch 12/100
19/20 [===========================>..] - ETA: 0s - loss: 0.4146 - angle_out_loss: 0.4579 - throttle_out_loss: 0.2518
Epoch 00012: val_loss did not improve from 0.82425
20/20 [==============================] - 5s 257ms/step - loss: 0.4151 - angle_out_loss: 0.4584 - throttle_out_loss: 0.2515 - val_loss: 1.0301 - val_angle_out_loss: 1.1419 - val_throttle_out_loss: 0.2402
Epoch 13/100
19/20 [===========================>..] - ETA: 0s - loss: 0.3793 - angle_out_loss: 0.4188 - throttle_out_loss: 0.2400
Epoch 00013: val_loss did not improve from 0.82425
20/20 [==============================] - 5s 260ms/step - loss: 0.3852 - angle_out_loss: 0.4254 - throttle_out_loss: 0.2399 - val_loss: 0.9044 - val_angle_out_loss: 1.0022 - val_throttle_out_loss: 0.2393
Epoch 14/100
19/20 [===========================>..] - ETA: 0s - loss: 0.3562 - angle_out_loss: 0.3932 - throttle_out_loss: 0.2391
Epoch 00014: val_loss did not improve from 0.82425
20/20 [==============================] - 5s 258ms/step - loss: 0.3517 - angle_out_loss: 0.3881 - throttle_out_loss: 0.2393 - val_loss: 0.9473 - val_angle_out_loss: 1.0499 - val_throttle_out_loss: 0.2421
Epoch 15/100
19/20 [===========================>..] - ETA: 0s - loss: 0.3180 - angle_out_loss: 0.3507 - throttle_out_loss: 0.2434
Epoch 00015: val_loss did not improve from 0.82425
20/20 [==============================] - 5s 255ms/step - loss: 0.3207 - angle_out_loss: 0.3537 - throttle_out_loss: 0.2430 - val_loss: 1.1386 - val_angle_out_loss: 1.2624 - val_throttle_out_loss: 0.2394
Epoch 16/100
19/20 [===========================>..] - ETA: 0s - loss: 0.3302 - angle_out_loss: 0.3642 - throttle_out_loss: 0.2413
Epoch 00016: val_loss did not improve from 0.82425
20/20 [==============================] - 5s 261ms/step - loss: 0.3250 - angle_out_loss: 0.3585 - throttle_out_loss: 0.2412 - val_loss: 1.3519 - val_angle_out_loss: 1.4994 - val_throttle_out_loss: 0.2426
Epoch 00016: early stopping
!cd d2/data/tub_18 && cat record_2427.json
{"user/mode": "user", "cam/image_array": "2427_cam-image_array_.jpg", "user/angle": -0.05310220648823512, "user/throttle": 0.25}
7. 在小车上运行自动驾驶模型
通过ssh连接到小车的树莓派3上，并将模型文件mypilot.h5从高性能实例上下载到小车d2工程中
小车系统同样需要进行步骤1~4操作
用自动驾驶模型启动小车，并在赛道中自动驾驶
!cd d2 && python3 manage.py drive --model=mypilot.h5
8. 在TensorBoard上查看训练过程
数据流图
损失值
9. 其他
- 注意小车的tensorflow版本需与训练模型的GPU实例所安装的tensorflow版本保持一致。