Lanenet车道线检测 --运行别人代码（1）

2018.2发表出来的，文章下载地址：https://arxiv.org/abs/1802.05591

github上代码：https://github.com/MaybeShewill-CV/lanenet-lane-detection

【本文用的还是18年的，没有用最新的】

---

运行代码时可能出错:

处理：

程序里添加路径

import sys 
sys.path.append('/home/zengjun/下载/lanenet-lane-detection-master')
sys.path.append('/home/zengjun/下载/lanenet-lane-detection-master/config')
sys.path.append('/home/zengjun/下载/lanenet-lane-detection-masterdata_provider')
sys.path.append('/home/zengjun/下载/lanenet-lane-detection-master/lanenet_model')
sys.path.append('/home/zengjun/下载/lanenet-lane-detection-master/encoder_decoder_model')
sys.path.append('/home/zengjun/下载/lanenet-lane-detection-master/tools')

对应文件夹里加入一个空的 __init__.py文件

遇到库文件比如cv2等问题，在pycharm里使用更为方便。本文用的python3.5

测试结果：

2019-05-30 20:47:00.556062: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4861 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1060 6GB, pci bus id: 0000:01:00.0, compute capability: 6.1)
INFO:tensorflow:Restoring parameters from model/tusimple_lanenet/tusimple_lanenet_vgg_2018-10-19-13-33-56.ckpt-200000
I0530 20:47:00.557337 7261 tf_logging.py:115] Restoring parameters from model/tusimple_lanenet/tusimple_lanenet_vgg_2018-10-19-13-33-56.ckpt-200000
I0530 20:47:01.835264 7261 test_lanenet.py:127] 单张图像车道线预测耗时: 1.20088s

预测多张，FPS应该会提高，因为预处理时间。

python tools/test_lanenet.py --is_batch True --batch_size 2 --save_dir data/tusimple_test_image/ret  --weights_path model/tusimple_lanenet/tusimple_lanenet_vgg_2018-10-19-13-33-56.ckpt-200000  --image_path data/tusimple_test_image/

I0530 21:02:25.781537 9119 test_lanenet.py:208] [Epoch:2] 预处理2张图像, 共耗时: 0.01650s, 平均每张耗时: 0.00825
I0530 21:02:25.835193 9119 test_lanenet.py:215] [Epoch:2] 预测2张图像车道线, 共耗时: 0.05352s, 平均每张耗时: 0.02676s
I0530 21:02:25.957873 9119 test_lanenet.py:245] [Epoch:2] 进行2张图像车道线聚类, 共耗时: 0.10769s, 平均每张耗时: 0.05384
I0530 21:02:25.957981 9119 test_lanenet.py:198] [Epoch:3] 开始图像读取和预处理...
I0530 21:02:25.975388 9119 test_lanenet.py:208] [Epoch:3] 预处理2张图像, 共耗时: 0.01733s, 平均每张耗时: 0.00866
I0530 21:02:26.029834 9119 test_lanenet.py:215] [Epoch:3] 预测2张图像车道线, 共耗时: 0.05430s, 平均每张耗时: 0.02715s
I0530 21:02:26.092286 9119 test_lanenet.py:245] [Epoch:3] 进行2张图像车道线聚类, 共耗时: 0.04747s, 平均每张耗时: 0.02374
I0530 21:02:26.092393 9119 test_lanenet.py:198] [Epoch:4] 开始图像读取和预处理...
I0530 21:02:26.105244 9119 test_lanenet.py:208] [Epoch:4] 预处理2张图像, 共耗时: 0.01278s, 平均每张耗时: 0.00639
I0530 21:02:26.158836 9119 test_lanenet.py:215] [Epoch:4] 预测2张图像车道线, 共耗时: 0.05346s, 平均每张耗时: 0.02673s
I0530 21:02:26.210763 9119 test_lanenet.py:245] [Epoch:4] 进行2张图像车道线聚类, 共耗时: 0.04204s, 平均每张耗时: 0.02102
I0530 21:02:26.210871 9119 test_lanenet.py:198] [Epoch:5] 开始图像读取和预处理...
I0530 21:02:26.227664 9119 test_lanenet.py:208] [Epoch:5] 预处理2张图像, 共耗时: 0.01670s, 平均每张耗时: 0.00835
I0530 21:02:26.280578 9119 test_lanenet.py:215] [Epoch:5] 预测2张图像车道线, 共耗时: 0.05277s, 平均每张耗时: 0.02638s
I0530 21:02:26.344519 9119 test_lanenet.py:245] [Epoch:5] 进行2张图像车道线聚类, 共耗时: 0.04929s, 平均每张耗时: 0.02465

---

这下面部分运行除了问题，不要看：

自己加了个视频处理显示的py

#video_test.py   可以读取视频与处理后显示
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# @Time    : 18-5-23 上午11:33
# @Author  : Luo Yao
# @Site    : https://github.com/MaybeShewill-CV/lanenet-lane-detection
# @File    : test_lanenet.py
# @IDE: PyCharm Community Edition
"""
测试LaneNet模型
"""
import sys
ros_path = '/opt/ros/kinetic/lib/python2.7/dist-packages'
if ros_path in sys.path:

    sys.path.remove(ros_path)

import cv2

sys.path.append('/opt/ros/kinetic/lib/python2.7/dist-packages')
sys.path.append('/home/zengjun/下载/lanenet-lane-detection-master')
sys.path.append('/home/zengjun/下载/lanenet-lane-detection-master/config')
sys.path.append('/home/zengjun/下载/lanenet-lane-detection-masterdata_provider')
sys.path.append('/home/zengjun/下载/lanenet-lane-detection-master/lanenet_model')
sys.path.append('/home/zengjun/下载/lanenet-lane-detection-master/encoder_decoder_model')
sys.path.append('/home/zengjun/下载/lanenet-lane-detection-master/tools')

import os
import os.path as ops
import argparse
import time
import math

import tensorflow as tf
import glob
import glog as log
import numpy as np
import matplotlib.pyplot as plt
import cv2

from lanenet_model import lanenet_merge_model
from lanenet_model import lanenet_cluster
from lanenet_model import lanenet_postprocess
from config import global_config

CFG = global_config.cfg
VGG_MEAN = [103.939, 116.779, 123.68]


def init_args():
    """

    :return:
    """
    parser = argparse.ArgumentParser()
    parser.add_argument('--image_path', type=str, help='The image path or the src image save dir')
    parser.add_argument('--weights_path', type=str, help='The model weights path')
    parser.add_argument('--is_batch', type=str, help='If test a batch of images', default='false')
    parser.add_argument('--batch_size', type=int, help='The batch size of the test images', default=32)
    parser.add_argument('--save_dir', type=str, help='Test result image save dir', default=None)
    parser.add_argument('--use_gpu', type=int, help='If use gpu set 1 or 0 instead', default=1)

    return parser.parse_args()


def minmax_scale(input_arr):
    """

    :param input_arr:
    :return:
    """
    min_val = np.min(input_arr)
    max_val = np.max(input_arr)

    output_arr = (input_arr - min_val) * 255.0 / (max_val - min_val)

    return output_arr

'''
def test_lanenet(image_path, weights_path, use_gpu):
    """

    :param image_path:
    :param weights_path:
    :param use_gpu:
    :return:
    """
    assert ops.exists(image_path), '{:s} not exist'.format(image_path)

    log.info('开始读取图像数据并进行预处理')
    t_start = time.time()
    image = cv2.imread(image_path, cv2.IMREAD_COLOR)
    image_vis = image
    image = cv2.resize(image, (512, 256), interpolation=cv2.INTER_LINEAR)
    image = image - VGG_MEAN
    log.info('图像读取完毕, 耗时: {:.5f}s'.format(time.time() - t_start))

    input_tensor = tf.placeholder(dtype=tf.float32, shape=[1, 256, 512, 3], name='input_tensor')
    phase_tensor = tf.constant('test', tf.string)

    net = lanenet_merge_model.LaneNet(phase=phase_tensor, net_flag='vgg')
    binary_seg_ret, instance_seg_ret = net.inference(input_tensor=input_tensor, name='lanenet_model')

    cluster = lanenet_cluster.LaneNetCluster()
    postprocessor = lanenet_postprocess.LaneNetPoseProcessor()

    saver = tf.train.Saver()

    # Set sess configuration
    if use_gpu:
        sess_config = tf.ConfigProto(device_count={'GPU': 1})
    else:
        sess_config = tf.ConfigProto(device_count={'CPU': 0})
    sess_config.gpu_options.per_process_gpu_memory_fraction = CFG.TEST.GPU_MEMORY_FRACTION
    sess_config.gpu_options.allow_growth = CFG.TRAIN.TF_ALLOW_GROWTH
    sess_config.gpu_options.allocator_type = 'BFC'

    sess = tf.Session(config=sess_config)

    with sess.as_default():

        saver.restore(sess=sess, save_path=weights_path)

        t_start = time.time()
        binary_seg_image, instance_seg_image = sess.run([binary_seg_ret, instance_seg_ret],
                                                        feed_dict={input_tensor: [image]})
        t_cost = time.time() - t_start
        log.info('单张图像车道线预测耗时: {:.5f}s'.format(t_cost))

        binary_seg_image[0] = postprocessor.postprocess(binary_seg_image[0])
        mask_image = cluster.get_lane_mask(binary_seg_ret=binary_seg_image[0],
                                           instance_seg_ret=instance_seg_image[0])

        for i in range(4):
            instance_seg_image[0][:, :, i] = minmax_scale(instance_seg_image[0][:, :, i])
        embedding_image = np.array(instance_seg_image[0], np.uint8)

        plt.figure('mask_image')
        plt.imshow(mask_image[:, :, (2, 1, 0)])
        plt.figure('src_image')
        plt.imshow(image_vis[:, :, (2, 1, 0)])
        plt.figure('instance_image')
        plt.imshow(embedding_image[:, :, (2, 1, 0)])
        plt.figure('binary_image')
        plt.imshow(binary_seg_image[0] * 255, cmap='gray')
        plt.show()

    sess.close()

    return
'''

##加入视频转换
#def detect_video(yolo, video_path, output_path=""):
def test_lanenet(video_path, weights_path, use_gpu,output_path=''):
#def detect_video(yolo, output_path=""):
    import cv2
    from timeit import default_timer as timer
    from PIL import Image, ImageFont, ImageDraw

    print(video_path)
    vid = cv2.VideoCapture(video_path)
    #vid = cv2.VideoCapture(0)
    if not vid.isOpened():
        raise IOError("Couldn't open webcam or video")
    video_FourCC    = int(vid.get(cv2.CAP_PROP_FOURCC))
    video_fps       = vid.get(cv2.CAP_PROP_FPS)
    video_size      = (int(vid.get(cv2.CAP_PROP_FRAME_WIDTH)),
                        int(vid.get(cv2.CAP_PROP_FRAME_HEIGHT)))
    isOutput = True if output_path != "" else False
    if isOutput:
        print("!!! TYPE:", type(output_path), type(video_FourCC), type(video_fps), type(video_size))
        out = cv2.VideoWriter(output_path, video_FourCC, video_fps, video_size)
    accum_time = 0
    curr_fps = 0
    fps = "FPS: ??"
    prev_time = timer()
    while True:

        return_value, frame = vid.read()
        tf.reset_default_graph()
        image = Image.fromarray(frame)
        image = cv2.cvtColor(np.asarray(image), cv2.COLOR_RGB2BGR)
        log.info('开始读取图像数据并进行预处理')
        t_start = time.time()
        #image = cv2.imread(image_path, cv2.IMREAD_COLOR)
        image_vis = image
        image = cv2.resize(image, (512, 256), interpolation=cv2.INTER_LINEAR)
        image = image - VGG_MEAN
        log.info('图像读取完毕, 耗时: {:.5f}s'.format(time.time() - t_start))

        input_tensor = tf.placeholder(dtype=tf.float32, shape=[1, 256, 512, 3], name='input_tensor')
        phase_tensor = tf.constant('test', tf.string)

        net = lanenet_merge_model.LaneNet(phase=phase_tensor, net_flag='vgg')
        #tf.reset_default_graph()  # zj添加 参考网址：https://blog.csdn.net/mr_brooks/article/details/80393396
        binary_seg_ret, instance_seg_ret = net.inference(input_tensor=input_tensor, name='lanenet_model')

        cluster = lanenet_cluster.LaneNetCluster()
        postprocessor = lanenet_postprocess.LaneNetPoseProcessor()

        saver = tf.train.Saver()

        # Set sess configuration
        if use_gpu:
            sess_config = tf.ConfigProto(device_count={'GPU': 1})
        else:
            sess_config = tf.ConfigProto(device_count={'CPU': 0})
        sess_config.gpu_options.per_process_gpu_memory_fraction = CFG.TEST.GPU_MEMORY_FRACTION
        sess_config.gpu_options.allow_growth = CFG.TRAIN.TF_ALLOW_GROWTH
        sess_config.gpu_options.allocator_type = 'BFC'

        sess = tf.Session(config=sess_config)

        with sess.as_default():

            saver.restore(sess=sess, save_path=weights_path)

            t_start = time.time()
            binary_seg_image, instance_seg_image = sess.run([binary_seg_ret, instance_seg_ret],
                                                            feed_dict={input_tensor: [image]})
            t_cost = time.time() - t_start
            log.info('单张图像车道线预测耗时: {:.5f}s'.format(t_cost))

            binary_seg_image[0] = postprocessor.postprocess(binary_seg_image[0])
            mask_image = cluster.get_lane_mask(binary_seg_ret=binary_seg_image[0],
                                               instance_seg_ret=instance_seg_image[0])

            for i in range(4):
                instance_seg_image[0][:, :, i] = minmax_scale(instance_seg_image[0][:, :, i])
            embedding_image = np.array(instance_seg_image[0], np.uint8)
            '''
            plt.figure('mask_image')
            plt.imshow(mask_image[:, :, (2, 1, 0)])
            plt.figure('src_image')
            plt.imshow(image_vis[:, :, (2, 1, 0)])
            plt.figure('instance_image')
            plt.imshow(embedding_image[:, :, (2, 1, 0)])
            plt.figure('binary_image')
            plt.imshow(binary_seg_image[0] * 255, cmap='gray')
            plt.show()
            '''
            result = np.asarray(embedding_image[:, :, (2, 1, 0)])
            curr_time = timer()
            exec_time = curr_time - prev_time
            prev_time = curr_time
            accum_time = accum_time + exec_time
            curr_fps = curr_fps + 1
            if accum_time > 1:
                accum_time = accum_time - 1
                fps = "FPS: " + str(curr_fps)
                curr_fps = 0
            cv2.putText(result, text=fps, org=(3, 15), fontFace=cv2.FONT_HERSHEY_SIMPLEX,
                       fontScale=0.50, color=(255, 0, 0), thickness=2)
            cv2.namedWindow("result", cv2.WINDOW_NORMAL)
            cv2.imshow("result", result)
            if isOutput:
                out.write(result)
            if cv2.waitKey(1) & 0xFF == ord('q'):
                break
        sess.close()
    return 






def test_lanenet_batch(image_dir, weights_path, batch_size, use_gpu, save_dir=None):
    """

    :param image_dir:
    :param weights_path:
    :param batch_size:
    :param use_gpu:
    :param save_dir:
    :return:
    """
    assert ops.exists(image_dir), '{:s} not exist'.format(image_dir)

    log.info('开始获取图像文件路径...')
    image_path_list = glob.glob('{:s}/**/*.jpg'.format(image_dir), recursive=True) + \
                      glob.glob('{:s}/**/*.png'.format(image_dir), recursive=True) + \
                      glob.glob('{:s}/**/*.jpeg'.format(image_dir), recursive=True)

    input_tensor = tf.placeholder(dtype=tf.float32, shape=[None, 256, 512, 3], name='input_tensor')
    phase_tensor = tf.constant('test', tf.string)

    net = lanenet_merge_model.LaneNet(phase=phase_tensor, net_flag='vgg')
    binary_seg_ret, instance_seg_ret = net.inference(input_tensor=input_tensor, name='lanenet_model')

    cluster = lanenet_cluster.LaneNetCluster()
    postprocessor = lanenet_postprocess.LaneNetPoseProcessor()

    saver = tf.train.Saver()

    # Set sess configuration
    if use_gpu:
        sess_config = tf.ConfigProto(device_count={'GPU': 1})
    else:
        sess_config = tf.ConfigProto(device_count={'GPU': 0})
    sess_config.gpu_options.per_process_gpu_memory_fraction = CFG.TEST.GPU_MEMORY_FRACTION
    sess_config.gpu_options.allow_growth = CFG.TRAIN.TF_ALLOW_GROWTH
    sess_config.gpu_options.allocator_type = 'BFC'

    sess = tf.Session(config=sess_config)

    with sess.as_default():

        saver.restore(sess=sess, save_path=weights_path)

        epoch_nums = int(math.ceil(len(image_path_list) / batch_size))

        for epoch in range(epoch_nums):
            log.info('[Epoch:{:d}] 开始图像读取和预处理...'.format(epoch))
            t_start = time.time()
            image_path_epoch = image_path_list[epoch * batch_size:(epoch + 1) * batch_size]
            image_list_epoch = [cv2.imread(tmp, cv2.IMREAD_COLOR) for tmp in image_path_epoch]
            image_vis_list = image_list_epoch
            image_list_epoch = [cv2.resize(tmp, (512, 256), interpolation=cv2.INTER_LINEAR)
                                for tmp in image_list_epoch]
            image_list_epoch = [tmp - VGG_MEAN for tmp in image_list_epoch]
            t_cost = time.time() - t_start
            log.info('[Epoch:{:d}] 预处理{:d}张图像, 共耗时: {:.5f}s, 平均每张耗时: {:.5f}'.format(
                epoch, len(image_path_epoch), t_cost, t_cost / len(image_path_epoch)))

            t_start = time.time()
            binary_seg_images, instance_seg_images = sess.run(
                [binary_seg_ret, instance_seg_ret], feed_dict={input_tensor: image_list_epoch})
            t_cost = time.time() - t_start
            log.info('[Epoch:{:d}] 预测{:d}张图像车道线, 共耗时: {:.5f}s, 平均每张耗时: {:.5f}s'.format(
                epoch, len(image_path_epoch), t_cost, t_cost / len(image_path_epoch)))

            cluster_time = []
            for index, binary_seg_image in enumerate(binary_seg_images):
                t_start = time.time()
                binary_seg_image = postprocessor.postprocess(binary_seg_image)
                mask_image = cluster.get_lane_mask(binary_seg_ret=binary_seg_image,
                                                   instance_seg_ret=instance_seg_images[index])
                cluster_time.append(time.time() - t_start)
                mask_image = cv2.resize(mask_image, (image_vis_list[index].shape[1],
                                                     image_vis_list[index].shape[0]),
                                        interpolation=cv2.INTER_LINEAR)

                if save_dir is None:
                    plt.ion()
                    plt.figure('mask_image')
                    plt.imshow(mask_image[:, :, (2, 1, 0)])
                    plt.figure('src_image')
                    plt.imshow(image_vis_list[index][:, :, (2, 1, 0)])
                    plt.pause(3.0)
                    plt.show()
                    plt.ioff()

                if save_dir is not None:
                    mask_image = cv2.addWeighted(image_vis_list[index], 1.0, mask_image, 1.0, 0)
                    image_name = ops.split(image_path_epoch[index])[1]
                    image_save_path = ops.join(save_dir, image_name)
                    cv2.imwrite(image_save_path, mask_image)

            log.info('[Epoch:{:d}] 进行{:d}张图像车道线聚类, 共耗时: {:.5f}s, 平均每张耗时: {:.5f}'.format(
                epoch, len(image_path_epoch), np.sum(cluster_time), np.mean(cluster_time)))

    sess.close()

    return


if __name__ == '__main__':
    # init args
    '''
    args = init_args()

    if args.save_dir is not None and not ops.exists(args.save_dir):
        log.error('{:s} not exist and has been made'.format(args.save_dir))
        os.makedirs(args.save_dir)

    if args.is_batch.lower() == 'false':
        # test hnet model on single image
        test_lanenet(args.video_path, args.weights_path, args.use_gpu)
    else:
        # test hnet model on a batch of image
        test_lanenet_batch(image_dir=args.image_path, weights_path=args.weights_path,
                           save_dir=args.save_dir, use_gpu=args.use_gpu, batch_size=args.batch_size)
    '''
    '''
    python tools/vedio_test.py --is_batch False  --weights_path model/tusimple_lanenet/tusimple_lanenet_vgg_2018-10-19-13-33-56.ckpt-200000 --video_path /home/zengjun/下载/lanenet-lane-detection-master/laneSource.mp4 
    python tools/vedio_test.py --is_batch False --batch_size 1  --video_path laneSource.mp4  --weights_path path/to/your/model_weights_file  
    '''
    test_lanenet('/home/zengjun/下载/lanenet-lane-detection-master/laneSource.mp4','/home/zengjun/下载/lanenet-lane-detection-master/model/tusimple_lanenet/tusimple_lanenet_vgg_2018-10-19-13-33-56.ckpt-200000', 1)