基于深度学习的交通视频分析系统的设计与实现

哈尔滨工业大学工程硕士学位论文

Abstract

In recent years, with more and more cars entering ordinary families, private cars occupy an increasingly important position in people's daily life. At the same time, with the rapid increase of the number of cars, more and more illegal activities related to the car, so the structural analysis of traffic video is an urgent need in the field of security. With the rapid development of artificial intelligence technology, more and more attention has been paid to the field of intelligent security. As an important part of the security field, traffic video analysis has also attracted much attention.

In the early years, due to the immature technology, video analysis of complex scenes has been difficult to achieve. However, with the emergence of deep learning, the video structuring of this large-scale complex scene becomes possible. At present, deep learning has been widely used in various fields of research and application, such as image recognition, speech recognition, and natural language process. Traffic video structure mainly relies on the intelligent analysis of monitoring video, and extracts basic information of the vehicle from the video, including the vehicle color, brand, license plate and other information. The extracted information will be saved to the database for later analysis and effective information mining.

In the process of traffic video, vehicle and license plate detection, license plate recognition, vehicle brand identification and target tracking are involved. In this paper, a series of methods are proposed to solve the problem of image recognition in the process of structuring traffic video with the latest techniques in target detection, OCR, image classification and target tracking. At the same time, in the process of developing the system, a framework for training and reasoning of deep learning models has been developed.

In view of vehicle detection and license plate detection in traffic video analysis, a fast target detection algorithm based on deep convolution neural network is used in this paper. The algorithm combines the existing one stage detection algorithm and uses the IOU loss instead of the traditional L1 loss. In view of the recognition of the character and color of the license plate, this paper combines the deep convolution network, the CTC loss function and the cross entropy, and completes the recognition of the character and color simultaneously in a single frame. In view of the recognition of vehicle color and property, this paper uses a deep convolution neural network, focal loss and cross entropy to complete the classification of vehicle brand and color in a single frame. For multi target vehicle tracking, this paper combines detection results and Kalman filter, and tracks vehicle trajectory according to various attributes, and

哈尔滨工业大学工程硕士学位论文

obtains real-time tracking effect. At the same time, the deep learning framework developed in this paper uses GPU in the training process, and GPU and AI chips are used in the inference process.

After the test of the actual environment, the algorithm proposed in this paper has achieved a precision of more than 0.9 map in the search of the vehicle and the license plate. The recognition algorithm of the character and color of the license plate has obtained the precision of the character 98.7% and the color 99.5%, and the recognition of the brand and color of the vehicle also obtained the accuracy of 90% and 96%.

Keywords:deep learning, vehicle detection, license plate detection, license plate recognition, vehicle brand recognition.

哈尔滨工业大学工程硕士学位论文

摘要 .......................................................................................................................... I ABSTRACT ................................................................................................................ II 第1章绪论 .. (1)

1.1课题背景及研究目的 (1)

1.2与课题相关的国内外研究综述 (3)

1.3本文的主要研究内容及论文结构 (10)

第2章通视频分析系统的需求分析和总体设计 (11)

2.1交通视频分析系统的需求分析 (11)

2.2功能需求 (12)

2.2.1车辆检测 (12)

2.2.2车牌检测 (12)

2.2.3车牌识别 (13)

2.2.4车辆品牌和颜色的识别 (13)

2.2.5车辆的轨迹跟踪 (13)

2.2.6深度学习算法训练及推理 (13)

2.3非功能需求 (17)

2.3.1性能需求 (17)

2.3.2速度需求 (18)

2.4系统的总体设计 (18)

2.4.1系统总体架构 (18)

2.4.2深度学习算法训练及推理的功能结构图 (19)

2.4.3深度学习训练及推理框架的主要功能活动 (19)

2.5本章小结 (20)

第3章交通视频分析系统的设计与实现 (21)

3.1车辆检测及车牌检测功能的详细设计 (21)

3.1.1交并比损失(IOU Loss) (21)

3.1.2正负样本的选择 (24)

3.1.3两阶段训练 (24)

3.1.4结果评价 (25)

3.1.4交并比损失的具体实现 (25)

哈尔滨工业大学工程硕士学位论文

3.2车牌识别功能的详细设计 (26)

3.2.1连接时序分类 (26)

3.2.2输出标记 (27)

3.2.3前向后向算法 (28)

3.2.4损失函数 (30)

3.2.5解码算法 (32)

3.2.6结果评价 (33)

3.2.7车牌识别网络结构 (33)

3.2.8 CTC损失具体实现 (34)

3.4车辆品牌颜色识别功能的详细设计 (34)

3.4.1焦点损失(focal loss) (35)

3.4.2车牌品牌颜色识别的网络结构 (36)

3.5目标跟踪的详细设计 (36)

3.5.1卡尔曼滤波器 (37)

3.5.2卡尔曼滤波器流程图 (37)

3.5.3车辆跟踪流程图 (38)

3.6深度学习训练及推理框架的详细设计和实现 (40)

3.6.1配置文件解析模块的设计与实现 (40)

3.6.2训练数据管理模块的设计与实现 (44)

3.6.3网络结构模块的设计与实现 (45)

3.6.4 AI芯片上的深度学习算法推理优化策略及实现 (47)

3.7本章小结 (50)

第4章交通视频分析系统测试 (51)

4.1车辆检测模型训练和测试 (51)

4.1.1实验数据和网络设置 (51)

4.1.2参数设置及训练过程 (52)

4.1.3测试及结果分析 (54)

4.2车牌检测模型训练 (55)

4.2.1实验数据和网络设置 (55)

4.2.2参数设置和训练过程 (56)

4.2.3测试及结果分析 (57)

4.3车牌识别 (58)

4.3.1实验数据和网络设置 (58)

4.3.2参数设置和训练过程 (60)

哈尔滨工业大学工程硕士学位论文

4.3.3测试及结果分析 (61)

4.4车辆品牌识别 (62)

4.4.1实验数据和网络设置 (62)

4.4.2参数设置和训练过程 (63)

4.4.3测试及结果分析 (64)

4.5目标跟踪 (65)

4.5.1实验数据 (65)

4.5.2实验过程及结果 (65)

4.6本章小结 (65)

结论 (67)

参考文献 (68)

哈尔滨工业大学学位论文原创性声明和使用权限 (71)

致谢 (72)

个人简历 (73)