RK3568笔记二十八:CRNN车牌识别部署

news/2024/7/16 16:45:24

若该文为原创文章,转载请注明原文出处。

想在RK3568上实现车牌识别,想到的方法是使用LPRNet网络识别或CRNN识别,本篇记录使用CRNN识别车牌,也可以换成LPRNet模型,原理一样。

一、平台介绍

1、训练平台:Autodl

2、运行板子:ATK-DLRK3568

二、环境搭建

使用Autodl平台的GPU训练测试

1、创建虚拟环境

conda create -n cnn_plate_env python=3.8

2、激活

conda activate cnn_plate_env

3、安装依赖项

pip install easydict -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install pyyaml -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install opencv-python -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install tensorboardX -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install imgaug -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install torch -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install torchvision -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install tensorboardX -i https://pypi.tuna.tsinghua.edu.cn/simple

4、下载源码

# 源码地址
https://github.com/we0091234/crnn_plate_recognition

5、准备数据集

车牌识别数据集CCPD+CRPD

  1. 从CCPD和CRPD截下来的车牌小图以及我自己收集的一部分车牌 

  2. 数据集打上标签,生成train.txt和val.txt

如有需要,联系博主。

然后执行如下命令,得到train.txt和val.txt

python plateLabel.py --image_path your/train/img/path/ --label_file datasets/train.txt
python plateLabel.py --image_path your/val/img/path/ --label_file datasets/val.txt

数据格式如下:

6、修改配置文件

将train.txt val.txt路径写入lib/config/360CC_config.yaml 中

JSON_FILE: {'train': '/root/crnn_plate_recognition/datasets/train.txt', 'val': '/root/crnn_plate_recognition/datasets/val.txt'}

注意路径

7、修改trian.py

由于安装的ymal版本过高,会导致无法加载。所以需要修改加载方式。

出错:TypeError: load() missing 1 required positional argument: 'Loader'

处理:TypeError: load() missing 1 required positional argument: ‘Loader‘-CSDN博客

8、训练

 python train.py --cfg lib/config/360CC_config.yaml

默认100轮,计时2-3小时,结果保存再output文件夹中

9、测试

python demo.py --model_path saved_model/best.pth --image_path images/test.jpg

三、导出ONNX并推理测试

导出需要安装onnx

pip install onnx
pip install onnxsim

1、导出ONNX

python export.py --weights saved_model/best.pth --save_path saved_model/best.onnx  --simplify

2、推理测试

python onnx_infer.py --onnx_file saved_model/best.onnx  --image_path images/test.jpg

四、转成RKNN

转成RKNN和部署需要用到rknn-toolkit2-1.5.0

把训练转换后的onn拷贝到同级目录下,执行下面转换程序,即可导出RKNN

recongniton_cov_rknn.py

import numpy as np
import cv2
from rknn.api import RKNN
from PIL import Image, ImageDraw, ImageFontif __name__ == '__main__':# Create RKNN objectrknn2 = RKNN(verbose=False)# pre-process configprint('--> Config model')rknn2.config(mean_values=[[150.54, 150.54, 150.54]], std_values=[[49.215, 49.215, 49.215]], target_platform="rk3568")print('done')# Load ONNX modelprint('--> Loading model')ret = rknn2.load_onnx(model="./best.onnx")if ret != 0:print('Load model failed!')exit(ret)print('done')# Build modelprint('--> Building model')ret = rknn2.build(do_quantization=False)if ret != 0:print('Build model failed!')exit(ret)print('done')# Export RKNN modelprint('--> Export rknn model')ret = rknn2.export_rknn("./best.rknn")if ret != 0:print('Export rknn model failed!')exit(ret)print('done')# Init runtime environmentprint('--> Init runtime environment')ret = rknn2.init_runtime()if ret != 0:print('Init runtime environment failed!')exit(ret)print('done')rknn2.release()

五、部署测试

把转换后的RKNN模型拷贝到虚拟机内。

测试RKNN模型

import numpy as np
import cv2
from rknn.api import RKNN
from PIL import Image, ImageDraw, ImageFontplate_chr = "#京沪津渝冀晋蒙辽吉黑苏浙皖闽赣鲁豫鄂湘粤桂琼川贵云藏陕甘青宁新学警港澳挂使领民航危0123456789ABCDEFGHJKLMNPQRSTUVWXYZ险品"OBJ_THRESH = 0.25
NMS_THRESH = 0.45
IMG_SIZE = 640CLASSES = ("plate", )def decodePlate(preds):  # 识别后处理  pre = 0newPreds = []for i in range(len(preds[0])):if (preds[0][i] != 0).all() and (preds[0][i] != pre).all():newPreds.append(preds[0][i])pre = preds[0][i]plate = ""for i in newPreds:plate += plate_chr[int(i)]print(plate)return platedef cv2ImgAddText(img, text, left, top, textColor=(0, 255, 0), textSize=20):if (isinstance(img, np.ndarray)):  # 判断是否OpenCV图片类型img = Image.fromarray(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))# 创建一个可以在给定图像上绘图的对象draw = ImageDraw.Draw(img)# 字体的格式   fontStyle = ImageFont.truetype("simsun.ttc", textSize, encoding="utf-8")# 绘制文本draw.text((left, top), text, textColor, font=fontStyle)# 转换回OpenCV格式return cv2.cvtColor(np.asarray(img), cv2.COLOR_RGB2BGR)def sigmoid(x):# return 1 / (1 + np.exp(-x))return xdef xywh2xyxy(x):# Convert [x, y, w, h] to [x1, y1, x2, y2]y = np.copy(x)y[:, 0] = x[:, 0] - x[:, 2] / 2  # top left xy[:, 1] = x[:, 1] - x[:, 3] / 2  # top left yy[:, 2] = x[:, 0] + x[:, 2] / 2  # bottom right xy[:, 3] = x[:, 1] + x[:, 3] / 2  # bottom right yreturn ydef process(input, mask, anchors):anchors = [anchors[i] for i in mask]grid_h, grid_w = map(int, input.shape[0:2])box_confidence = sigmoid(input[..., 4])box_confidence = np.expand_dims(box_confidence, axis=-1)box_class_probs = sigmoid(input[..., 5:])box_xy = sigmoid(input[..., :2])*2 - 0.5col = np.tile(np.arange(0, grid_w), grid_w).reshape(-1, grid_w)row = np.tile(np.arange(0, grid_h).reshape(-1, 1), grid_h)col = col.reshape(grid_h, grid_w, 1, 1).repeat(3, axis=-2)row = row.reshape(grid_h, grid_w, 1, 1).repeat(3, axis=-2)grid = np.concatenate((col, row), axis=-1)box_xy += gridbox_xy *= int(IMG_SIZE/grid_h)box_wh = pow(sigmoid(input[..., 2:4])*2, 2)box_wh = box_wh * anchorsbox = np.concatenate((box_xy, box_wh), axis=-1)return box, box_confidence, box_class_probsdef filter_boxes(boxes, box_confidences, box_class_probs):"""Filter boxes with box threshold. It's a bit different with origin yolov5 post process!# Argumentsboxes: ndarray, boxes of objects.box_confidences: ndarray, confidences of objects.box_class_probs: ndarray, class_probs of objects.# Returnsboxes: ndarray, filtered boxes.classes: ndarray, classes for boxes.scores: ndarray, scores for boxes."""boxes = boxes.reshape(-1, 4)box_confidences = box_confidences.reshape(-1)box_class_probs = box_class_probs.reshape(-1, box_class_probs.shape[-1])_box_pos = np.where(box_confidences >= OBJ_THRESH)boxes = boxes[_box_pos]box_confidences = box_confidences[_box_pos]box_class_probs = box_class_probs[_box_pos]class_max_score = np.max(box_class_probs, axis=-1)classes = np.argmax(box_class_probs, axis=-1)_class_pos = np.where(class_max_score >= OBJ_THRESH)boxes = boxes[_class_pos]classes = classes[_class_pos]scores = (class_max_score* box_confidences)[_class_pos]return boxes, classes, scoresdef nms_boxes(boxes, scores):"""Suppress non-maximal boxes.# Argumentsboxes: ndarray, boxes of objects.scores: ndarray, scores of objects.# Returnskeep: ndarray, index of effective boxes."""x = boxes[:, 0]y = boxes[:, 1]w = boxes[:, 2] - boxes[:, 0]h = boxes[:, 3] - boxes[:, 1]areas = w * horder = scores.argsort()[::-1]keep = []while order.size > 0:i = order[0]keep.append(i)xx1 = np.maximum(x[i], x[order[1:]])yy1 = np.maximum(y[i], y[order[1:]])xx2 = np.minimum(x[i] + w[i], x[order[1:]] + w[order[1:]])yy2 = np.minimum(y[i] + h[i], y[order[1:]] + h[order[1:]])w1 = np.maximum(0.0, xx2 - xx1 + 0.00001)h1 = np.maximum(0.0, yy2 - yy1 + 0.00001)inter = w1 * h1ovr = inter / (areas[i] + areas[order[1:]] - inter)inds = np.where(ovr <= NMS_THRESH)[0]order = order[inds + 1]keep = np.array(keep)return keepdef yolov5_post_process(input_data):masks = [[0, 1, 2], [3, 4, 5], [6, 7, 8]]anchors = [[10, 13], [16, 30], [33, 23], [30, 61], [62, 45],[59, 119], [116, 90], [156, 198], [373, 326]]boxes, classes, scores = [], [], []for input, mask in zip(input_data, masks):b, c, s = process(input, mask, anchors)b, c, s = filter_boxes(b, c, s)boxes.append(b)classes.append(c)scores.append(s)boxes = np.concatenate(boxes)boxes = xywh2xyxy(boxes)classes = np.concatenate(classes)scores = np.concatenate(scores)nboxes, nclasses, nscores = [], [], []for c in set(classes):inds = np.where(classes == c)b = boxes[inds]c = classes[inds]s = scores[inds]keep = nms_boxes(b, s)nboxes.append(b[keep])nclasses.append(c[keep])nscores.append(s[keep])if not nclasses and not nscores:return None, None, Noneboxes = np.concatenate(nboxes)classes = np.concatenate(nclasses)scores = np.concatenate(nscores)return boxes, classes, scoresdef draw(image, boxes, scores, classes):"""Draw the boxes on the image.# Argument:image: original image.boxes: ndarray, boxes of objects.classes: ndarray, classes of objects.scores: ndarray, scores of objects.all_classes: all classes name."""for box, score, cl in zip(boxes, scores, classes):top, left, right, bottom = box# print('class: {}, score: {}'.format(CLASSES[cl], score))# print('box coordinate left,top,right,down: [{}, {}, {}, {}]'.format(top, left, right, bottom))top = int(top)left = int(left)right = int(right)bottom = int(bottom)cv2.rectangle(image, (top, left), (right, bottom), (255, 0, 0), 2)cv2.putText(image, '{0} {1:.2f}'.format(CLASSES[cl], score),(top, left - 6),cv2.FONT_HERSHEY_SIMPLEX,0.6, (0, 0, 255), 2)def letterbox(im, new_shape=(640, 640), color=(0, 0, 0)):# Resize and pad image while meeting stride-multiple constraintsshape = im.shape[:2]  # current shape [height, width]if isinstance(new_shape, int):new_shape = (new_shape, new_shape)# Scale ratio (new / old)r = min(new_shape[0] / shape[0], new_shape[1] / shape[1])# Compute paddingratio = r, r  # width, height ratiosnew_unpad = int(round(shape[1] * r)), int(round(shape[0] * r))dw, dh = new_shape[1] - new_unpad[0], new_shape[0] - new_unpad[1]  # wh paddingdw /= 2  # divide padding into 2 sidesdh /= 2if shape[::-1] != new_unpad:  # resizeim = cv2.resize(im, new_unpad, interpolation=cv2.INTER_LINEAR)top, bottom = int(round(dh - 0.1)), int(round(dh + 0.1))left, right = int(round(dw - 0.1)), int(round(dw + 0.1))im = cv2.copyMakeBorder(im, top, bottom, left, right, cv2.BORDER_CONSTANT, value=color)  # add borderreturn im, ratio, (dw, dh)if __name__ == '__main__':# Create RKNN objectrknn1 = RKNN(verbose=False)rknn2 = RKNN(verbose=False)# pre-process configprint('--> Config model')rknn1.config(mean_values=[[0, 0, 0]], std_values=[[255, 255, 255]], target_platform="rk3568")rknn2.config(mean_values=[[150.54, 150.54, 150.54]], std_values=[[49.215, 49.215, 49.215]], target_platform="rk3568")print('done')# Load ONNX modelprint('--> Loading model')ret = rknn1.load_onnx(model="./yolov5n.onnx")ret = rknn2.load_onnx(model="./best.onnx")if ret != 0:print('Load model failed!')exit(ret)print('done')# Build modelprint('--> Building model')ret = rknn1.build(do_quantization=True, dataset="dataset.txt")ret = rknn2.build(do_quantization=False)if ret != 0:print('Build model failed!')exit(ret)print('done')# Export RKNN modelprint('--> Export rknn model')ret = rknn1.export_rknn("./yolov5n.rknn")ret = rknn2.export_rknn("./best.rknn")if ret != 0:print('Export rknn model failed!')exit(ret)print('done')# Init runtime environmentprint('--> Init runtime environment')ret = rknn1.init_runtime()ret = rknn2.init_runtime()if ret != 0:print('Init runtime environment failed!')exit(ret)print('done')# Set inputsimg = cv2.imread("./car.jpg")# img, ratio, (dw, dh) = letterbox(img, new_shape=(IMG_SIZE, IMG_SIZE))img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)img = cv2.resize(img, (IMG_SIZE, IMG_SIZE))# Inferenceprint('--> Running model')outputs = rknn1.inference(inputs=[img])print('done')# post processinput0_data = outputs[0]input1_data = outputs[1]input2_data = outputs[2]input0_data = input0_data.reshape([3, -1]+list(input0_data.shape[-2:]))input1_data = input1_data.reshape([3, -1]+list(input1_data.shape[-2:]))input2_data = input2_data.reshape([3, -1]+list(input2_data.shape[-2:]))input_data = list()input_data.append(np.transpose(input0_data, (2, 3, 0, 1)))input_data.append(np.transpose(input1_data, (2, 3, 0, 1)))input_data.append(np.transpose(input2_data, (2, 3, 0, 1)))boxes, classes, scores = yolov5_post_process(input_data)img_1 = cv2.cvtColor(img, cv2.COLOR_RGB2BGR)if boxes is not None:print(boxes)# draw(img_1, boxes, scores, classes)for box in boxes:x1, y1, x2, y2 = boxx1 = int(x1)x2 = int(x2)y1 = int(y1)y2 = int(y2)cv2.rectangle(img_1, (x1, y1), (x2, y2), (0, 255, 0), 1, 1)roi = img[y1:y2, x1:x2]try:roi = cv2.resize(roi, (168, 48))print('333')output = rknn2.inference(inputs=[roi])input_data = np.swapaxes(output[0], 1, 2)index = np.argmax(input_data, axis=1)plate_no = decodePlate(index)img_1 = cv2ImgAddText(img_1, str(plate_no), x1, y1-30, (0, 255, 0), 30)print(str(plate_no))except:continue# show outputcv2.imshow("post process result", img_1)cv2.waitKey(0)cv2.destroyAllWindows()rknn1.release()rknn2.release()

程序大概思路是:

1、读取图片 

2、加载模型

3、检测车牌

4、识别车牌

5、显示图片 

上面是在虚拟机模拟运行,结果正常。

六、C++板载部署

自行部署,参考讯为电子的程序,可以部署成功。

如有侵权,或需要完整代码,请及时联系博主。


https://www.xjx100.cn/news/3417927.html

相关文章

CV每日论文---2024.6.3

1、Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis 中文标题&#xff1a;Video-MME&#xff1a;视频分析领域首个多模态法学硕士综合评估基准 简介&#xff1a;Video-MME 是一个全面评估多模态大语言模型&#xff08;M…

PostgreSQL 修改表结构卡住不动

目录 1 问题2 实现 1 问题 今天遇到的一个问题记录一下&#xff0c;因为系统上的一个改动需要同步脚本至测试库上&#xff0c;具体的脚本内容也很简单,就是修改了某张表的一个字段。但是无论怎么操作都是一直卡住&#xff0c;表的数据量很小就十几条数据所以初步怀疑是表被锁了…

第二讲笔记:隐私计算助力数据要素流通

1、数据要素流转与数据 2、数据外循环中的信任 焦虑 信任焦虑背后的代表性案例 内鬼门 &#xff1a; 2023 年 &#xff0c; 美国科技公司 Ubiquiti在2021年1月曝出数据泄露事 件&#xff0c; “攻击者”在随后的“谈判”中试 图向该企业勒索近200万美元&#xff08;50比特 币&…

Taro(React)使用富文本编辑器Editor

在写项目的过程中很容易涉及到让使用者&#xff0c;输入一些介绍或者文本等相关功能&#xff0c; 前端在拿到这些文本时是需要直接渲染的&#xff0c;如果不使用富文本编辑器&#xff0c;这个时候得到的文章是没有格式的&#xff0c;这对于使用者来说体验非常的不好&#xff0c…

一分钟了解香港的场外期权报价

香港的场外期权报价 在香港这个国际金融中心&#xff0c;场外期权交易是金融市场不可或缺的一部分。场外期权&#xff0c;作为一种非标准化的金融衍生品&#xff0c;为投资者提供了在特定时间以约定价格买入或卖出某种资产的机会。对于希望参与这一市场的投资者来说&#xff0…

HMM地图匹配算法库Barefoot环境搭建

1.引入gps路径匹配开源项目barefoot 克隆仓库 git clone https://github.com/bmwcarit/barefoot.git打开项目执行mvn命令将项目打包到maven仓库 mvn install -DskipTests在自己的maven项目中引入barefoot依赖 <dependency><groupId>com.bmw-carit</groupId&g…

React 之 mobx-state-tree(Redux替代品) 状态管理

MST(mobx-state-tree)、redux做多组件间全局state管理&#xff08;类比vuex&#xff0c;父 孙组件状态传递解耦&#xff09;。 tree type state 树中的每个节点都由两件事来描述: type (事物的形状) 和 data (它当前所处的状态). 最简单的树如下所示&#xff1a; 1.声明类…

数据结构——哈希表、哈希桶

哈希概念 顺序结构以及平衡树中&#xff0c;元素关键码与其存储位置之间没有对应的关系&#xff0c;因此在查找一个元素时&#xff0c;必须要经过关键码的多次比较&#xff0c;顺序查找时间复杂度为O(N),平衡树中为树的高度,即O(logN),搜索的效率取决于搜索过程种元素的比较次…