Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

执行可执行文件的时候报错了 #5

Open
xunxuntu opened this issue Aug 16, 2023 · 12 comments
Open

执行可执行文件的时候报错了 #5

xunxuntu opened this issue Aug 16, 2023 · 12 comments

Comments

@xunxuntu
Copy link 

root@rtx2060:/home/tuc/TensorRT_Inference_Demo-main/bin# ./object_detection yolov5 /home/tuc/yolov5-6.1/data/images/bus.jpg 
[08/16/2023-14:32:00] [I] [TRT] [MemUsageChange] Init CUDA: CPU +547, GPU +0, now: CPU 554, GPU 243 (MiB)
[08/16/2023-14:32:01] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +273, GPU +68, now: CPU 846, GPU 311 (MiB)
[08/16/2023-14:32:01] [I] [TRT] ----------------------------------------------------------------
[08/16/2023-14:32:01] [I] [TRT] Input filename:   ../weights/yolov5/yolov5s.onnx
[08/16/2023-14:32:01] [I] [TRT] ONNX IR version:  0.0.6
[08/16/2023-14:32:01] [I] [TRT] Opset version:    11
[08/16/2023-14:32:01] [I] [TRT] Producer name:    pytorch
[08/16/2023-14:32:01] [I] [TRT] Producer version: 1.11.0
[08/16/2023-14:32:01] [I] [TRT] Domain:           
[08/16/2023-14:32:01] [I] [TRT] Model version:    0
[08/16/2023-14:32:01] [I] [TRT] Doc string:       
[08/16/2023-14:32:01] [I] [TRT] ----------------------------------------------------------------
[08/16/2023-14:32:01] [W] [TRT] onnx2trt_utils.cpp:369: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[08/16/2023-14:32:01] [W] [TRT] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[08/16/2023-14:32:01] [E] [TRT] 4: [network.cpp::operator()::3020] Error Code 4: Internal Error (images: kOPT dimensions in profile 0 are [8,3,640,640] but input has static dimensions [1,3,640,640].)
[08/16/2023-14:32:01] [E] [TRT] 2: [builder.cpp::buildSerializedNetwork::636] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
object_detection: /home/tuc/TensorRT_Inference_Demo-main/src/basemodel.cpp:57: void Model::OnnxToTRTModel(): Assertion `data' failed.
Aborted (core dumped)
按照您的教程readme文档的说明，使用yolov5-v6.1 仓库，执行完 
python export.py --weights=yolov5s.pt  --dynamic --simplify --include=onnx --opset 11
命令得到了onnx模型，并将onnx模型放入到 configs 文件夹的同级目录下的 weights 文件夹下的yolov5文件夹中。

接着进入 bin 文件夹，执行 
./object_detection yolov5 /home/tuc/yolov5-6.1/data/images/bus.jpg
命令，就出现了开头的core dump

请问 configs 下的 yolov5.yaml 文件中的 engine_file 字段的trt 文件是需要自己手动生成吗？
@Li-Hongda
Copy link
Owner

input has static dimensions [1,3,640,640] 报错的原因是ONNX文件为静态shape，可以使用netron检查一下，然后重新生成一下onnx。或者注释掉代码中关于dynamic shape的处理

@xunxuntu
Copy link
Author

我用trtexec工具生成了yolov5s.trt，并放在了weights下，然后执行
./object_detection yolov5 /home/tuc/yolov5-6.1/data/images
命令，显示了推理时间，却没有在保存的图片上将检测框画出来。
如下图

image

image

@Li-Hongda
Copy link
Owner

这个位置要确保scale为float，这里我前段时间修改过，你是否用的最新的代码呢？

@xunxuntu
Copy link
Author

是的，是最新的代码。昨天才git下来的

@Li-Hongda
Copy link
Owner

那就已经是float了，我刚刚测试了这两张图应该是没有问题的，因此我认为还是trt转换出了问题，你可以把config中的dynamic设置为0重新生成一下（因为从报错信息来看onnx为静态文件）

@xunxuntu
Copy link
Author

我把dynamic设为了0，还是不起作用。
您是怎么得到trt文件的，我是用这个命令得到的：

root@rtx2060:/usr/local/TensorRT-8.4.3.1/bin# ./trtexec --onnx=/home/tuc/yolov5-6.1/weights/yolov5s.onnx --saveEngine=/home/tuc/yolov5-6.1/weights/outfpbest.trt --workspace=2048 --best

# 导出onnx是用这个命令，yolov5版本是6.1
python export.py --weights=yolov5s.pt  --dynamic --simplify --include=onnx --opset 11

会不会和环境有关
我用的是cuda 11.2 , cudnn 8.4.0 , TensorRT-8.4.3.1
rtx2060

@xunxuntu
Copy link
Author

那就已经是float了，我刚刚测试了这两张图应该是没有问题的，因此我认为还是trt转换出了问题，你可以把config中的dynamic设置为0重新生成一下（因为从报错信息来看onnx为静态文件）

之前的报错信息在我导出 trt 模型后，放在weights路径下就自动消失了。

@Li-Hongda
Copy link
Owner

我这个代码是如果weights路径下没有检测到相应的权重，就调用转换onnx转trt函数，之前的报错是发生在转换这一步；你用trt转好之后就不经过这一步了，所以不报错。但是从当时的报错信息能看出来，你所用的onnx是静态图的，所以在转换成trt时应该把dynamic设置为0，我的意思就是重新转换一下（先把weights下的engine删掉）。版本可能也有关吧，这个得查一下相关的更新文件，但应该不是主要原因。

@xunxuntu
Copy link
Author

很奇怪，我删除了原先的onnx 和 trt 文件
重新使用export.py导出了onnx，python export.py --weights=weights/yolov5s.pt --include=onnx

(tc_yolov5) root@rtx2060:/home/tuc/yolov5-6.1# python export.py --weights=weights/yolov5s.pt --include=onnx
export: data=data/coco128.yaml, weights=['weights/yolov5s.pt'], imgsz=[640, 640], batch_size=1, device=cpu, half=False, inplace=False, train=False, optimize=False, int8=False, dynamic=False, simplify=False, opset=12, verbose=False, workspace=4, nms=False, agnostic_nms=False, topk_per_class=100, topk_all=100, iou_thres=0.45, conf_thres=0.25, include=['onnx']
YOLOv5 🚀 2022-2-22 torch 1.11.0 CPU

Fusing layers... 
Model Summary: 213 layers, 7225885 parameters, 0 gradients

PyTorch: starting from weights/yolov5s.pt with output shape (1, 25200, 85) (14.8 MB)

ONNX: starting export with onnx 1.14.0...
ONNX: export success, saved as weights/yolov5s.onnx (29.3 MB)

Export complete (2.41s)
Results saved to /home/tuc/yolov5-6.1/weights
Detect:          python detect.py --weights weights/yolov5s.onnx
PyTorch Hub:     model = torch.hub.load('ultralytics/yolov5', 'custom', 'weights/yolov5s.onnx')
Validate:        python val.py --weights weights/yolov5s.onnx
Visualize:       https://netron.app
(tc_yolov5) root@rtx2060:/home/tuc/yolov5-6.1#

然后将configs/yolov5.yaml 下的dynamic设为0
image

执行 ./object_detection yolov5 /home/tuc/yolov5-6.1/data/images 依旧不起作用。

root@rtx2060:/home/tuc/TensorRT_Inference_Demo-main/bin# ./object_detection yolov5 /home/tuc/yolov5-6.1/data/images
[08/16/2023-20:09:42] [I] [TRT] [MemUsageChange] Init CUDA: CPU +547, GPU +0, now: CPU 554, GPU 243 (MiB)
[08/16/2023-20:09:43] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +273, GPU +68, now: CPU 846, GPU 311 (MiB)
[08/16/2023-20:09:43] [I] [TRT] ----------------------------------------------------------------
[08/16/2023-20:09:43] [I] [TRT] Input filename:   ../weights/yolov5/yolov5s.onnx
[08/16/2023-20:09:43] [I] [TRT] ONNX IR version:  0.0.7
[08/16/2023-20:09:43] [I] [TRT] Opset version:    12
[08/16/2023-20:09:43] [I] [TRT] Producer name:    pytorch
[08/16/2023-20:09:43] [I] [TRT] Producer version: 1.11.0
[08/16/2023-20:09:43] [I] [TRT] Domain:           
[08/16/2023-20:09:43] [I] [TRT] Model version:    0
[08/16/2023-20:09:43] [I] [TRT] Doc string:       
[08/16/2023-20:09:43] [I] [TRT] ----------------------------------------------------------------
[08/16/2023-20:09:43] [W] [TRT] onnx2trt_utils.cpp:369: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[08/16/2023-20:09:43] [W] [TRT] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[08/16/2023-20:09:43] [W] [TRT] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[08/16/2023-20:09:43] [W] [TRT] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[08/16/2023-20:09:44] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +670, GPU +148, now: CPU 1549, GPU 459 (MiB)
[08/16/2023-20:09:44] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +245, GPU +54, now: CPU 1794, GPU 513 (MiB)
[08/16/2023-20:09:44] [W] [TRT] TensorRT was linked against cuDNN 8.4.1 but loaded cuDNN 8.4.0
[08/16/2023-20:09:44] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[08/16/2023-20:11:29] [I] [TRT] Detected 1 inputs and 7 output network tensors.
[08/16/2023-20:11:29] [I] [TRT] Total Host Persistent Memory: 151744
[08/16/2023-20:11:29] [I] [TRT] Total Device Persistent Memory: 1760768
[08/16/2023-20:11:29] [I] [TRT] Total Scratch Memory: 0
[08/16/2023-20:11:29] [I] [TRT] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 0 MiB, GPU 0 MiB
[08/16/2023-20:11:29] [I] [TRT] [BlockAssignment] Algorithm ShiftNTopDown took 16.8801ms to assign 7 blocks to 127 nodes requiring 35635200 bytes.
[08/16/2023-20:11:29] [I] [TRT] Total Activation Memory: 35635200
[08/16/2023-20:11:29] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +0, GPU +0, now: CPU 0, GPU 0 (MiB)
[08/16/2023-20:11:29] [W] [TRT] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[08/16/2023-20:11:29] [W] [TRT] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[08/16/2023-20:11:29] [I] [TRT] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 2299, GPU 647 (MiB)
[08/16/2023-20:11:29] [I] [TRT] Loaded engine size: 31 MiB
[08/16/2023-20:11:29] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +0, now: CPU 0, GPU 0 (MiB)
[08/16/2023-20:11:29] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +0, now: CPU 0, GPU 0 (MiB)
batch preprocess time: 0.4705ms batch inference time: 0.5246ms batch postprocess time: 3.6264ms 
batch preprocess time: 0.3895ms batch inference time: 0.4900ms batch postprocess time: 3.5747ms 
Average processing time is 11.3451ms 
Average FPS is 88.1440

@xunxuntu
Copy link
Author

会不会 是你代码里的后处理的问题呀，能够打印推理的时间，但是不能画框，是不是检测的那部分。

@Li-Hongda
Copy link
Owner

我暂时也无法定位问题，因为我复现不出来。。

@xunxuntu
Copy link
Author

我暂时也无法定位问题，因为我复现不出来。。

博主，您好，我将我本地的模型和其代码上传到 https://github.com/xunxuntu/check_todo.git 这里了，您有空可以git 下来帮我看看吗 😁😁😁😁😁😁😁😁😁

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Li-Hongda @xunxuntu