When running tracer.export_caffe2() on an Detecton2 model inside Google Colab I’m getting:
[11/25 21:54:49 d2.export.caffe2_export]: Exporting a Caffe2GeneralizedRCNN model via ONNX ... Some warnings from ONNX are expected and are usually not to worry about.
/usr/local/lib/python3.7/dist-packages/torch/onnx/utils.py:269: UserWarning: `add_node_names' can be set to True only when 'operator_export_type' is `ONNX`. Since 'operator_export_type' is not set to 'ONNX', `add_node_names` argument will be ignored.
"`{}` argument will be ignored.".format(arg_name, arg_name))
/usr/local/lib/python3.7/dist-packages/torch/onnx/utils.py:269: UserWarning: `do_constant_folding' can be set to True only when 'operator_export_type' is `ONNX`. Since 'operator_export_type' is not set to 'ONNX', `do_constant_folding` argument will be ignored.
"`{}` argument will be ignored.".format(arg_name, arg_name))
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-14-9d750fee609f> in <module>()
1 # convert and save caffe2 model
----> 2 caffe2_model = tracer.export_caffe2()
3 caffe2_model.save_protobuf(output)
4 # draw the caffe2 graph
5 caffe2_model.save_graph(os.path.join(output, "model.svg"), inputs=first_batch)
20 frames
/usr/local/lib/python3.7/dist-packages/detectron2/export/c10.py in _generate_proposals(self, images, objectness_logits_pred, anchor_deltas_pred, gt_instances)
201 angle_bound_hi=180,
202 clip_angle_thresh=1.0, # Default
--> 203 legacy_plus_one=False,
204 )
205 rpn_rois_list.append(rpn_rois)
RuntimeError: CUDA error: invalid argument
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
This error didn’t occur when using Detectron v0.5. I suspect this is connected to the PyTorch and Cuda versions used in Colab but I wasn’t able to fix it yet. Any help is greatly appreciated.
Colab notebook: Google Colab