Why torch.onnx.export() generate so many files?

jit
1

I used torch.onnx.export() to convert my torchscript to onnx. But the result files can have so many look like weight / bias files:
image

2

Could you post the code which is creating these files, please?

3

I find that if the exported onnx file size exceed a range, eg. 2g, it will generate too much files more 1 onnx file. Is it a limition or something else? Have you got some update for it.

4

thanks for the info, I will look this very later :joy:

5

I’m also having this problem, and I think Jing Xu might be right, because the model I’m trying to export is more than 2GB too (protobuf limit?)

@ptrblck here’s some code that replicates this - just click “Runtime > Run all”, but I think you’ll need a to use a high-RAM runtime: Google Colab

I’ve made an issue here which includes some more details and speculations.