Why torch.onnx.export() generate so many files?

jit
1

I used torch.onnx.export() to convert my torchscript to onnx. But the result files can have so many look like weight / bias files:
image

2

Could you post the code which is creating these files, please?

3

I find that if the exported onnx file size exceed a range, eg. 2g, it will generate too much files more 1 onnx file. Is it a limition or something else? Have you got some update for it.

4

thanks for the info, I will look this very later :joy:

5

I’m also having this problem, and I think Jing Xu might be right, because the model I’m trying to export is more than 2GB too (protobuf limit?)

@ptrblck here’s some code that replicates this - just click “Runtime > Run all”, but I think you’ll need a to use a high-RAM runtime: Google Colab

I’ve made an issue here which includes some more details and speculations.

6

It seems it is normal behaviour. Each file is a separate weight. The set of files you got is still a valid onnx model. You can load it with

onnx_model = onnx.load("model.onnx")

the model.onnx would be your main model file, onnx loader will automatically find all other files.

Now, when the model is under onnx control, you can save it again in more common format, where weighs are stored in one separate .data file.

from onnx.external_data_helper import convert_model_to_external_data

convert_model_to_external_data(
    onnx_model,
    all_tensors_to_one_file=True,  
    location="model.data",          
    size_threshold=0,              
    convert_attribute=False        
)

onnx.save_model(
    onnx_model,
    "model_consolidated.onnx",
    save_as_external_data=True,
    all_tensors_to_one_file=True,
    location="model.data",
    size_threshold=0,
)

Moreover this two files now can be used to create tensorrt engine, its common onnx parser reads it correct. Tested it with 5Gb SDXL, works for me.