Issue in converting PyTorch model to Keras

thearheus · January 14, 2020, 9:00am

Hello, I’m trying to convert my pytorch model to keras and I have ready onnx file for It. But when I started to converting onnx to keras, I’ve got next error:

DEBUG:onnx2keras:Check if all inputs are available:
DEBUG:onnx2keras:Check input 0 (name 645).
DEBUG:onnx2keras:Check input 1 (name 646).
DEBUG:onnx2keras:... found all, continue
DEBUG:onnx2keras:mul:Convert inputs to Keras/TF layers if needed.
WARNING:onnx2keras:mul:Failed to use keras.layers.Multiply. Fallback to TF lambda.
WARNING:tensorflow:Layer 647 is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.

If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.

To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.

WARNING:tensorflow:Layer 647 is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.

If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.

To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.

DEBUG:onnx2keras:######
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Converting ONNX operation
DEBUG:onnx2keras:type: Cast
DEBUG:onnx2keras:node_name: 648
DEBUG:onnx2keras:node_params: {'to': 7, 'change_ordering': False, 'name_policy': None}
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Check if all inputs are available:
DEBUG:onnx2keras:Check input 0 (name 647).
DEBUG:onnx2keras:... found all, continue
DEBUG:onnx2keras:######
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Converting ONNX operation
DEBUG:onnx2keras:type: Cast
DEBUG:onnx2keras:node_name: 649
DEBUG:onnx2keras:node_params: {'to': 11, 'change_ordering': False, 'name_policy': None}
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Check if all inputs are available:
DEBUG:onnx2keras:Check input 0 (name 648).
DEBUG:onnx2keras:... found all, continue
DEBUG:onnx2keras:######
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Converting ONNX operation
DEBUG:onnx2keras:type: Constant
DEBUG:onnx2keras:node_name: 650
DEBUG:onnx2keras:node_params: {'value': array(1.), 'change_ordering': False, 'name_policy': None}
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Check if all inputs are available:
DEBUG:onnx2keras:... found all, continue
DEBUG:onnx2keras:######
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Converting ONNX operation
DEBUG:onnx2keras:type: Div
DEBUG:onnx2keras:node_name: 651
DEBUG:onnx2keras:node_params: {'change_ordering': False, 'name_policy': None}
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Check if all inputs are available:
DEBUG:onnx2keras:Check input 0 (name 650).
DEBUG:onnx2keras:Check input 1 (name 649).
DEBUG:onnx2keras:... found all, continue
DEBUG:onnx2keras:div:Convert inputs to Keras/TF layers if needed.
WARNING:tensorflow:Layer 651 is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.

If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.

To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.

WARNING:tensorflow:Layer 651 is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.

If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.

To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.

DEBUG:onnx2keras:######
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Converting ONNX operation
DEBUG:onnx2keras:type: Constant
DEBUG:onnx2keras:node_name: 652
DEBUG:onnx2keras:node_params: {'value': array(224.), 'change_ordering': False, 'name_policy': None}
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Check if all inputs are available:
DEBUG:onnx2keras:... found all, continue
DEBUG:onnx2keras:######
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Converting ONNX operation
DEBUG:onnx2keras:type: Mul
DEBUG:onnx2keras:node_name: 653
DEBUG:onnx2keras:node_params: {'change_ordering': False, 'name_policy': None}
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Check if all inputs are available:
DEBUG:onnx2keras:Check input 0 (name 651).
DEBUG:onnx2keras:Check input 1 (name 652).
DEBUG:onnx2keras:... found all, continue
DEBUG:onnx2keras:mul:Convert inputs to Keras/TF layers if needed.
WARNING:onnx2keras:mul:Failed to use keras.layers.Multiply. Fallback to TF lambda.
WARNING:tensorflow:Layer 653 is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.

If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.

To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.

WARNING:tensorflow:Layer 653 is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.

If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.

To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.

DEBUG:onnx2keras:######
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Converting ONNX operation
DEBUG:onnx2keras:type: Cast
DEBUG:onnx2keras:node_name: 654
DEBUG:onnx2keras:node_params: {'to': 7, 'change_ordering': False, 'name_policy': None}
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Check if all inputs are available:
DEBUG:onnx2keras:Check input 0 (name 653).
DEBUG:onnx2keras:... found all, continue
DEBUG:onnx2keras:######
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Converting ONNX operation
DEBUG:onnx2keras:type: Mul
DEBUG:onnx2keras:node_name: 655
DEBUG:onnx2keras:node_params: {'change_ordering': False, 'name_policy': None}
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Check if all inputs are available:
DEBUG:onnx2keras:Check input 0 (name 654).
DEBUG:onnx2keras:Check input 1 (name 654).
DEBUG:onnx2keras:... found all, continue
DEBUG:onnx2keras:mul:Convert inputs to Keras/TF layers if needed.
WARNING:onnx2keras:mul:Failed to use keras.layers.Multiply. Fallback to TF lambda.
DEBUG:onnx2keras:######
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Converting ONNX operation
DEBUG:onnx2keras:type: Unsqueeze
DEBUG:onnx2keras:node_name: 657
DEBUG:onnx2keras:node_params: {'axes': [0], 'change_ordering': False, 'name_policy': None}
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Check if all inputs are available:
DEBUG:onnx2keras:Check input 0 (name 639).
DEBUG:onnx2keras:... found all, continue
DEBUG:onnx2keras:unsqueeze:Work with numpy types.
DEBUG:onnx2keras:######
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Converting ONNX operation
DEBUG:onnx2keras:type: Unsqueeze
DEBUG:onnx2keras:node_name: 659
DEBUG:onnx2keras:node_params: {'axes': [0], 'change_ordering': False, 'name_policy': None}
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Check if all inputs are available:
DEBUG:onnx2keras:Check input 0 (name 655).
DEBUG:onnx2keras:... found all, continue
DEBUG:onnx2keras:######
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Converting ONNX operation
DEBUG:onnx2keras:type: Concat
DEBUG:onnx2keras:node_name: 660
DEBUG:onnx2keras:node_params: {'axis': 0, 'change_ordering': False, 'name_policy': None}
DEBUG:onnx2keras:...
DEBUG:onnx2keras:Check if all inputs are available:
DEBUG:onnx2keras:Check input 0 (name 657).
DEBUG:onnx2keras:Check input 1 (name 1057).
DEBUG:onnx2keras:The input not found in layers / model inputs.
DEBUG:onnx2keras:Found in weights, add as a numpy constant.
DEBUG:onnx2keras:Check input 2 (name 659).
DEBUG:onnx2keras:... found all, continue
DEBUG:onnx2keras:concat:Concat Keras layers.
WARNING:onnx2keras:concat:!!! IMPORTANT INFORMATION !!!
WARNING:onnx2keras:concat:Something goes wrong with concat layers. Will use TF fallback.
WARNING:onnx2keras:concat:---
Traceback (most recent call last):
  File "C:\Users\1\Anaconda3\lib\site-packages\onnx2keras\reshape_layers.py", line 110, in convert_concat
    name=keras_name)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\keras\layers\merge.py", line 705, in concatenate
    return Concatenate(axis=axis, **kwargs)(inputs)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 745, in __call__
    inputs = nest.map_structure(_convert_non_tensor, inputs)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\util\nest.py", line 535, in map_structure
    structure[0], [func(*x) for x in entries],
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\util\nest.py", line 535, in <listcomp>
    structure[0], [func(*x) for x in entries],
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 743, in _convert_non_tensor
    return ops.convert_to_tensor(x)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1184, in convert_to_tensor
    return convert_to_tensor_v2(value, dtype, preferred_dtype, name)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1242, in convert_to_tensor_v2
    as_ref=False)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1296, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\framework\tensor_conversion_registry.py", line 52, in _default_conversion_function
    return constant_op.constant(value, dtype, name=name)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\framework\constant_op.py", line 227, in constant
    allow_broadcast=True)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\framework\constant_op.py", line 235, in _constant_impl
    t = convert_to_eager_tensor(value, ctx, dtype)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\framework\constant_op.py", line 96, in convert_to_eager_tensor
    return ops.EagerTensor(value, ctx.device_name, dtype)
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type NoneType).

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/1/PycharmProjects/untitled/onnx_.py", line 6, in <module>
    k_model = onnx2keras.onnx_to_keras(model, ["input_data"])
  File "C:\Users\1\Anaconda3\lib\site-packages\onnx2keras\converter.py", line 177, in onnx_to_keras
    keras_names
  File "C:\Users\1\Anaconda3\lib\site-packages\onnx2keras\reshape_layers.py", line 122, in convert_concat
    layers[node_name] = lambda_layer(layer_input)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 745, in __call__
    inputs = nest.map_structure(_convert_non_tensor, inputs)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\util\nest.py", line 535, in map_structure
    structure[0], [func(*x) for x in entries],
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\util\nest.py", line 535, in <listcomp>
    structure[0], [func(*x) for x in entries],
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 743, in _convert_non_tensor
    return ops.convert_to_tensor(x)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1184, in convert_to_tensor
    return convert_to_tensor_v2(value, dtype, preferred_dtype, name)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1242, in convert_to_tensor_v2
    as_ref=False)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1296, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\framework\tensor_conversion_registry.py", line 52, in _default_conversion_function
    return constant_op.constant(value, dtype, name=name)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\framework\constant_op.py", line 227, in constant
    allow_broadcast=True)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\framework\constant_op.py", line 235, in _constant_impl
    t = convert_to_eager_tensor(value, ctx, dtype)
  File "C:\Users\1\Anaconda3\lib\site-packages\tensorflow_core\python\framework\constant_op.py", line 96, in convert_to_eager_tensor
    return ops.EagerTensor(value, ctx.device_name, dtype)
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type NoneType).

There is my code:

import onnx
import onnx2keras

model = onnx.load_model("model.onnx")

k_model = onnx2keras.onnx_to_keras(model, ["input_data"])

And here is my onnx file to reproduce my errors:

Thank you.

onomatopoeia · January 14, 2020, 9:20am

A somewhat related discussion without a solution (sadly): Exporting FasterRCNN (fasterrcnn_resnet50_fpn) to ONNX

dhpollack · January 14, 2020, 10:45am

Why are you using float64 tensors? Are you creating tensors from numpy arrays in the pytorch model that you export to onnx? You should make sure the type of the numpy arrays are float32 and not the numpy default float64 or create the tensors with pytorch directly instead of numpy.

thearheus · January 14, 2020, 12:14pm

Hi, I searched in my code, but I almost everywhere using PyTorch FloatTensor with dtype=float32 in code. In line where I using numpy Is float32 also.