Transferring weights from plantnet pytorch model to keras model

1

I want to use the weights of a plantnet model, available online for pytorch, in keras. As I am only familiar with keras.
The model architecture I am using is efficientnet b4. In keras, this model has a list of 611 weight tensors. The .tar file found online has of a list of 706 weight tensors. Removing the weight tensors with the name ‘…num_batches_tracked’ (I don’t think these are important) leaves me with 610 tensors. I changed the dimensions of the tensors and starting from the fourth tensors, there dimensions now match exactly. But the first three tensors of the keras_model (normalization_2/mean, normalization_2/variance, normalization_2/count) are missing. The pytorch tensors list also has two more tensors at the end. Can somebody explain if these weights are important or if I can initialize them randomly?
Here is my code:

filename = “efficientnet_b4_weights_best_acc.tar”
model = efficientnet_b4(weights=None)
state_dict = torch.load(filename, map_location=torch.device(‘cpu’))[‘model’]
model.load_state_dict(state_dict, strict=False)

keras_model = EfficientNetB4(
include_top=False,
weights=None,
input_tensor=None,
input_shape=None,
pooling=None,
classes=1081,
classifier_activation=“softmax”
)
state_dict = {key: state_dict[key] for key in state_dict if not ‘num_batches_tracked’ in key}

keras_weights = [w.numpy() for w in state_dict.values()]
for i in range(len(keras_weights)):
if len(keras_weights[i].shape) == 4:
keras_weights[i] = np.moveaxis(keras_weights[i], [0,1], [-1,-2])
if keras_weights[i].shape[2] == 1:
keras_weights[i] = np.moveaxis(keras_weights[i], 2, -1)
if len(keras_weights[i].shape) == 2:
keras_weights[i] = keras_weights[i].T

Thank you

2

Transferring PyTorch model weights to Keras can be challenging since the two frameworks use different formats to store their model weights. However, it is possible to transfer the weights using a few steps:

  1. Convert the PyTorch model to ONNX format: ONNX is an open format for representing deep learning models that can be used by various frameworks, including Keras. You can use the torch.onnx.export() function to convert your PyTorch model to ONNX format.
  2. Convert the ONNX model to Keras format: You can use the onnx2keras package to convert the ONNX model to Keras format. This package provides a onnx_to_keras() function that can convert the ONNX model to a Keras model.
  3. Load the weights into the Keras model: Once you have the Keras model, you can load the weights from the PyTorch model into the Keras model using the model.set_weights() function.

Here is an example code snippet that shows how to transfer PyTorch model weights to Keras:

import torch
import onnx
from onnx2keras import onnx_to_keras

# Load PyTorch model
pytorch_model = torch.load('model.pt')

# Convert PyTorch model to ONNX format
dummy_input = torch.randn(1, 3, 224, 224)
torch.onnx.export(pytorch_model, dummy_input, 'model.onnx', input_names=['input'], output_names=['output'])

# Convert ONNX model to Keras format
onnx_model = onnx.load('model.onnx')
keras_model = onnx_to_keras(onnx_model, ['input'])

# Load weights into Keras model
weights = pytorch_model.state_dict()
keras_model.set_weights([weights[key].numpy() for key in weights.keys()])

# Use Keras model for inference
output = keras_model.predict(input_data)
3

Hi
Thank you very much for your response.

I ran into a value error during the convertion: ValueError: ‘/features/features.0/features.0.0/Conv_output_0_pad/’ is not a valid root scope name. A root scope name has to match the following pattern: ^[A-Za-z0-9.][A-Za-z0-9_.\/>-]*$

I don’t know where this scope name is stored so I can change this…