How to avoid `intimplicit` when `torch.jit.trace` a Model?

DavidG · February 20, 2024, 5:33pm

I am trying to convert a PyTorch model to CoreML but CoreML needs a traced model BUT does not support all the ops generated by torch.jit.trace

For example, trying to coremltools.converters.convert a traced PyTorch model and I got an error:

PyTorch convert function for op 'intimplicit' not implemented

I am trying to convert a RVC model from github.

I traced the model with torch.jit.trace and it fails. So I traced down the problematic part to the WN layer:

github.com

RVC-Project/Retrieval-based-Voice-Conversion-WebUI/blob/main/infer/lib/infer_pack/modules.py#L188


      
                  # last one is not necessary

                  if i < n_layers - 1:

                      res_skip_channels = 2 * hidden_channels

                  else:

                      res_skip_channels = hidden_channels

          

                  res_skip_layer = torch.nn.Conv1d(hidden_channels, res_skip_channels, 1)

                  res_skip_layer = torch.nn.utils.weight_norm(res_skip_layer, name="weight")

                  self.res_skip_layers.append(res_skip_layer)

          

          def forward(

              self, x: torch.Tensor, x_mask: torch.Tensor, g: Optional[torch.Tensor] = None

          ):

              output = torch.zeros_like(x)

              n_channels_tensor = torch.IntTensor([self.hidden_channels])

          

              if g is not None:

                  g = self.cond_layer(g)

          

              for i, (in_layer, res_skip_layer) in enumerate(

                  zip(self.in_layers, self.res_skip_layers)

import torch
import coremltools as ct
from infer.lib.infer_pack.modules import WN

model = WN(192, 5, dilation_rate=1, n_layers=16, gin_channels=256, p_dropout=0)
model.remove_weight_norm()
model.eval()

test_x = torch.rand(1, 192, 200)
test_x_mask = torch.rand(1, 1, 200)
test_g = torch.rand(1, 256, 1)

traced_model = torch.jit.trace(model, (test_x, test_x_mask, test_g), check_trace = True)

x = ct.TensorType(name='x', shape=test_x.shape)
x_mask = ct.TensorType(name='x_mask', shape=test_x_mask.shape)
g = ct.TensorType(name='g', shape=test_g.shape)

mlmodel = ct.converters.convert(traced_model, inputs=[x, x_mask, g])

I got an error RuntimeError: PyTorch convert function for op 'intimplicit' not implemented.

How could I modify the WN::forward so it does not generate an intimplicit operator ?

Thanks

David

DavidG · February 22, 2024, 5:36pm

I finally found the source of this issue.

Model is calling a function that has @torch.jit.script decorator :

github.com

RVC-Project/Retrieval-based-Voice-Conversion-WebUI/blob/main/infer/lib/infer_pack/commons.py#L108


      
              signal = get_timing_signal_1d(length, channels, min_timescale, max_timescale)

              return torch.cat([x, signal.to(dtype=x.dtype, device=x.device)], axis)

          

          

          def subsequent_mask(length):

              mask = torch.tril(torch.ones(length, length)).unsqueeze(0).unsqueeze(0)

              return mask

          

          

          @torch.jit.script

          def fused_add_tanh_sigmoid_multiply(input_a, input_b, n_channels):

              n_channels_int = n_channels[0]

              in_act = input_a + input_b

              t_act = torch.tanh(in_act[:, :n_channels_int, :])

              s_act = torch.sigmoid(in_act[:, n_channels_int:, :])

              acts = t_act * s_act

              return acts

          

          

          # def convert_pad_shape(pad_shape):

          #     l = pad_shape[::-1]

Removing this decorator, torch.jit.trace no longer add a intimplicit operator in the trace and CoreML tools could succesfully convert the model.