Can't run nn.Conv1d with libtorch on iOS

Mobile
1

I get an error when I want to use nn.Conv1d on iOS using libtorch. I have this simple model:

class Conv1dTestModel(nn.Module):
    def __init__(self):
        super(Conv1dTestModel, self).__init__()
        self.conv1d_stack = nn.Sequential(
            nn.Conv1d(1, 64, kernel_size=128, padding=8),
            nn.MaxPool1d(kernel_size=5, padding=1),
        )
        
    def forward(self, x):
        N, C, L = x.shape[0], x.shape[1], x.shape[2]
        return self.conv1d_stack(x)
    
N = 16
L = 1024
C = 1
T = torch.rand(N, C, L)

model = Conv1dTestModel()
model(T).shape

torch.jit.script(model.eval().to('cpu')).save('Conv1dTestModel_scripted.pt')

When trying to run this model on iOS with a random tensor of shape {16, 1, 1024} I get the following error message

NNPACK SpatialConvolution_updateOutput failed
The above operation failed in interpreter.
Traceback (most recent call last):
  File "/home/dnagy/anaconda3/envs/pt11/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 201
                            self.weight, self.bias, self.stride,
                            _single(0), self.dilation, self.groups)
        return F.conv1d(input, self.weight, self.bias, self.stride,
               ~~~~~~~~ <--- HERE
                        self.padding, self.dilation, self.groups)
Serialized   File "code/__torch__/torch/nn/modules/conv.py", line 23
      _8 = _2
    else:
      _8 = torch.conv1d(input, self.weight, self.bias, [1], [8], [1], 1)
           ~~~~~~~~~~~~ <--- HERE
    return _8

I’m using PyTorch 1.4.0

2

Hi @dnnagy, sorry for the late reply. I need sometime to debug into NNPACK to figure out why this happens. If this blocks you, you can get around with it by disabling NNPACK in the iOS build script and recompile the static libraries from source code. Here is what I did

  1. Add CMAKE_ARGS+=("-DUSE_NNPACK=OFF") CMAKE_ARGS+=("-DUSE_XNNPACK=OFF")to build_ios.sh
  2. Run BUILD_PYTORCH_MOBILE=1 IOS_ARCH=arm64 ./scripts/build_ios.sh
  3. Follow the tutorial to link the static libraries in XCode

I was able to run your model above without NNPACK, and it worked fine on both of my iphonex and desktop simulator

    torch::autograd::AutoGradMode guard(false);
    NSString* path = [[NSBundle mainBundle]pathForResource:@"model.pt" ofType:@""];
    auto model = torch::jit::load(path.UTF8String);
    model.eval();
    auto input = torch::ones({16,1,1024});
    auto result = model.forward({input}).toTensor(); 
    result.print(); //[CPUFloatType [16, 64, 183]]
3

Hi @xta0, thank you very much for your anwser.
I am facing the same problem as @dnnagy but I can’t seem to find the build_ios.sh file.

When installing Libtorch with POD and running my code on the simulator everything works fine until I want to port it on my iphone. Should I rebuild Libtorch locally without POD with the arguments you have listed above ?
If so, do I need to jit.trace my model again ?
Thank you very much !

4

@Matthieu_TPops, The reason why simulator works is that we’ve disabled the NNPACK for the simulator build. You can definitely rebuild your pod via the custom build script, please refer to the tutorial here - https://pytorch.org/mobile/ios/#build-pytorch-ios-libraries-from-source. As for the model, you don’t need to trace it again. but one thing to keep in mind is that if you build your pod from master, you need to also use the pytorch master code to trace the model. Otherwise, you might run into some backward compatibility issues.

6

Thank You very much ! It seems linking the static libraries is not enough.

Is it necessary to change the TorchBridge as well ? ( As Libtorch.h is removed from the Podfile)

7

If you’re using Swift, you need a bridge to call objective-c code which wraps the C++ torch libraries.