How to add preprocess into the model

Is it possible to add preprocess into the model? @ptrblck

Please don’t tag specific people in a post, as this might discourage others to post an answer and also the tagged user might not even know the best answer.

That being said: as explained in your other threads, yes that’s possible.
E.g. you could add it in your forward method.

1 Like

Thank you , but the machine learning engineer told me it’s impossible to add the preprocess into the model. I reviewed the code and then find it doesn’t have the code (No normalize), does it influence the effect?
We compose a sequence of transformation to pre-process the image:

import torchvision.transforms as transforms

transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

It is possible as shown here:

class MyModel(nn.Module):
    def __init__(self, transform):
        super(MyModel, self).__init__()
        self.conv = nn.Conv2d(3, 1, 3, 1, 1)
        self.transform = transform
        
    def forward(self, x):
        xs = []
        for x_ in x:
            x_ = self.transform(x_)
            xs.append(x_)
        xs = torch.stack(xs)
        x = self.conv(xs)
        return x

transform = transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
model = MyModel(transform)
x = torch.randn(1, 3, 24, 24)
output = model(x)

If you were using normalization in your PyTorch model, you have to use the same preprocessing pipeline in your CoreML model.

1 Like

Yeah if I didn’t do the normalize, it means I won’t do preprocess in coreML, right?

Yes, if you remove the preprocessing in PyTorch, retrain the model and the training looks good without any preprocessing (which might be hard), you could skip the preprocessing as well in your CoreML deployment.

But in CoreML you must tell the range of pixel buffer,or it doesn’t know how to deal with the data. In other deep learning framework it has image preprocess. But the machine learning engineer told me it’s no way to add preprocess in the model. You must make a standard rule to let us know each other , so the image preprocess is the basic rule to let us know each other.

1 Like