So, i created this model for translation that works nice enough, now i want to use as a “product”, like i want to be able to input a sentence and compute it’s translation, how do i go about doing that, do i make a custom data loader for it, wouldn’t i run into trouble with drop_last = True for data loader.
How would you like to deploy your model?
If you would like to feed the inputs “live” , you wouldn’t necessarily need a DataLoader.
@jspisak mentioned Flask as a valid approach here, if you don’t need low latency and also highlights a proposal of Amazon for a high performance PyTorch C++ serving platform.