Hi,
I am working on a text classification project, and I am researching the best way to do batching if I want to send a csv file to the model to classify at once. I couldn’t find examples that show batching for text, but I came across this example for Image Classification that describes batching using two methods:
- TorchServe Batching
- Client-Side Batching
At some stage, instead of using curl, I want to implement a UI application that sends either raw text or csv file to be classified. My idea is that the csv file would be batched on the client side and then sent as a payload in JSON to requests.post(url, data=payload, headers=headers)
I wonder which method is better for my use case. Also, if the TorchServe Batching is better, can you please recommend any pointers for its implementation?