How to get fast inference with Pytorch and MXNet model using GPU?

3

Thank you @ptrblck.

For your question,

  1. I just get numpy.array features from MXNet1 and MXnet2 as a following figure.

    image
    image1010×314 61.5 KB

    I put all the models on the same GPU (GPU:0) as you can see below.
    image

  2. Input size is not that big. Using cv2 based library, I get frame using rtsp and resize the frame into (270, 470, 3) and throw it to the model.

  3. I need to measure the processing time at each step. Is it recommend to use time.time() before and after the code block?