A question about model output

data
1

Hi.
Recently, I want to show tello stream with image detection. My question is how to convert the output to data type that cv2.imshow() can work. Is this possible? My first thought is to save model’s output to my local with save() method, and show it by cv2.imshow() method. It works but the stream with objects detected will have a delay about 4~5 second.
Does anyone have ideas to convert model’ output?
Appreciate.

2

cv2.imshow expects numpy arrays and you can transform PyTorch tensors to an np.array via tensor.numpy(). Depending on your use case you might need to push the tensor to the CPU first and maybe detach it additionally:

arr = tensor.detach().cpu().numpy()

Once this is done, make sure the expected dtype and shape is passed to imshow, so you might need to permute the tensor to the channels-last memory layout and also swap the color channel order from RGB to BGR if needed.

1 Like
3

Hi @ptrblck,
Thanks for replay!.

I have edit my code like this:

from threading import Thread
from djitellopy import Tello
import cv2, math, time
import torch
import os
import numpy as np
import asyncio
import imutils
from PIL import Image


path = r'C:\yolov5-master'
model = torch.hub.load(path, 'yolov5s',source='local', pretrained=True)
tello = Tello()
tello.connect()
tello.streamon()

frame_read = tello.get_frame_read()
    
class VideoStreamWidget(object):
    def __init__(self, src=0):
        # Start the thread to read frames from the video stream
        self.thread = Thread(target=self.update, args=())
        self.thread.daemon = True
        self.thread.start()

    def update(self):
        # Read the next frame from the stream
        global frame
        while True:
            self.frame = cv2.cvtColor(frame_read.frame,cv2.COLOR_RGB2BGR)
            time.sleep(.01)

    def show_frame(self): 
        # Display frames in main program
        wee = model(self.frame)
        arr = wee.datah().cpu().numpy()
        img = Image.fromarray.fromarray(arr, 'RGB')
        result = cv2.cvtColor(img,cv2.COLOR_RGB2BGR)
        cv2.imshow('frame', result)

        key = cv2.waitKey(1)

if __name__ == '__main__':
    video_stream_widget = VideoStreamWidget()
    while True:
        try:
            video_stream_widget.show_frame()
        except AttributeError:
            pass

I noticed after I put self.frame into model( ) method in line 35. It will jump out show_frame(self).

I’m wondering what’s going on here. I’m also wondering what type is the output of model( ). Since the issue above, I can’t check if my convert progress before imshow is correct or not ( I think it is not.).

4

Python shouldn’t “jump out” of functions without a return statement so is a runtime error or so raised?

You can check your model definition and what type is returned in the forward method.
I would guess it’s a tensor.