Random results in object detection when using a custom trained model(yolov8s as well yolo11s)
YAML data file:
path: (…folder path)
test: test\imagestrain:
train\images
val: validation\imagesnc: 1
names: [“Apple”]
All folders(test, train, validate) contain “images” and “labels” folders, all images all unique(no repeating images in any of the folders).
I run the training with this command “yolo detect train data=data.yaml model=yolov8s.pt epochs=90 imgsz=640 profile = True”.
Once the training is done, i copy the file “best.pt” from Data->runs->detect->train->weights, then i convert it with a script to .onnx since i am using it in OpenCV.
I am converting as well the trained models, so the conversion to .onnx isn’t the issue.
Correct results rectangle output: (Link to image: good result — Freeimage.host)
x 139.614 y 117.224 w 180.547 h 177.173 class_id 47 class_score 0.959822
x 319.697 y 323.333 w 636.135 h 627.457 class_id 60 class_score 0.26418
x 493.684 y 249.387 w 236.908 h 206.874 class_id 46 class_score 0.924733
BAD results rectangle output: (Link to image: bad result — Freeimage.host)
x 132.563 y 616.468 w 266.61 h 47.1793 class_id 12 class_score 621.335
x 144.472 y 617.792 w 283.971 h 44.7845 class_id 7 class_score 621.335
x 166.978 y 620.281 w 331.142 h 40.2383 class_id 2 class_score 621.335
x 192.263 y 621.335 w 379.946 h 38.3646 class_id 2 class_score 620.519
x 212.024 y 620.519 w 434.511 h 40.2094 class_id 2 class_score 618.833
x 234.077 y 618.833 w 470.669 h 43.7068 class_id 2 class_score 617.004
x 263.682 y 617.004 w 470.693 h 47.3287 class_id 2 class_score 614.085
x 297.421 y 614.085 w 470.935 h 53.1138 class_id 2 class_score 610.107
x 331.886 y 610.107 w 474.087 h 61.0482 class_id 2 class_score 607.41
x 361.037 y 607.41 w 480.529 h 66.4008 class_id 27 class_score 607.363
x 387.238 y 606.562 w 482.929 h 67.9863 class_id 22 class_score 607.363
x 414.299 y 605.055 w 474.496 h 70.7969 class_id 17 class_score 607.363
x 425.989 y 600.909 w 436.541 h 78.6324 class_id 12 class_score 607.363
You can clearly see, that the max “class_score” is equal to the next rectangle Y pos… Which obviously produces broken results, as well as a decreasing value pattern.
This model has been trained with only one class (id 0) but the class ids 12, 7, 2, 27, 22, 17 are as well either random results or something else.
There are 8400 rectangles detected, some are reduced by NMSBoxes(opencv), the same value as the output in rows. Since the class_score is random, all the rectangles predictions are random as well.
I tried to load the custom model into pytorch directly with python and no c++ OpenCV, but i get some weird error when i try load the model, it says:
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "......py_detect.py", line 21, in <module>
pred = model([img])
TypeError: 'dict' object is not callable
I don’t use python, i don’t know what it is, and i didn’t find any specific solution for this, and i have no more time, so i am unable to test if by chance it works correctly this way.
The last thing, in this video (put this code “3XmjyBE8xVA“ in youtube’s watch?v= since i can put only 2 links) at time 13:29, the dude uses this code:
He multiplies the x and w by the with and height in pixels, which suggests that the X/Y values MUST be in 0-1 range, meaning that to get 320x320 pixels, the value must be (0.5, 0.5) which as well he suggested, then the rectangle is adjusted by it’s center
float x = data[0], y = data[1], w = data[2], h = data[3];
int left = x * modelInput.cols - w * modelInput.cols / 2;
int top = y * modelInput.rows - h * modelInput.rows / 2;
int width = w * modelInput.cols;
int height = h * modelInput.rows;
However i am getting the values in pixels in both cases(custom broken model and the working downloaded ones), if i use his suggestions then i will get values like 40000x50000 which is obviously out of boundary
int left = x - w/2;
int top = y - h/2;
int width = w;
int height = h;
I tried to do everything i could, but nothing seems to work. I include as well the full training config
task: detect
mode: train
model: yolov8s.pt
data: data.yaml
epochs: 90
time: null
patience: 100
batch: 16
imgsz: 640
save: true
save_period: -1
cache: false
device: null
workers: 8
project: null
name: train2
exist_ok: false
pretrained: true
optimizer: auto
verbose: true
seed: 0
deterministic: true
single_cls: false
rect: false
cos_lr: false
close_mosaic: 10
resume: false
amp: true
fraction: 1.0
profile: true
freeze: null
multi_scale: false
compile: false
overlap_mask: true
mask_ratio: 4
dropout: 0.0
val: true
split: val
save_json: false
conf: null
iou: 0.7
max_det: 300
half: false
dnn: false
plots: true
source: null
vid_stride: 1
stream_buffer: false
visualize: false
augment: false
agnostic_nms: false
classes: null
retina_masks: false
embed: null
show: false
save_frames: false
save_txt: false
save_conf: false
save_crop: false
show_labels: true
show_conf: true
show_boxes: true
line_width: null
format: torchscript
keras: false
optimize: false
int8: false
dynamic: false
simplify: true
opset: null
workspace: null
nms: false
lr0: 0.01
lrf: 0.01
momentum: 0.937
weight_decay: 0.0005
warmup_epochs: 3.0
warmup_momentum: 0.8
warmup_bias_lr: 0.1
box: 7.5
cls: 0.5
dfl: 1.5
pose: 12.0
kobj: 1.0
nbs: 64
hsv_h: 0.015
hsv_s: 0.7
hsv_v: 0.4
degrees: 0.0
translate: 0.1
scale: 0.5
shear: 0.0
perspective: 0.0
flipud: 0.0
fliplr: 0.5
bgr: 0.0
mosaic: 1.0
mixup: 0.0
cutmix: 0.0
copy_paste: 0.0
copy_paste_mode: flip
auto_augment: randaugment
erasing: 0.4
cfg: null
tracker: botsort.yaml