Problem with objects detection then change model

I see into the work of YOLO in pytorch . Then I run detection with yolov3.cfg , yolov3.weights and coco.names (80 class list) all working good at video and images, and accuracy too fine. But if I change cfg to yolo3_openimages.cfgweights to yolov3-openimages.weights and openimages.name(601 class list) accuracy is so bad. At the images and videos all classes are generalized. Bee like animal, horse like enimal cars like vehicles and many other objects are not recognized. If you have experience with yolo, please advise me what to do?