Issue: Train new dataset: zeros after conv3 in vgg16

This code (FASTER RCNN in PyTorch) works perfectly fine for the PASCAL_VOC dataset but it fails in custom dataset with following error.

File “train.py”, line 127, in
net(im_data, im_info, gt_boxes, gt_ishard, dontcare_areas)
File “/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py”, line 206, in call
result = self.forward(*input, **kwargs)
File “/data/code/faster_rcnn_pytorch/faster_rcnn/faster_rcnn.py”, line 219, in forward
roi_data = self.proposal_target_layer(rois, gt_boxes, gt_ishard, dontcare_areas, self.n_classes)
File “/data/code/faster_rcnn_pytorch/faster_rcnn/faster_rcnn.py”, line 287, in proposal_target_layer
proposal_target_layer_py(rpn_rois, gt_boxes, gt_ishard, dontcare_areas, num_classes)
File “/data/code/faster_rcnn_pytorch/faster_rcnn/rpn_msr/proposal_target_layer.py”, line 66, in proposal_target_layer
np.hstack((zeros, np.vstack((gt_easyboxes[:, :-1], jittered_gt_boxes[:, :-1]))))))
File “/usr/local/lib/python2.7/dist-packages/numpy/core/shape_base.py”, line 234, in vstack
return _nx.concatenate([atleast_2d(_m) for _m in tup], 0)
ValueError: all the input array dimensions except for the concatenation axis must match exactly