Create and save .h5 file for image datasets in PyTorch

vision
1

I am working on how to create HDF5 file for age and gender image recognition datasets. I want to produce an HDF5 file for the aforementioned dataset so that I may utilise it .h5 file for image recognition in a webcam application.
I made a.h5 file.
‘’’

import h5py as h5
def train_model(model, dataloaders, optimizer, criterion1, criterion2, num_epochs):  
    since = time.time()
    history = {
    #'train_acc': [],
    'train_loss': [],
    'val_loss': [],
    'age_accuracy': [],
    'gender_accuracy': []
}

    for epoch in range(num_epochs):
        print("Epoch {}/{}".format(epoch, num_epochs-1))
        print("-" * 10)

    for phase in ["train", "test"]:
        running_loss = 0.0
        running_acc1 = 0.0
        running_acc2 = 0.0
        total_1 = 0
        total_2 = 0
        if phase == "train":
            model.train()
        else:
            model.eval()

        for inputs, label1, label2 in tqdm(dataloaders[phase]):
            inputs = inputs.to(device)
            label1 = label1.long()
            label2 = label2.long()
            label1 = label1.to(device)
            label2 = label2.to(device)
            optimizer.zero_grad()

            with torch.autograd.set_grad_enabled(phase=="train"):
                outputs = model(inputs)
                loss1 = criterion1(outputs[0], label1)
                loss2 = criterion2(outputs[1], label2)
                loss = loss1 #+ loss2
                _, preds1 = torch.max(outputs[0], 1)
                _, preds2 = torch.max(outputs[1], 1)

                if phase == "train":
                    loss.backward()
                    optimizer.step()

            running_loss += loss.item() * inputs.size(0) #
          
            running_acc1 += torch.sum(preds1 == label1.data)
            running_acc2 += torch.sum(preds2 == label2.data)
            total_1 += label1.size(0)
            total_2 += label2.size(0)

        epoch_loss = running_loss / len(dataloaders[phase].dataset)
        age_epoch_acc1 = 100 * running_acc1 // total_1
        gender_epoch_acc2 = 100 * running_acc2 // total_2
        print("{} Loss: {:.4f} Age_Acc: {:.4f} gender_Acc: {:.4f} ".format(phase, epoch_loss, age_epoch_acc1, gender_epoch_acc2))  #, gender_epoch_acc2 "gender_Acc: {:.4f}"


        if phase == 'test':
            history['age_accuracy'].append(age_epoch_acc1)
            history['gender_accuracy'].append(gender_epoch_acc2)
            history['val_loss'].append(epoch_loss)
        if phase == 'train':
            history['train_loss'].append(epoch_loss)

    print()

torch.save(model, '/content/drive/MyDrive/ageNgendermodel.h5')

time_elapsed = time.time() - since
print("Training compete in {:.0f}m {:.0f}s".format(time_elapsed // 60, time_elapsed % 60))

return model, history

‘’’

My “ageNgendermodel.h5” file is created in Google Drive after training the model. However, when I download this file and utilise it in a web cam programme in a Jupyter notebook, I get the message “OSError: Unable to open file (File signature not found).”

I used the following code to determine if an HDF5 file was valid or not:
‘’’

import h5py
if h5py.is_hdf5('/content/drive/MyDrive/ageNgendermodel.h5'):
    print("File is a valid HDF5 file")

else:
    print("File is not a valid HDF5 file")

‘’’
for above code it shows “File is not a valid HDF5 file”.
I’m not sure where my code is going wrong. I’ve been attempting to address this problem for two days and haven’t been successful.
Please assist me in resolving this issue.

2

torch.save will not respect the file extension and will store the data in its own format.
You might thus need to create a proper h5file manually and store the data there.

3

Okay. How to create h5 file manually. and Is there another solution to create file.
As I have studied for web cam application it requires
faceProto = “\opencv_face_detector.pbtxt”
faceModel = “\opencv_face_detector_uint8.pb”

ageProto = “\age_deploy.prototxt”
ageModel = “\age_net.caffemodel”

genderProto = “\gender_deploy.prototxt”
genderModel = “\gender_net.caffemodel”

So, how to create .prototxt file and .caffemodel file. basically I want create my own these files for my own model.
Please assist me in resolving this issue.

4

This guide might be helpful.

.caffemodel files (and their corresponding .prototxt) were used in Caffe which was deprecated a long time ago, so I would not recommend trying to revive it.

5

Okay. Then how generate .h5 file in PyTorch.
I have search for this it shows
"’

torch.save(model, filepath)

‘’’
to save the .h5 file.
Or shall I use `
‘’’

torch.save(model.state_dict(), PATH)

‘’’
to save .h5 file.

6

As described before, PyTorch will not generate h5 files but use it’s own format.
If you want to create an h5 file (for some reason), refer to the linked guide.
I also don’t know why you want to create an h5 file in the first place as it seems you want to use Caffe based on your previous post and I don’t see the connection to PyTorch here.

7

First, I’d want to make a.h5 file for my own dataset, which will be utilised in a Realtime age and gender identification application. I attempted to generate it but was unsuccessful. Then I looked for a lot of Realtime age and gender recognition programmes that used.caffemodel and.prototxt files.
So, in my opinion, use PyTorch to generate the.caffemodel and.prototxt files if the.h5 file generation fails.

8

This won’t work as Caffe is deprecated.

9

So, which files should I create so that I may utilise them in my real-time age and gender recognition model or any future applications?

10

Please assist me in resolving this issue.