It is the result after print:
‘’’
Done.
torch.Size([3, 128, 128]) 0
Label : 0
Length of Train Data : 273800
Length of Validation Data : 2679
Traceback (most recent call last):
File “CNNFinal.py”, line 135, in
print(nn.Module.conv_layer1.weight.device)
AttributeError: type object ‘Module’ has no attribute ‘conv_layer1’
‘’’
Access the layer from the model object, not from the nn.Module type.
I.e. in your code you’ve created a model e.g. via:
model = ConvNeuralNet()
then access the layer from model.
Did you call model.to(device) before?
Haha, I did not.
I have just called it. It is running now.
Thank you for response and help. Another thing, Is it possible to suggest me any tutorial to improve my knowledge about CNN?
Good to hear it’s working now!
Maybe the CIFAR10 tutorial would be a good starter, but I’m unsure what exactly you are looking for.
I will take care of the Cifar10 tutorial, but specifically at this step I want to create a CNN model for Herbarium Dataset. There are several Data set in every year Kaggle competition. I start with the 2019 that was simple but my goal is Dataset 2022 Kaggle competition. In 2019 Dataset I resize the image to 128X128 and do data Augmentation to increase the dataset sizes 7times. The accuracy is 65.83. In this dataset how I can improve the accuracy? Just I should say the number of classes is almost 700.
For 2022 Dataset, I have problem to figure out about the dataset labels. It is a JSON files and it seems the class has hierarchy. I am confuse how I can create the label in this dataset?
It makes sense to “cache” the data after the collation, doesn’t it?
If so, this should be done in the data loader.
What is the current recipe?
I encountered the same issue while running the pytorch tutorial(Learn the Basics — PyTorch Tutorials 2.3.0+cu121 documentation): GPU usage was about 15%.
I tried replacing my loader init from:
train_dataloader = DataLoader(training_data, batch_size=batch_size)
test_dataloader = DataLoader(test_data, batch_size=batch_size)
to:
train_tensor = TensorDataset(training_data.data.to(device), training_data.targets.to(device))
test_tensor = TensorDataset(test_data.data.to(device), test_data.targets.to(device))
train_dataloader = DataLoader(train_tensor, batch_size=batch_size)
test_dataloader = DataLoader(test_tensor, batch_size=batch_size)
Without any custom loader.
With this the GPU usage went close to 100% and training time got reduced from 90 seconds to 17 seconds.
Funny thing: GPU memory usage went down from 2 GB to 1.1 GB with the new implementation.
However: there is a very suspect “accuracy” starting at 86% right at the beginning of training.
Running testing predictions separately after the training finished revealed the true accuracy was actually around 30-45%. I think I messed something up.
I tried to move all data at start to gpu:
training_data.data = training_data.data.to(device)
training_data.targets = training_data.targets.to(device)
test_data.data = test_data.data.to(device)
test_data.targets = test_data.targets.to(device)
but this resulted in this error on the enumerate line:
Traceback (most recent call last):
File "C:\Users\wyesp\PycharmProjects\PyTorch\training.py", line 129, in <module>
train_loop(train_dataloader, model, loss_fn, optimizer)
File "C:\Users\wyesp\PycharmProjects\PyTorch\training.py", line 84, in train_loop
for batch, (X, y) in enumerate(dataloader):
File "C:\Users\wyesp\anaconda3\Lib\site-packages\torch\utils\data\dataloader.py", line 631, in __next__
data = self._next_data()
^^^^^^^^^^^^^^^^^
File "C:\Users\wyesp\anaconda3\Lib\site-packages\torch\utils\data\dataloader.py", line 675, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\wyesp\anaconda3\Lib\site-packages\torch\utils\data\_utils\fetch.py", line 51, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\wyesp\anaconda3\Lib\site-packages\torch\utils\data\_utils\fetch.py", line 51, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
~~~~~~~~~~~~^^^^^
File "C:\Users\wyesp\anaconda3\Lib\site-packages\torchvision\datasets\mnist.py", line 143, in __getitem__
img = Image.fromarray(img.numpy(), mode="L")
^^^^^^^^^^^
TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.
If I interpret this correctly, it looks like the FashionMNIST dataset needs its data to be on the cpu.
Moving data to/from gpu every training batch is expensive.
I think I still don’t understand what a TensorDataset is.
I can store tensors in a Dataset without TensorDataset.