Hello PyTorch community,
I hope this message finds you well. I am facing an issue while training a comment classification model using PyTorch Lightning with a pre-trained BERT model.
I encountered the following error during the training process:
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
To provide some context, I have already enabled gradients for all parameters of the model using the function
enable_gradients(model). However, the error still persists.
The model I am using is based on the
aubmindlab/bert-base-arabertv02-twitter pre-trained model, and I noticed that some weights of the BERT model were not initialized properly upon loading. I have ensured that I am using the latest versions of PyTorch, Transformers, and PyTorch Lightning.
I attempted to pretrain the BERT model on a downstream task before training my specific model, but the error remains unresolved.
I am reaching out to the community for any guidance or insights on how to resolve this issue. I would be more than happy to provide additional details about my training environment, code, and model setup if needed.
Thank you in advance for your support and advice. Your expertise and suggestions are highly appreciated.