My GPUs (2x Tesla V-100) usage loooks like this (attached .png) also is 82 & 79 C temp. normal during training. can I run my gpus like this for 10-12 days.
Depending on your use case and workload the graph might be expected. Without any information we can only guess.
I am using PeFT on a llama-8B model on tyrone DIT4000TR-48RL.
I have loaded the model in 8-bit to save space.
Also what about temp of 83/79 C of GPUs are they ok?
Also, it would be really helpful if you could suggest some ways to load models efficiently so that I can save space.