Best affordable GPU/TPU for advance or intermediate deep learning?

I have been using google colab pro with their Tesla p100 like 16GB of ram. But I feel it is only like 60 to 70% of it. In addition, I often need to train something that takes like 24 hours or more. To me, colab pro only last about 12 hours or 16 hours. If it is unlucky, which I did not save the trained model during 50% of training, I will need to retrain everything again. Thus, I think the more realistic way is buy a GPU or TPU and build a new desktop for training, but it is not for gaming. I hope I can have a good suggestion since this cost me a lot. I can afford like 100 to 550 US dollars (350 is the best) for GPU or TPU.

For CPU, I will use AMD 3300X or below since CPU is not really a big deal in deep learning.

For Nvidia, I feel most of their GPUs are about gamming rather have an affordable TPU or deep learning GPU.

I have been google searching online on this question, but most of them suggest buying an RTX like 700-1000 US dollars. I am not that rich, but I hope I can do an intermediate research level or maybe training NPL like BERT(Maybe).

For the cloud, many people say using google could for one month or one day can cost 100 dollars.
I have been training model on satellite images with 720p or above and Japanese anime coloring that really requires a good GPU. Even though the suggested GPU is not as good as google GPU Tesla P100, I can at least train it more than 24 hours.