Pytorch not able to use GPU || AWS Graviton Instance

I installed pytorch and in my .py file I gave

os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"]="0,1"

My NVIDIA-SMI output looks like this

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 530.41.03              Driver Version: 530.41.03    CUDA Version: 12.1     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                  Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf            Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA T4G                      Off| 00000000:00:1E.0 Off |                    0 |
| N/A   52C    P8               10W /  70W|      2MiB / 15360MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   1  NVIDIA T4G                      Off| 00000000:00:1F.0 Off |                    0 |
| N/A   52C    P8               10W /  70W|      2MiB / 15360MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+

When I run my application, The model is not loaded in GPU memory. Is there anything that I’m missing ?

Do I need to lower my CUDA version?

Also when I installed pytorch from
pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu121

It says

ERROR: Could not find a version that satisfies the requirement torch (from versions: none) ERROR: No matching distribution found for torch
How can I make sure that pytorch uses GPU here

The binaries for ARM do not support CUDA as these were built for Mac only.
You could either build PyTorch from source with CUDA support or e.g. use an NGC container.