Hi. i’m new to Pytorch and CUDA, I’m trying to run a llm locally and attempting to offload some of its layer to the GPU and encountering this error. Please advise.
This is the output for: python -m torch.utils.collect_env
PyTorch version: 2.0.1+cu118
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A
OS: Microsoft Windows 10 Home Single Language
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.26.0-msvc3
Libc version: N/A
Python version: 3.11.2 (tags/v3.11.2:878ead1, Feb 7 2023, 16:38:35) [MSC v.1934 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19045-SP0
Is CUDA available: True
CUDA runtime version: 12.2.128
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce 930MX
Nvidia driver version: 536.67
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Architecture=9
CurrentClockSpeed=2703
DeviceID=CPU0
Family=198
L2CacheSize=512
L2CacheSpeed=
Manufacturer=GenuineIntel
MaxClockSpeed=2904
Name=Intel(R) Core™ i7-7500U CPU @ 2.70GHz
ProcessorType=3
Revision=
Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.25.2
[pip3] torch==2.0.1+cu118
[pip3] torchaudio==2.0.2+cu118
[pip3] torchvision==0.15.2+cu118
[conda] Could not collect