Trying to run PyTorch and Stable Diffusion and / or ComfyUI on FreeBSD

1

Hello.

I’m trying to run PyTorch and Stable Diffusion and / or ComfyUI on FreeBSD. The whole procedure is below. Unfortunately it does not work with a nvidia driver greater than 525…

marietto @ marietto:~ $ git clone h t t p s : / / g i t h u b . c o m / outpaddling /freebsd-ports-wip . git

marietto@marietto:~ $ cd /usr/ports/wip
marietto@marietto:~ $ pkg install auto-admin
marietto@marietto:~ $ auto-ports-wip-checkout
marietto@marietto:/usr/ports/wip $ cd linux-miniconda-installer
marietto@marietto:/usr/ports/wip/linux-miniconda-installer $ ls

Makefile    distinfo    pkg-descr    work

marietto@marietto:/usr/ports/wip/linux-miniconda-installer $ make clean

===>  Cleaning for linux-miniconda-installer-0.1.0.3
===>   /usr/ports/wip/linux-miniconda-installer/work not writable, skipping

marietto @ marietto:/usr/ports/wip/linux-miniconda-installer $ su
Password:

root @ marietto:/usr/ports/wip/linux-miniconda-installer # make clean

===>  Cleaning for linux-miniconda-installer-0.1.0.3

root @ marietto:/usr/ports/wip/linux-miniconda-installer # make

===>  License BSD2CLAUSE accepted by the user
===>   linux-miniconda-installer-0.1.0.3 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by linux-miniconda-installer-0.1.0.3 for building
===>  Extracting for linux-miniconda-installer-0.1.0.3
=> SHA256 Checksum OK for outpaddling-miniconda-installer-0.1.0-3-g0aa48c6_GH0.tar.gz.
===>  Patching for linux-miniconda-installer-0.1.0.3
===>   linux-miniconda-installer-0.1.0.3 depends on file: /usr/local/bin/python3.11 - found
===>  Configuring for linux-miniconda-installer-0.1.0.3
===>  Staging for linux-miniconda-installer-0.1.0.3
===>   linux-miniconda-installer-0.1.0.3 depends on package: auto-admin>=0.8.0 - found
===>   linux-miniconda-installer-0.1.0.3 depends on package: linux_base-rl9>=9.2 - found
===>   linux-miniconda-installer-0.1.0.3 depends on file: /usr/local/bin/python3.11 - found
===>   Generating temporary packing list
install  -m 555 /usr/ports/wip/linux-miniconda-installer/work/miniconda-installer-0.1.0-3-g0aa48c6/miniconda-installer /usr/ports/wip/linux-miniconda-installer/work/miniconda-installer-0.1.0-3-g0aa48c6/conda-shell  /usr/ports/wip/linux-miniconda-installer/work/stage/usr/local/bin
====> Compressing man pages (compress-man)

root @ marietto:/usr/ports/wip/linux-miniconda-installer # make install

===>  Installing for linux-miniconda-installer-0.1.0.3
===>  Checking if linux-miniconda-installer is already installed
===>   Registering installation for linux-miniconda-installer-0.1.0.3

Installing linux-miniconda-installer-0.1.0.3...

marietto @ marietto:/$ nvidia-smi

Thu May  8 00:51:53 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 565.77                 Driver Version: 565.77         CUDA Version: 12.7     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1060 3GB    Off |   00000000:01:00.0  On |                  N/A |
| 53%   32C    P8              7W /  120W |     339MiB /   3072MiB |      8%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce RTX 2080 Ti     Off |   00000000:02:00.0 Off |                  N/A |
| 31%   35C    P8             21W /  250W |       2MiB /  11264MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                            
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      4116      G   /usr/local/libexec/Xorg                       190MiB |
|    0   N/A  N/A      4154      G   xfwm4                                           2MiB |
|    0   N/A  N/A      4234      G   /usr/local/lib/firefox/firefox                142MiB |
+-----------------------------------------------------------------------------------------+

# pkg install linux-rl9-devtools linux-rl9-libglvnd
# cd /home/marietto
# conda-shell
# source conda.sh
# conda activate
# conda create --name pytorch python=3.12
# conda activate pytorch

# pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

# LD_PRELOAD="/home/marietto/.config/gpu/dummy-uvm.so" python3 -c 'import torch; print(torch.cuda.is_available())'

/home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/cuda/__init__.py:181: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount().

Did you run some cuda functions before calling NumCudaDevices() that might have already set an error?

Error 304: OS call failed or operation not supported on this OS (Triggered internally at /pytorch/c10/cuda/CUDAFunctions.cpp:109.)  return torch._C._cuda_getDeviceCount() > 0
False


$ pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu127

Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu127

Requirement already satisfied: torch in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (2.8.0.dev20250507+cu128)
Requirement already satisfied: torchvision in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (0.22.0.dev20250507+cu128)
Requirement already satisfied: torchaudio in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (2.6.0.dev20250507+cu128)
Requirement already satisfied: filelock in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (3.16.1)
Requirement already satisfied: typing-extensions>=4.10.0 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (4.12.2)
Requirement already satisfied: setuptools in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (78.1.1)
Requirement already satisfied: sympy>=1.13.3 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (1.13.3)
Requirement already satisfied: networkx in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (3.4.2)
Requirement already satisfied: jinja2 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (3.1.4)
Requirement already satisfied: fsspec in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (2024.10.0)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.8.61 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (12.8.61)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.8.57 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (12.8.57)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.8.57 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (12.8.57)
Requirement already satisfied: nvidia-cudnn-cu12==9.8.0.87 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (9.8.0.87)
Requirement already satisfied: nvidia-cublas-cu12==12.8.3.14 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (12.8.3.14)
Requirement already satisfied: nvidia-cufft-cu12==11.3.3.41 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (11.3.3.41)
Requirement already satisfied: nvidia-curand-cu12==10.3.9.55 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (10.3.9.55)
Requirement already satisfied: nvidia-cusolver-cu12==11.7.2.55 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (11.7.2.55)
Requirement already satisfied: nvidia-cusparse-cu12==12.5.7.53 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (12.5.7.53)
Requirement already satisfied: nvidia-cusparselt-cu12==0.6.3 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (0.6.3)
Requirement already satisfied: nvidia-nccl-cu12==2.26.2 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (2.26.2)
Requirement already satisfied: nvidia-nvtx-cu12==12.8.55 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (12.8.55)
Requirement already satisfied: nvidia-nvjitlink-cu12==12.8.61 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (12.8.61)
Requirement already satisfied: nvidia-cufile-cu12==1.13.0.11 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (1.13.0.11)
Requirement already satisfied: pytorch-triton==3.3.0+git96316ce5 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (3.3.0+git96316ce5)
Requirement already satisfied: numpy in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torchvision) (2.1.2)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torchvision) (11.0.0)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from sympy>=1.13.3->torch) (1.3.0)
Requirement already satisfied: MarkupSafe>=2.0 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from jinja2->torch) (2.1.5)

# conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia

# git clone https://github.com/comfyanonymous/ComfyUI.git


(pytorch) I have no name!@marietto:/usr/home/marietto$ cd ComfyUI
(pytorch) I have no name!@marietto:/usr/home/marietto/ComfyUI$ pip install -r requirements.txt
(pytorch) I have no name!@marietto:/usr/home/marietto/ComfyUI$ python -c 'import torch; print(torch.cuda.is_available())'

/home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/cuda/__init__.py:129: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). 

Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? 

Error 304: OS call failed or operation not supported on this OS (Triggered internally at /opt/conda/conda-bld/pytorch_1729647378361/work/c10/cuda/CUDAFunctions.cpp:108.)  return torch._C._cuda_getDeviceCount() > 0

False


(pytorch) I have no name!@marietto:/usr/home/marietto/ComfyUI$ LD_PRELOAD="${HOME}/.config/gpu/dummy-uvm.so" python -c 'import torch; print(torch.cuda.is_available())'

/home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/cuda/__init__.py:129: 

UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? 

Error 304: OS call failed or operation not supported on this OS (Triggered internally at /opt/conda/conda-bld/pytorch_1729647378361/work/c10/cuda/CUDAFunctions.cpp:108.)
  return torch._C._cuda_getDeviceCount() > 0
False


(pytorch) I have no name!@marietto:/usr/home/marietto$ conda install pytorch torchvision torchaudio pytorch-cuda=12.7 -c pytorch -c nvidia

Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - pytorch-cuda=12.7

In this thread you suggested to use cu126 :

so :

(pytorch) I have no name! @ marietto:/usr/home/marietto$ pip3 install --pre torch torchvision torchaudio --index-url h t t p s : / / d o w n l o a d . p y  to  r c h . o r g /w h l / nightly/ cu126

Looking in indexes: https://download.pytorch.org/whl/nightly/cu126

Requirement already satisfied: torch in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (2.8.0.dev20250507+cu128)
Requirement already satisfied: torchvision in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (0.22.0.dev20250507+cu128)
Requirement already satisfied: torchaudio in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (2.6.0.dev20250507+cu128)
Requirement already satisfied: filelock in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (3.16.1)
Requirement already satisfied: typing-extensions>=4.10.0 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (4.12.2)
Requirement already satisfied: setuptools in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (78.1.1)
Requirement already satisfied: sympy>=1.13.3 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (1.13.3)
Requirement already satisfied: networkx in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (3.4.2)
Requirement already satisfied: jinja2 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (3.1.4)
Requirement already satisfied: fsspec in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (2024.10.0)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.8.61 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (12.8.61)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.8.57 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (12.8.57)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.8.57 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (12.8.57)
Requirement already satisfied: nvidia-cudnn-cu12==9.8.0.87 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (9.8.0.87)
Requirement already satisfied: nvidia-cublas-cu12==12.8.3.14 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (12.8.3.14)
Requirement already satisfied: nvidia-cufft-cu12==11.3.3.41 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (11.3.3.41)
Requirement already satisfied: nvidia-curand-cu12==10.3.9.55 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (10.3.9.55)
Requirement already satisfied: nvidia-cusolver-cu12==11.7.2.55 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (11.7.2.55)
Requirement already satisfied: nvidia-cusparse-cu12==12.5.7.53 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (12.5.7.53)
Requirement already satisfied: nvidia-cusparselt-cu12==0.6.3 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (0.6.3)
Requirement already satisfied: nvidia-nccl-cu12==2.26.2 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (2.26.2)
Requirement already satisfied: nvidia-nvtx-cu12==12.8.55 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (12.8.55)
Requirement already satisfied: nvidia-nvjitlink-cu12==12.8.61 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (12.8.61)
Requirement already satisfied: nvidia-cufile-cu12==1.13.0.11 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (1.13.0.11)
Requirement already satisfied: pytorch-triton==3.3.0+git96316ce5 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch) (3.3.0+git96316ce5)
Requirement already satisfied: numpy in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torchvision) (2.1.2)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torchvision) (11.0.0)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from sympy>=1.13.3->torch) (1.3.0)
Requirement already satisfied: MarkupSafe>=2.0 in /home/marietto/miniconda3/envs/pytorch/lib/python3.12/site-packages (from jinja2->torch) (2.1.5)

no way…the only way to make work Stable diffusion is to use the nvidia driver and the linux-nvidia-libs vers. 525.78.01 and “torch==1.12.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113” and it works like a charme :

(pytorch) I have no name! @ marietto:/usr/home/marietto$ LD_PRELOAD="/compat/dummy-uvm.so" python3 -c 'import torch; print(torch.cuda.is_available())'

True

(pytorch) I have no name!@marietto:/usr/home/marietto$ LD_PRELOAD="/compat/dummy-uvm.so" python3 -c 'import torch; print(torch.cuda.get_device_name(0))'

NVIDIA GeForce RTX 2080 Ti

Unfortunately this driver and this version of pytorch/cuda is very old…

1 Like
2

Based on the error message it seems the new driver wasn’t properly installed. After installation you could compile any CUDA sample and execute it as a smoke test.

3

Are you sure ? because it works with the driver 525 and it stops working with a greater driver version. I don’t change anything from an experiment to another one,except the driver version.

4

It’s my best guess as it seems the driver fails to initialize the device:

CUDA initialization: Unexpected error from cudaGetDeviceCount()

so I would still recommend trying to run any small CUDA sample in this setup before trying to run a PyTorch workload.

5

This is what I did :

# pkg install libc6-shim linux-miniconda-installer linux-c7-devtools linux-c7 linux-c7-libglvnd

# fetch h t t p s : / / g i s t . g i t h u b u s e r c o n t e n t . c o m / shkhln / 40ef290463e78fb2b0000c60f4ad797e / raw / f640983249607e38af405c95c457ce4afc85c608 / uvm_ioctl_override.c

# /compat/linux/usr/bin # ./gcc --sysroot=/compat/linux -m64 -std=c99 -Wall -ldl -fPIC -shared -o /compat/linux/dummy-uvm.so /mnt/da2p2/Backup/uvm_ioctl_override.c
# /compat/linux/usr/bin# cp /compat/linux/dummy-uvm.so /home/marietto

CUDA tests :

  1. PASSED :
# nv-sglrun nvidia-smi

/usr/local/lib/libc6-shim/libc6.so: shim init
Fri May  9 01:31:16 2025

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.78.01    Driver Version: 525.78.01    CUDA Version: 12.0
  1. PASSED :
# nv-sglrun ffmpeg -i sample_1280x720_surfing_with_audio.mkv -vcodec hevc_nvenc -acodec copy test.mkv

/usr/local/lib/libc6-shim/libc6.so: shim init
ffmpeg version 6.1.2 Copyright (c) 2000-2024 the FFmpeg developers
  built with FreeBSD clang version 18.1.6
  configuration: --prefix=/usr/local --mandir=/usr/local/share/man --datadir=/usr/local/share/ffmpeg --docdir=/usr/local/share/doc/ffmpeg --pkgconfigdir=/usr/local/libdata/pkgconfig --disable-static --disable-libcelt --enable-shared --enable-pic --enable-gpl --cc=cc --cxx=c++ --disable-alsa --disable-libopencore-amrnb --disable-libopencore-amrwb --enable-libaom --disable-libaribb24 --disable-libaribcaption --enable-asm --enable-libass --disable-libbs2b --disable-libcaca --disable-libcdio --disable-libcodec2 --enable-libdav1d --disable-libdavs2 --disable-libdc1394 --disable-debug --enable-htmlpages --enable-libdrm --disable-libfdk-aac --disable-libflite --enable-fontconfig --enable-libfreetype --enable-frei0r --disable-libfribidi --disable-gcrypt --disable-libglslang --disable-libgme --enable-gmp --enable-gnutls --enable-version3 --disable-libgsm --enable-libharfbuzz --enable-iconv --disable-libilbc --disable-libjack --enable-libjxl --disable-libklvanc --disable-libkvazaar --disable-ladspa --enable-libmp3lame --enable-lcms2 --disable-liblensfun --disable-libbluray --enable-libplacebo --disable-librsvg --enable-libxml2 --enable-lto --disable-lv2 --disable-mbedtls --disable-libmfx --disable-libmodplug --disable-libmysofa --enable-network --disable-nonfree --enable-nvenc --disable-openal --disable-opencl --disable-opengl --disable-libopenh264 --disable-libopenjpeg --disable-libopenmpt --disable-openssl --disable-libopenvino --enable-optimizations --enable-libopus --disable-pocketsphinx --disable-libpulse --disable-librabbitmq --disable-librav1e --disable-librist --enable-runtime-cpudetect --disable-librubberband --disable-sdl2 --enable-libshaderc --disable-libsmbclient --disable-libsnappy --disable-sndio --disable-libsoxr --disable-libspeex --disable-libsrt --disable-libssh --enable-libsvtav1 --disable-libtensorflow --disable-libtesseract --disable-libtheora --disable-libtwolame --disable-libuavs3d --enable-libv4l2 --enable-vaapi --disable-vapoursynth --enable-vdpau --disable-libvidstab --enable-libvmaf --enable-libvorbis --disable-libvo-amrwbenc --disable-libvpl --enable-libvpx --enable-vulkan --enable-libwebp --enable-libx264 --enable-libx265 --disable-libxavs2 --enable-libxcb --disable-libxvid --disable-outdev=xv --disable-libzimg --disable-libzmq --disable-libzvbi
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, matroska,webm, from 'sample_1280x720_surfing_with_audio.mkv':
  Metadata:
    COMPATIBLE_BRANDS: isomiso2avc1mp41
    MAJOR_BRAND     : mp42
    MINOR_VERSION   : 512
    ENCODER         : Lavf58.45.100
  Duration: 00:03:03.13, start: 0.000000, bitrate: 3111 kb/s
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn (default)
    Metadata:
      HANDLER_NAME    : VideoHandler
      ENCODER         : Lavc58.91.100 libx264
      DURATION        : 00:03:03.061000000
  Stream #0:1(eng): Audio: vorbis, 48000 Hz, stereo, fltp (default)
    Metadata:
      HANDLER_NAME    : Stereo
      ENCODER         : Lavc58.91.100 libvorbis
      DURATION        : 00:03:03.129000000
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_nvenc))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
Output #0, matroska, to 'test.mkv':
  Metadata:
    COMPATIBLE_BRANDS: isomiso2avc1mp41
    MAJOR_BRAND     : mp42
    MINOR_VERSION   : 512
    encoder         : Lavf60.16.100
  Stream #0:0: Video: hevc (Main), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 2000 kb/s, 23.98 fps, 1k tbn (default)
    Metadata:
      HANDLER_NAME    : VideoHandler
      DURATION        : 00:03:03.061000000
      encoder         : Lavc60.31.102 hevc_nvenc
    Side data:
      cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: N/A
  Stream #0:1(eng): Audio: vorbis (oV[0][0] / 0x566F), 48000 Hz, stereo, fltp (default)
    Metadata:
      HANDLER_NAME    : Stereo
      ENCODER         : Lavc58.91.100 libvorbis
      DURATION        : 00:03:03.129000000
[out#0/matroska @ 0x85fa174c0] video:42344kB audio:2371kB subtitle:0kB other streams:0kB global headers:4kB muxing overhead: 0.222003%
frame= 4389 fps=669 q=17.0 Lsize=   44815kB time=00:03:03.11 bitrate=2004.9kbits/s speed=27.9x
  1. PASSED :
marietto@marietto:/usr/home/marietto/Desktop $ nv-sglrun clpeak

/usr/local/lib/libc6-shim/libc6.so: shim init

Platform: NVIDIA CUDA
  Device: NVIDIA GeForce RTX 2080 Ti
    Driver version  : 565.77 (FreeBSD)
    Compute units   : 68
    Clock frequency : 1545 MHz

    Global memory bandwidth (GBPS)
      float   : 505.42
      float2  : 533.59
      float4  : 549.18
      float8  : 551.64
      float16 : 491.24

Now let’s see if I’m able to run Stable Diffusion using the driver vers. 565.77 + CUDA Version: 12.7

I’ve followed this tutorial :

h t t p s : / / g i t h u b . c o m / v e r m / freebsd-stable-diffusion ? tab=readme-ov-file#stable-diffusion-webui

(base) I have no name!@marietto:/usr/home/marietto$ conda create --name pytorch python=3.10

Retrieving notices: ...working... done
WARNING: A conda environment already exists at '/home/marietto/miniconda3/envs/pytorch'
Remove existing environment (y/[n])? y

Collecting package metadata (current_repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 22.11.1
  latest version: 25.3.1

Please update conda by running

    $ conda update -n base -c defaults conda

Or to minimize the number of packages updated during conda update use

     conda install conda=25.3.1

## Package Plan ##

  environment location: /home/marietto/miniconda3/envs/pytorch

  added / updated specs:
    - python=3.10


The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main 
  _openmp_mutex      pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu 
  bzip2              pkgs/main/linux-64::bzip2-1.0.8-h5eee18b_6 
  ca-certificates    pkgs/main/linux-64::ca-certificates-2025.2.25-h06a4308_0 
  ld_impl_linux-64   pkgs/main/linux-64::ld_impl_linux-64-2.40-h12ee557_0 
  libffi             pkgs/main/linux-64::libffi-3.4.4-h6a678d5_1 
  libgcc-ng          pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1 
  libgomp            pkgs/main/linux-64::libgomp-11.2.0-h1234567_1 
  libstdcxx-ng       pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1 
  libuuid            pkgs/main/linux-64::libuuid-1.41.5-h5eee18b_0 
  ncurses            pkgs/main/linux-64::ncurses-6.4-h6a678d5_0 
  openssl            pkgs/main/linux-64::openssl-3.0.16-h5eee18b_0 
  pip                pkgs/main/noarch::pip-25.1-pyhc872135_2 
  python             pkgs/main/linux-64::python-3.10.16-he870216_1 
  readline           pkgs/main/linux-64::readline-8.2-h5eee18b_0 
  setuptools         pkgs/main/linux-64::setuptools-78.1.1-py310h06a4308_0 
  sqlite             pkgs/main/linux-64::sqlite-3.45.3-h5eee18b_0 
  tk                 pkgs/main/linux-64::tk-8.6.14-h39e8969_0 
  tzdata             pkgs/main/noarch::tzdata-2025b-h04d1e81_0 
  wheel              pkgs/main/linux-64::wheel-0.45.1-py310h06a4308_0 
  xz                 pkgs/main/linux-64::xz-5.6.4-h5eee18b_1 
  zlib               pkgs/main/linux-64::zlib-1.2.13-h5eee18b_1 


Proceed ([y]/n)? y


Downloading and Extracting Packages

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate pytorch
#
# To deactivate an active environment, use
#
#     $ conda deactivate

(base) I have no name!@marietto:/usr/home/marietto$ conda update -n base -c defaults conda

Collecting package metadata (current_repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 22.11.1
  latest version: 25.3.1

Please update conda by running

    $ conda update -n base -c defaults conda

Or to minimize the number of packages updated during conda update use

     conda install conda=25.3.1

# All requested packages already installed.

(base) I have no name!@marietto:/usr/home/marietto$ conda activate pytorch

(pytorch) I have no name!@marietto:/usr/home/marietto$ pip install torch==1.12.1+cu113 --extra-index-url  h t t p s : / / d o w n l o a d . pytorch . org / whl / cu113
Looking in indexes: h t t p s : / / p y p i . o r g / simple, h t t p s : / / d o w n l o a d . pytorch.org/ whl / cu113

Collecting torch==1.12.1+cu113
  Using cached h t t p s : / / d o w n l o a d . pytorch . org /whl / cu113 / torch-1.12.1%2Bcu113-cp310-cp310-linux_x86_64.whl (1837.7 MB)
Collecting typing-extensions (from torch==1.12.1+cu113)
  Using cached typing_extensions-4.13.2-py3-none-any.whl.metadata (3.0 kB)
Using cached typing_extensions-4.13.2-py3-none-any.whl (45 kB)
Installing collected packages: typing-extensions, torch
Successfully installed torch-1.12.1+cu113 typing-extensions-4.13.2

(pytorch) I have no name!@marietto:/usr/home/marietto$ LD_PRELOAD="./dummy-uvm.so" python3 -c 'import torch; print(torch.cuda.is_available())'

/home/marietto/miniconda3/envs/pytorch/lib/python3.10/site-packages/torch/cuda/__init__.py:83: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 304: OS call failed or operation not supported on this OS (Triggered internally at  ../c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0
False

FAILED : It will work if the nvidia driver is 525 or 535. With an higher version you saw what happens. My suspect is that python 3.10 and/ or torch==1.12.1+cu113 are wrong. Now the ball is in your hand.