Conda - Huge disk usage

1

I just saw that my conda environment is 13GB in size. I looked up which packages are taking more space and they are mostly pytorch / cuda modules.

I wonder if that’s normal or if there’s any way to avoid using so much disk space.

This is the list of the modules that take more space in my environment and their respective sizes.


pytorch-1.13.0-py3.10_cuda11.7_cudnn8.5.0_0.json:  "size":  1229942678,
nsight-compute-2022.3.0.22-0.json:                 "size":  639598244,
libcublas-dev-11.11.3.6-0.json:                    "size":  413253930,
libcublas-11.11.3.6-0.json:                        "size":  381637889,
libcusparse-dev-11.7.5.86-0.json:                  "size":  377165256,
libcufft-dev-10.9.0.58-0.json:                     "size":  289240757,
mkl-2021.4.0-h06a4308_640.json:                    "size":  229783051,
libcusparse-11.7.5.86-0.json:                      "size":  184891885,
libnpp-11.8.0.86-0.json:                           "size":  154995740,
libnpp-dev-11.8.0.86-0.json:                       "size":  151472361,
libcufft-10.9.0.58-0.json:                         "size":  149741913,
cuda-nvvp-11.8.87-0.json:                          "size":  119905249,
cuda-nsight-11.8.86-0.json:                        "size":  119143043,
libcusolver-11.4.1.48-0.json:                      "size":  101143771,

I used grep '"size":' ${CONDA_PREFIX}/conda-meta/*.json | sort -k3rn | sed 's/.*conda-meta\///g' | column -tto output the previous list.

Thanks,
B.

2

I’m unsure how the 13GB are calculated as your output shows ~3-4GB of usage.
In any case, the issue is also tracked here and this PR should already reduce the size.

3

The 13GB were calculated with: du -sh miniconda3/envs/env_name .

The output I pasted is not the whole output of the command in the initial message, it was just the first few lines (out of 150). I saw that among the ones that took more space there are pytorch and cuda-related stuff. I was suspecting these are taking a large contribution in the 13GB, and wondering if this is normal.

I agree there are other packages that take a lot of space, but it feels like it is always the environments where I use PyTorch with CUDA, that take way more disk space than the rest, whether directly through PyTorch/CUDA or through its dependencies.

Thanks,
B.