# Memory Format **URL:** https://discuss.pytorch.org/c/memory-format/23.md [Latest](https://discuss.pytorch.org/latest.md) · [Categories](https://discuss.pytorch.org/categories.md) --- ## [About the Memory Format category](https://discuss.pytorch.org/t/about-the-memory-format-category/73663)
**Author:** [@albanD](https://discuss.pytorch.org/u/albanD)\ **Replies:** 0
--- ## [\[Project Showcase\] Disk Sparse Adam (DSA) — A disk-backed SparseAdam for large embeddings and GNNs](https://discuss.pytorch.org/t/project-showcase-disk-sparse-adam-dsa-a-disk-backed-sparseadam-for-large-embeddings-and-gnns/225269)
**Author:** [@Maksim\_Khotinsky](https://discuss.pytorch.org/u/Maksim_Khotinsky)\ **Replies:** 0\ **Last updated:** [August 8, 2026, 11:47pm UTC](https://discuss.pytorch.org/t/project-showcase-disk-sparse-adam-dsa-a-disk-backed-sparseadam-for-large-embeddings-and-gnns/225269 "2026-08-08T23:47:02Z")
Hi PyTorch Community! I wanted to share an open-source library I’ve developed to solve a common memory bottleneck when training large-scale sparse models: Disk Sparse Adam (DSA). :red\_circle: The Problem: Memory Overhe… --- ## [Cuda Memory Use Batched Matrix Multiplication](https://discuss.pytorch.org/t/cuda-memory-use-batched-matrix-multiplication/164131)
**Author:** [@buddit](https://discuss.pytorch.org/u/buddit)\ **Replies:** 7\ **Last updated:** [February 11, 2026, 10:45pm UTC](https://discuss.pytorch.org/t/cuda-memory-use-batched-matrix-multiplication/164131 "2026-02-11T22:45:42Z")
I’m trying to perform a batched matrix multiplication on a GPU and I get a CUDA out of memory error that doesn’t make a lot of sense to me. I’m hoping someone can shed some light on it. Specifically, I have a tensor A o… --- ## [Sparse bmm cause CUDA misaligned address error](https://discuss.pytorch.org/t/sparse-bmm-cause-cuda-misaligned-address-error/223661)
**Author:** [@gimal](https://discuss.pytorch.org/u/gimal)\ **Replies:** 2\ **Last updated:** [October 23, 2025, 8:17pm UTC](https://discuss.pytorch.org/t/sparse-bmm-cause-cuda-misaligned-address-error/223661 "2025-10-23T20:17:11Z")
Hi everyone, I’m new to pytorch, cuda and sparse memory format. I’m doing computation on batch sparse tensor, in this code: import torch from torch import Tensor SEED = 42 # torch.random.manual\_seed(SEED) def generat… --- ## [Free all GPU memory used in between runs](https://discuss.pytorch.org/t/free-all-gpu-memory-used-in-between-runs/168202)
**Author:** [@JSLJ23](https://discuss.pytorch.org/u/JSLJ23)\ **Replies:** 4\ **Last updated:** [August 18, 2025, 9:16am UTC](https://discuss.pytorch.org/t/free-all-gpu-memory-used-in-between-runs/168202 "2025-08-18T09:16:50Z")
Hi pytorch community, I was hoping to get some help on ways to completely free GPU memory after a single iteration of model training. This process is part of a Bayesian optimisation loop involving a molecular docking p… --- ## [Output mismatch with channels\_last\_3d?](https://discuss.pytorch.org/t/output-mismatch-with-channels-last-3d/221373)
**Author:** [@bartley](https://discuss.pytorch.org/u/bartley)\ **Replies:** 1\ **Last updated:** [July 9, 2025, 1:57pm UTC](https://discuss.pytorch.org/t/output-mismatch-with-channels-last-3d/221373 "2025-07-09T13:57:29Z")
Hi there, I am trying to implement a convnextv2 model in 3D. I would like the model to support torch.channels\_last\_3d memory format and torch.compile(). I think I am really close but I can not get the input/output to m… --- ## [Question: What takes so much gpu memory?](https://discuss.pytorch.org/t/question-what-takes-so-much-gpu-memory/221290)
**Author:** [@nanachiy](https://discuss.pytorch.org/u/nanachiy)\ **Replies:** 2\ **Last updated:** [July 6, 2025, 12:33pm UTC](https://discuss.pytorch.org/t/question-what-takes-so-much-gpu-memory/221290 "2025-07-06T12:33:11Z")
I am a college student majoring in astronomy, and would like to do FFT and de-dispersion in GPU with torch module. The original data is a time series of 4GB in float32 format, and the factor of de-dispersion is the same.… --- ## [Understand the memory allocation visualization](https://discuss.pytorch.org/t/understand-the-memory-allocation-visualization/221194)
**Author:** [@WindowsXp](https://discuss.pytorch.org/u/WindowsXp)\ **Replies:** 1\ **Last updated:** [July 1, 2025, 9:27am UTC](https://discuss.pytorch.org/t/understand-the-memory-allocation-visualization/221194 "2025-07-01T09:27:00Z")
I’m using https://docs.pytorch.org/memory\_viz (thanks for the amazing tool) to debug my OOM issue and finding it confused about the output. Take the screenshot below as an example, we focus on memory allocation A and B. … --- ## [Why the reserved memory is much larger than occupied memory?](https://discuss.pytorch.org/t/why-the-reserved-memory-is-much-larger-than-occupied-memory/221063)
**Author:** [@Zoe\_Zhao](https://discuss.pytorch.org/u/Zoe_Zhao)\ **Replies:** 1\ **Last updated:** [June 25, 2025, 10:06pm UTC](https://discuss.pytorch.org/t/why-the-reserved-memory-is-much-larger-than-occupied-memory/221063 "2025-06-25T22:06:03Z")
For my understanding, the y axis is the occupied memory. The part of the allocated memory that is actually being used to store data at a given moment — like active tensors, gradients, etc. Instead of frequently asking… --- ## [Why does GPU memory usage not double when loading two identical models in PyTorch?](https://discuss.pytorch.org/t/why-does-gpu-memory-usage-not-double-when-loading-two-identical-models-in-pytorch/220663)
**Author:** [@ximen0806449](https://discuss.pytorch.org/u/ximen0806449)\ **Replies:** 2\ **Last updated:** [June 9, 2025, 12:58pm UTC](https://discuss.pytorch.org/t/why-does-gpu-memory-usage-not-double-when-loading-two-identical-models-in-pytorch/220663 "2025-06-09T12:58:58Z")
Hi everyone, I have a question regarding GPU memory usage when loading identical models in PyTorch. I am working on a project where I need to load the same model twice onto the GPU. However, I noticed that when I load … --- ## [Questions about GPU memory usage](https://discuss.pytorch.org/t/questions-about-gpu-memory-usage/216246)
**Author:** [@rkd1137](https://discuss.pytorch.org/u/rkd1137)\ **Replies:** 2\ **Last updated:** [May 8, 2025, 1:25pm UTC](https://discuss.pytorch.org/t/questions-about-gpu-memory-usage/216246 "2025-05-08T13:25:39Z")
I’m trying to profile a model’s memory usage right now using this tutorial: Understanding GPU Memory 1: Visualizing All Allocations over Time | PyTorch. Looking at the output, almost all of the memory usage is listed as … --- ## [Out of Memory During LoRA Fine-Tuning on LLAMA-4-Scout-17B with H100 (80GB VRAM)](https://discuss.pytorch.org/t/out-of-memory-during-lora-fine-tuning-on-llama-4-scout-17b-with-h100-80gb-vram/219740)
**Author:** [@bensonbbn](https://discuss.pytorch.org/u/bensonbbn)\ **Replies:** 0\ **Last updated:** [May 4, 2025, 5:29pm UTC](https://discuss.pytorch.org/t/out-of-memory-during-lora-fine-tuning-on-llama-4-scout-17b-with-h100-80gb-vram/219740 "2025-05-04T17:29:51Z")
Dear community, I am the student currently working on training the LLAMA-4-Scout-17B-16E-Instruct model using LoRA, running on an H100 GPU with 80GB VRAM (on Lambda Labs). However, I have encountered an out of memory er… --- ## [Method for efficiently transferring non-autograd tensors to CPU from GPU?](https://discuss.pytorch.org/t/method-for-efficiently-transferring-non-autograd-tensors-to-cpu-from-gpu/218032)
**Author:** [@henryald](https://discuss.pytorch.org/u/henryald)\ **Replies:** 4\ **Last updated:** [April 9, 2025, 8:44am UTC](https://discuss.pytorch.org/t/method-for-efficiently-transferring-non-autograd-tensors-to-cpu-from-gpu/218032 "2025-04-09T08:44:59Z")
My code generates high dimensional Tensor objects on the GPU and subsequently stores these tensors for computation later on. However, there is a large possibility of the GPU running out of memory to store these samples. … --- ## [Clip Grad Norm on GPU without sync](https://discuss.pytorch.org/t/clip-grad-norm-on-gpu-without-sync/218045)
**Author:** [@PhysicsGaunt](https://discuss.pytorch.org/u/PhysicsGaunt)\ **Replies:** 1\ **Last updated:** [March 30, 2025, 1:38pm UTC](https://discuss.pytorch.org/t/clip-grad-norm-on-gpu-without-sync/218045 "2025-03-30T13:38:36Z")
( Not sure about the category, sorry :expressionless: ) I am running a training on CUDA and have noticed that the clip\_grad\_norm function seems to be quite slow, I believe this is due to the fact that is uses is\_nonzero… --- ## [Redefining model with fewer parameters → out of memory (OOM)?](https://discuss.pytorch.org/t/redefining-model-with-fewer-parameters-out-of-memory-oom/216793)
**Author:** [@Geremia](https://discuss.pytorch.org/u/Geremia)\ **Replies:** 1\ **Last updated:** [February 19, 2025, 10:58pm UTC](https://discuss.pytorch.org/t/redefining-model-with-fewer-parameters-out-of-memory-oom/216793 "2025-02-19T22:58:35Z")
Why when I redefine my model to have fewer parameters and retrain (after training once before) does PyTorch sometimes through an OOM error? In other words: How do I tune model hyperparameters without running into OOM er… --- ## [DtoH transfer of a partial tensor or untyped\_storage to tensor without memory allocation](https://discuss.pytorch.org/t/dtoh-transfer-of-a-partial-tensor-or-untyped-storage-to-tensor-without-memory-allocation/216884)
**Author:** [@aag](https://discuss.pytorch.org/u/aag)\ **Replies:** 0\ **Last updated:** [February 19, 2025, 11:25am UTC](https://discuss.pytorch.org/t/dtoh-transfer-of-a-partial-tensor-or-untyped-storage-to-tensor-without-memory-allocation/216884 "2025-02-19T11:25:41Z")
Hi there! I would like to convert a cupy function to torch: the purpose is to allocate pinned memory once, fill it with data (random size lower than allocated) and transfer it to the GPU. i tried with untyped\_storage b… --- ## [How can I decrease the pytorch confidence to hold too much reserved memory](https://discuss.pytorch.org/t/how-can-i-decrease-the-pytorch-confidence-to-hold-too-much-reserved-memory/215108)
**Author:** [@shwgao](https://discuss.pytorch.org/u/shwgao)\ **Replies:** 3\ **Last updated:** [January 8, 2025, 9:33pm UTC](https://discuss.pytorch.org/t/how-can-i-decrease-the-pytorch-confidence-to-hold-too-much-reserved-memory/215108 "2025-01-08T21:33:13Z")
In my case, the Pytorch seems ambitious to hold so much memory, and not free it give it back to cuda, I think it’s unreasonable, and I know it’s possible to free it through insert torch API in specific places in our code… --- ## [\`torch.cuda.is\_available()\` allocates unwanted memory?](https://discuss.pytorch.org/t/torch-cuda-is-available-allocates-unwanted-memory/196790)
**Author:** [@mmitjans](https://discuss.pytorch.org/u/mmitjans)\ **Replies:** 2\ **Last updated:** [December 11, 2024, 3:25pm UTC](https://discuss.pytorch.org/t/torch-cuda-is-available-allocates-unwanted-memory/196790 "2024-12-11T15:25:37Z")
I want to read how much total free memory each one of my GPU devices has, so that I can automatically assign the least used device to a new process I’m launching. For this, I’m using this function: def get\_least\_used\_gp… --- ## [Guarantee traversal order for optimiser states](https://discuss.pytorch.org/t/guarantee-traversal-order-for-optimiser-states/211476)
**Author:** [@Jackmin801](https://discuss.pytorch.org/u/Jackmin801)\ **Replies:** 0\ **Last updated:** [October 12, 2024, 1:56pm UTC](https://discuss.pytorch.org/t/guarantee-traversal-order-for-optimiser-states/211476 "2024-10-12T13:56:40Z")
Is there an easy way to consistently traverse the optimiser states? What I want to do is copy the optim states on gpu into a big flat tensor on cpu. In order to do this, I need to have an iterator that traverses the op… --- ## [How to share data among DataLoader processes to save memory](https://discuss.pytorch.org/t/how-to-share-data-among-dataloader-processes-to-save-memory/108772)
**Author:** [@blackberry](https://discuss.pytorch.org/u/blackberry)\ **Replies:** 6\ **Last updated:** [October 10, 2024, 2:03am UTC](https://discuss.pytorch.org/t/how-to-share-data-among-dataloader-processes-to-save-memory/108772 "2024-10-10T02:03:17Z")
PyTorch’s data loader uses multiprocessing in Python and each process gets a replica of the dataset. When the dataset is huge, this data replication leads to memory issues. Normally, multiple processes should use shared… --- ## [Libtorch CPP Api for Memory Format Channels Last](https://discuss.pytorch.org/t/libtorch-cpp-api-for-memory-format-channels-last/209218)
**Author:** [@Abhyuday98](https://discuss.pytorch.org/u/Abhyuday98)\ **Replies:** 1\ **Last updated:** [September 7, 2024, 3:29pm UTC](https://discuss.pytorch.org/t/libtorch-cpp-api-for-memory-format-channels-last/209218 "2024-09-07T15:29:40Z")
Background I am have been using the pytorch on Android. As mentioned in the following article link, the processing is faster on channel\_last format, and in a real world use case, I have also observed the same affect, wh… --- ## [Replacing torch.zeros internals with cudaMemset instead of fill kernel](https://discuss.pytorch.org/t/replacing-torch-zeros-internals-with-cudamemset-instead-of-fill-kernel/208272)
**Author:** [@anguswang](https://discuss.pytorch.org/u/anguswang)\ **Replies:** 2\ **Last updated:** [September 5, 2024, 6:15am UTC](https://discuss.pytorch.org/t/replacing-torch-zeros-internals-with-cudamemset-instead-of-fill-kernel/208272 "2024-09-05T06:15:35Z")
I’ve noticed that a fill kernel is called when constructing a tensor on the GPU with torch.zeros. Would it be possible and/or preferable to use cudaMemset instead? It seems like the more intuitive and efficient choice to… --- ## [Fancy idexing memory footprint](https://discuss.pytorch.org/t/fancy-idexing-memory-footprint/208288)
**Author:** [@Konsthr](https://discuss.pytorch.org/u/Konsthr)\ **Replies:** 0\ **Last updated:** [August 19, 2024, 2:31pm UTC](https://discuss.pytorch.org/t/fancy-idexing-memory-footprint/208288 "2024-08-19T14:31:45Z")
Hi! Is it possible to use the fancy indexing in PyTorch without returning a copy of the indexed tensor? For example I have a large 3D tensor X \[100,100,100\] and want to extract multiple (say 10) \[50,50,50\] slices to get… --- ## [Unable to free all GPU memory](https://discuss.pytorch.org/t/unable-to-free-all-gpu-memory/207894)
**Author:** [@Bar\_Dubovski](https://discuss.pytorch.org/u/Bar_Dubovski)\ **Replies:** 3\ **Last updated:** [August 12, 2024, 7:59am UTC](https://discuss.pytorch.org/t/unable-to-free-all-gpu-memory/207894 "2024-08-12T07:59:44Z")
Hello all, I have read many threads about ways to free memory and I wrote a simple example that tested my code, I believe I’m still missing something but cant seem to find what is it that I’m missing. As to my knowledg… --- ## [Why does tf.tile not make use of strided layout? And what about "inverse" strides?](https://discuss.pytorch.org/t/why-does-tf-tile-not-make-use-of-strided-layout-and-what-about-inverse-strides/207491)
**Author:** [@OEtzi007](https://discuss.pytorch.org/u/OEtzi007)\ **Replies:** 0\ **Last updated:** [August 3, 2024, 5:45pm UTC](https://discuss.pytorch.org/t/why-does-tf-tile-not-make-use-of-strided-layout-and-what-about-inverse-strides/207491 "2024-08-03T17:45:08Z")
I did a bit of experimentation on how torch makes use strided memory layout to save memory, and wondered about two things, that I think are related. First the simple one: tf.tile could make use of strided layout, but d… --- ## [Advanced Slicing](https://discuss.pytorch.org/t/advanced-slicing/207028)
**Author:** [@steinraf1](https://discuss.pytorch.org/u/steinraf1)\ **Replies:** 3\ **Last updated:** [July 28, 2024, 11:10am UTC](https://discuss.pytorch.org/t/advanced-slicing/207028 "2024-07-28T11:10:40Z")
I would like to slice a torch tensor in a pattern where the slice has a width that is not equal to one (what would basically be slicing off columns of a matrix) for a 1D tensor. \>\>\> a = torch.tensor(\[0.0, 0.0, 0.0, 0.0,… --- ## [Why aren't inputs to conv1d channels last?](https://discuss.pytorch.org/t/why-arent-inputs-to-conv1d-channels-last/132429)
**Author:** [@Yash\_Jakhotiya](https://discuss.pytorch.org/u/Yash_Jakhotiya)\ **Replies:** 3\ **Last updated:** [July 22, 2024, 4:49pm UTC](https://discuss.pytorch.org/t/why-arent-inputs-to-conv1d-channels-last/132429 "2024-07-22T16:49:33Z")
Just to replicate a historic approach (Kim, 2014), I am experimenting with conv1d for a sentence classification task. Embedding size for each word = 300. Sequence length (i.e. number of words in a sentence) = 512. So my… --- ## [Frombuffer() → "The given buffer is not writable"](https://discuss.pytorch.org/t/frombuffer-the-given-buffer-is-not-writable/204891)
**Author:** [@Geremia](https://discuss.pytorch.org/u/Geremia)\ **Replies:** 1\ **Last updated:** [June 19, 2024, 10:45pm UTC](https://discuss.pytorch.org/t/frombuffer-the-given-buffer-is-not-writable/204891 "2024-06-19T22:45:13Z")
¿Why when I run: file = open('/tmp/big.bin', 'rb') file\_bytes = file.read() mytensor = torch.frombuffer(file\_bytes, dtype=torch.float32).reshape(a, b) do I get this: UserWarning: The given buffer is not writable, and… --- ## [TensorDataset with lazy loading?](https://discuss.pytorch.org/t/tensordataset-with-lazy-loading/204191)
**Author:** [@Geremia](https://discuss.pytorch.org/u/Geremia)\ **Replies:** 4\ **Last updated:** [June 7, 2024, 6:28pm UTC](https://discuss.pytorch.org/t/tensordataset-with-lazy-loading/204191 "2024-06-07T18:28:43Z")
I have hundreds of gigs of data in the form of hundreds of thousands of files to use for training. I cannot load it all at once with torch.utils.data.TensorDataset() because I don’t have enough RAM to hold it all at once… --- ## [Understanding GPU memory visualization result](https://discuss.pytorch.org/t/understanding-gpu-memory-visualization-result/202002)
**Author:** [@111357](https://discuss.pytorch.org/u/111357)\ **Replies:** 1\ **Last updated:** [May 11, 2024, 1:13am UTC](https://discuss.pytorch.org/t/understanding-gpu-memory-visualization-result/202002 "2024-05-11T01:13:16Z")
There is a great tool for visualizing GPU memory. But I am a little confused about the allocator state history. Could someone assist in labeling the colors and the four rectangles? [Next page](https://discuss.pytorch.org/c/memory-format/23.md?page=1)