Hi all,
I have a question regarding the usage of max_split_size_mb. On a technical level, I don’t understand how setting this parameter reduces external memory fragmentation.
My understanding is that Pytorch allocates memory blocks dynamically rather than allocating fixed-size blocks. In that sense, isn’t it better to have higher values for max_split_size_mb, as that reduces the risk of lots of small blocks being allocated, with unusable blocks in between. However, if there is no default maximum value for max_split_size_mb, then surely this variable is already optimised for the type of memory allocation used by Pytorch?
Thanks