Copy.deepcopy or clone() cause UserWarning

1

Hi all,
I got the error message opt/conda/conda-bld/pytorch_1579022060824/work/aten/src/ATen/native/cudnn/RNN.cpp:1266: UserWarning: RNN module weights are not part of single contiguous chunk of memory. This means they need to be compacted at every call, possibly greatly increasing memory usage. when I tried to run the following code to store the weights inside optimizer class.

for p in group['params']:
    # p_data = p.data.clone() 
    p_data = copy.deepcopy(p.data)

I noticed people said this message is related to nn.Parallel, but I am not using parallel run. I tested if I remove clone or deepcopy, there won’t be warning message. Do you know how to fix this?
Thank you.

2

Hi,

Have you tried using module.flatten_parameters() on your RNN module to make sure the weights are nicely layed out in memory again?

3

Hi albanD,
Thank you. I created this module by combining nn.RNN and nn.Linear, say I name it rnn_model(). But rnn_model.flatten_parameters() gives object has no attribute ‘flatten_parameters’. Where should we call flatten_parameters to make sure the weights are nicely laid out in memory again in this case? Thank you for your time.

4

Hey,

I am not sure, that depends on your network structure.
Is your rnn_model wrapped into a Sequential or some container? Make sure to call that method on the rnn model itself.

1 Like