Can't call numpy() on Variable that requires grad

vision
1
  • Error informaiton :

There is only one GPU
Traceback (most recent call last):
File “train.py”, line 123, in
test_train.start_train(epoch=30,batch_display=2000,batch_size=50,save_freq=1)
File “train.py”, line 96, in start_train
pcc = scipy.stats.pearsonr(gtlabel, output.squeeze(-1))[0]
File “/home/simon/anaconda3/lib/python3.6/site-packages/scipy/stats/stats.py”, line 3003, in pearsonr
y = np.asarray(y)
File “/home/simon/anaconda3/lib/python3.6/site-packages/numpy/core/numeric.py”, line 492, in asarray
return array(a, dtype, copy=False, order=order)
File “/home/simon/anaconda3/lib/python3.6/site-packages/torch/tensor.py”, line 376, in array
return self.cpu().numpy()

  • Codes involved:
    labels_batch = torch.FloatTensor(labels_batch.view(-1,self.num_classes).detach().numpy())
2 Likes
2

Hi,

This is expected behavior because moving to numpy will break the graph and so no gradient will be computed.
If you don’t actually need gradients, then you can explicitly .detach() the Tensor that requires grad to get a tensor with the same content that does not require grad. This other Tensor can then be converted to a numpy array.

4 Likes
3

Hi albanD, I am a newer to the PyTorch, can you please provide me with more informaiton? Cheers!

4

I have tried the line as
labels_batch = torch.FloatTensor(labels_batch.wiew(-1,self.num_classes).numpy(), requires_grad=True)

but, there is the problem as follows

There is only one GPU
Traceback (most recent call last):
File “train.py”, line 123, in
test_train.start_train(epoch=30,batch_display=2000,batch_size=50,save_freq=1)
File “train.py”, line 85, in start_train
labels_batch = torch.FloatTensor(labels_batch.view(-1,self.num_classes).numpy(), requires_grad=True)
TypeError: new() received an invalid combination of arguments - got (numpy.ndarray, requires_grad=bool), but expected one of:

  • (torch.device device)
  • (tuple of ints size, torch.device device)
    didn’t match because some of the keywords were incorrect: requires_grad
  • (torch.Storage storage)
  • (Tensor other)
  • (object data, torch.device device)
    didn’t match because some of the keywords were incorrect: requires_grad

Can you plz help me solve it?

5

I don’t understand what do you try to achieve here?

6

After I deteleted the code of ‘requires_grad=True’, similar error occured as

There is only one GPU
Traceback (most recent call last):
File “train.py”, line 123, in
test_train.start_train(epoch=30,batch_display=2000,batch_size=50,save_freq=1)
File “train.py”, line 96, in start_train
pcc = scipy.stats.pearsonr(gtlabel, output.squeeze(-1))[0]
File “/home/simon/anaconda3/lib/python3.6/site-packages/scipy/stats/stats.py”, line 3003, in pearsonr
y = np.asarray(y)
File “/home/simon/anaconda3/lib/python3.6/site-packages/numpy/core/numeric.py”, line 492, in asarray
return array(a, dtype, copy=False, order=order)
File “/home/simon/anaconda3/lib/python3.6/site-packages/torch/tensor.py”, line 376, in array
return self.cpu().numpy()
RuntimeError: Can’t call numpy() on Variable that requires grad. Use var.detach().numpy() instead.

7

If you’re trying to compute your loss with pcc = scipy.stats.pearsonr(gtlabel, output.squeeze(-1))[0]. Then this is not possible. The only operations that are allowed to be able to differentiate them are pytorch operations.
If you’re using pcc = scipy.stats.pearsonr(gtlabel, output.squeeze(-1))[0] just to get some metric but don’t need to backprop through it, then you need to detach() your output before giving it to this function as output.detach().squeeze(-1).

8

The story behind this error can be described as follows:

At first, I used the MSE-based loss function in the python file as
loss_function = nn.MSELoss().cuda()

the loss was input as
loss = loss_function(output, gtlabel)

in addition, thee line of ‘labels_batch=…’ was input as
labels_batch = torch.FloatTensor(labels_batch.wiew(-1,self.num_classes).numpy())

These codes was run smoothly!

At this time, I want to use another loss function, i.e., 1-CCC,
and the codes are changed as

            pcc = scipy.stats.pearsonr(gtlabel, output.squeeze(-1))[0]
            loss = 1-2*pcc*np.std(gtlabel)*np.std(output)/((np.std(gtlabel))**2+(np.std(output))**2+(np.mean(output)-np.mean(gtlabel))**2)

When I use the line of " labels_batch = torch.FloatTensor(labels_batch.wiew(-1,self.num_classes).numpy())", there is the error listed in the previous posts,

So my colleague suggested me to add ‘requires_grad=True’ to this line, but there is the new error as listed in the previous post.
That’s all!

9

As I stated above, you cannot use numpy functions to compute your loss. Even you hack it to do so, gradient won’t be computed.
You have to use pytorch’s function only to be able to get gradients.

1 Like
10

I have got it and solved my problem. Cheers, man!

11

Hi I have the same error in new version of torch (1.5). in previous version (1.4) my program worked with no error. what do you suggest to do to solve the error.

12

Hi,

Do you have a code sample that work with 1.4 and fails now please?

13

Hi, Thank you for your reply. I found out that it is related to following part related to plotting my results.
I commented this part and solved the problem. since it was not so important, I ignored that part.

# def plot_constellation(

#     ax,

#     constellation,

#     channel,

#     decoder,

#     grid_step=0.05,

#     noise_samples=1000):

#     """

#     Plot a constellation with its decoder and channel noise.

#     :param ax: Matplotlib axes to plot on.

#     :param constellation: Constellation to plot.

#     :param channel: Channel model to use for generating noise.

#     :param decoder: Decoder function able to map the constellation points back

#     to the original messages.

#     :param grid_step: Grid step used for drawing the decision regions,

#     expressed as percentage of the total plot width (or equivalently height).

#     Lower steps makes more precise grids but takes more time to compute.

#     :param noise_samples: Number of noisy points to sample and plot.

#     """

#     ax.grid()

#     order = len(constellation)

#     color_map = ListedColormap(seaborn.color_palette('husl', n_colors=order))

#     color_norm = matplotlib.colors.BoundaryNorm(range(order + 1), order)

#     # Extend axes symmetrically around zero so that they fit data

#     axis_extent = max(

#         abs(constellation.min()),

#         abs(constellation.max())

#     ) * 1.05

#     ax.set_xlim(-axis_extent, axis_extent)

#     ax.set_ylim(-axis_extent, axis_extent)

#     # Hide borders but keep ticks

#     for direction in ['left', 'bottom', 'right', 'top']:

#         ax.axis[direction].line.set_color('#00000000')

#     # Show zero-centered axes without ticks

#     for direction in ['xzero', 'yzero']:

#         axis = ax.axis[direction]

#         axis.set_visible(True)

#         axis.set_axisline_style('-|>')

#         axis.major_ticklabels.set_visible(False)

#     # Add axis names

#     ax.annotate(

#         'I', (1, 0.5), xycoords='axes fraction',

#         xytext=(25, 0), textcoords='offset points',

#         va='center', ha='right'

#     )

#     ax.annotate(

#         'Q', (0.5, 1), xycoords='axes fraction',

#         xytext=(0, 25), textcoords='offset points',

#         va='center', ha='center'

#     )

#     # Plot decision regions

#     regions_extent = 2 * axis_extent

#     step = grid_step * regions_extent

#     grid_range = torch.arange(-regions_extent, regions_extent, step)

#     grid_y, grid_x = torch.meshgrid(grid_range, grid_range)

#     grid_points = torch.stack((grid_x, grid_y), dim=-1).flatten(end_dim=1)

#     grid_images = decoder(grid_points).argmax(dim=-1).reshape(grid_x.shape)

#     ax.imshow(

#         grid_images,

#         extent=(

#             -regions_extent, regions_extent,

#             -regions_extent, regions_extent

#         ),

#         aspect='auto',

#         origin='lower',

#         cmap=color_map,

#         norm=color_norm,

#         alpha=0.2

#     )

#     # Plot constellation

#     ax.scatter(

#         *zip(*constellation.tolist()),

#         zorder=10,

#         s=60,

#         c=range(order),

#         edgecolor='black',

#         cmap=color_map,

#         norm=color_norm,

#     )

#     # Plot constellation center

#     center = constellation.sum(dim=0) / order

#     ax.scatter(

#         center[0], center[1],

#         marker='X',

#     )

#     # Plot channel noise

#     if noise_samples > 0:

#         noisy_vectors = channel(constellation.repeat(noise_samples, 1))

#         ax.scatter(

#             *zip(*noisy_vectors.tolist()),

#             marker='.',

#             s=5,

#             c=list(range(order)) * noise_samples,

#             cmap=color_map,

#             norm=color_norm,

#             alpha=0.3,

#             zorder=8

#         )
14

You are most likely missing a detach in there if this is just plotting code.