Hi,
I noticed that the values of these two expressions output_teacher_batch.data.cpu().numpy(), output_teacher_batch.data.cpu() are different. In which the output is pytorch Variable. the following piece(training loop of CNN) of code is how I checked it. this is really confusing, why these two expressions have different values. appreciate any idea
with tqdm(total=len(dataloader)) as t:
for i, (train_batch, labels_batch) in enumerate(dataloader):
# move to GPU if available
if params.cuda:
train_batch, labels_batch = train_batch.cuda(async=True), labels_batch.cuda(async=True)
# convert to torch Variables
train_batch, labels_batch = Variable(train_batch), Variable(labels_batch)
# compute model output, fetch teacher output, and compute KD loss
output_batch = model(train_batch)
# get one batch output from teacher_outputs list
output_teacher_batch = torch.from_numpy(teacher_outputs[i])
if params.cuda:
output_teacher_batch = output_teacher_batch.cuda(async=True)
output_teacher_batch = Variable(output_teacher_batch, requires_grad=False)
## here we should pass only those values of output that related to each cluster not all
print('teacher')
np.savetxt('version1.txt', output_teacher_batch.data.cpu().numpy())
print(output_teacher_batch.data.cpu())
Are you sure the values are different or could it be just an issue of the printing precision?
You can change it with torch.set_printoptions(precision=6)
and see if numpy prints the same values as PyTorch.
This should be the case as the underlying storage is shared between numpy and PyTorch.
these are some of the results of the output_teacher_batch.data.cpu():
5.889701 -2.200435 2.725899 …
-1.187858 -2.000502 1.003952 …
0.092105 -1.677588 -0.427267 …
… ⋱
and these are the corresponding results of the output_teacher_batch.data.cpu().numpy():
2.333767890930175781e+00 1.281759548187255859e+01 -1.557306051254272461e+00 …
-1.450318336486816406e+00 -3.302952051162719727e+00 4.208591461181640625e+00 …
-5.082679986953735352e-01 -2.980945587158203125e+00 -1.338094949722290039e+00 …
it does not seem that the precision be the problem
Could you post the code you are using to print these two tensors?
In your code snippet I can only find the first print.
I saved the other one in the file version1.txt, through this line np.savetxt(‘version1.txt’, output_teacher_batch.data.cpu().numpy())
I cannot reproduce this issue. The saved numpy array equals the tensor in the print statement on my machine.
Could you print the following statement in your code:
np.allclose(output_teacher_batch.cpu(), output_teacher_batch.cpu().numpy())
did you np.allclose(output_teacher_batch.data.cpu(), output_teacher_batch.data.cpu().numpy()) ? if yes, I tried it and it return true. so I guess it means these two have equal values, but how come I comparing them myself they are not the same