I have the current situation: after getting the output from a simple neural network, let’s call this x, x is then used in another process, said process subsequently yields a result which is evaluated to compute a reward, which I then use to compute a loss function.
The x tensor object is not directly connected to the process, since I can only take the x.item<>() values to use it in the process. Hence, the output of the neural network is not directly connected to the loss function.
Is there a simple (or any) way in which the output tensor x could be connected to the loss function given the setting I described? This would be necessary to perform backpropagation given the loss function.
So in short you have an output, you pass that output through a blackbox and you get an scalar.
What you want is to backprop that scalar.
The easiest way is trying to write that blackbox in pytorch.
Else, (i think) you can write a module which does so but you would need to be able to calculate gradients of scalar wrt output.
Hello there! I’m having a very similar problem. In my case that “blackbox” is a script called pixelmatch which compares two images and returns a scalar in the form of a percentage of how similar the pixels in the images are. How would you go about implementing something like this in pytorch?
Thank you for your time!
Well again,
A) You need gradients
i)If you know the algorithm used you can implement it with pytorch functions.
ii) You can use the code as it is (won’t be very efficient) and implement the backward function. This is, telling pytorch how to get the gradients of the input wrt the output.
B) You don’t need gradients
Then you can just use the code as it is.
My issue is that I’m controlling a rendering software (Blender) with the outputs of my CNN and then I compare the rendered image to a target image. I don’t think there’s a way to convert this into pytorch functions.
And to be perfectly honest: I don’t know too much about CNNs but my understanding is that I need the gradients to backprop. But my percentage is calculated and output in the form of a single scalar with no graph. How would I plug that percentage into the graph so I can backpropagate?
And thank you for your help!
Yes, you need gradiens to backprop. I don’t know which kind of predictions your netowrk is doing but it sounds like you need to pose your problem in function of the network’s predictions rather than blender.
Anyway controlling a software sound like a very complicated task far away of what you can do just with a CNN. (You can google reinforcement learning)
Anyway, it seems you are getting a “command” from the cnn which later on you pass to blender. Then you try to match the result and a target “image”. You should try to pose the problem in a different way, where you know which command the network should predict to generate the desired effect and use that as target.
Hi, I also face the similar problem. Since the output of NN could not be used to calculate loss, it has to be transported to an observable sensor signal, which chould be compare with desired ones.
For real application, I need RNN to generate driving signal for my robot motor, however, I do not know the optimal driving signals. The only criteria is the performance of the robot under aforementioned driving signals matches what we want.
The relationship between driving signal and response are 90% known with some uncertainties, thus I would like not to use another NN to training the relationship, which increases the complexity.
Might I call it indirect loss function trainning? People suggest me to use Straight Through Estimator to connect the driving signal to the observable sensor signal. Also, I research on Directed Acyclic Graph, wondering how to make the “blackbox” operation in the graph for loss backpropagation.
Defining the mid-variables to be “requires_grad=True” seems not helpful, the loss keeps all the same during training.