Hi
Any idea how can I use an unlearned kernel and still updating the original filter using the response to that filter?
Thanks 
Would you like to keep the “unlearned kernel” in its original state and use the gradients to update a clone of this kernel?
If so, note that the clone of this kernel would eventually get gradients for completely different parameters.
I’m unsure if I understood the question correctly, so please feel free to add more information about the use case.