Is it possible to directly write activation to CPU memory?

I don’t think this is possible, but you could use the aforementioned utils. or offload the activation to the host after it was stored temporarily on the GPU as described here.