I’d like to use CNN for reinforcement Learning and I have a question regarding the size of the first dense layer after the feature maps.
I have a setup that seems to work for now: Input would be a (1,1,50,50) Tensor ie: One image with only one channel with dim 50x50. It goes in conv2d filter, a non-linear activation. Afterwards, it is flatten and sent to the dense layer to be used for action selection.
I’d like to send multiples image to train the network offline (ie: after finishing an episode, I want to pack all the states, all the images, and send them at once so I can get state estimations). Doing so, I always meet an error saying that the first dense layer after filters doesn’t have a good dimensionality.
Does it mean that I have to keep sending them one by one ?