Hi, I am trying to make an RL agent and environment, in this process I am using numpy array basically for action space which is a shape of [3]. So the main input and output of the neural net agent class which extends the torch.nn module, is a numpy ndarray. And the reward and other primary variables are numpy array too. But, the input of agent should be torch tensor. For example:
action, prediction = agent.act(state)
next_state, reward, done = env.step(action)
states.append(np.expand_dims(state, axis=0))
next_states.append(np.expand_dims(next_state, axis=0))
action_onehot = np.zeros(3)
In the code shown that is a function is based on numpy array like expand. And the challeng is transforming the numpy array to torch tensor and vice versa.
After all, I came up with three options and now I want to know which one is better to use(or any other option you may give):
1.Use the from_numpy function and convert the array to tensor before inputting it to the agent and then again transform it to numpy after prediction?
2.Use the transformers(from numpy and to numpy) in the forward function of the agent class?
3.Use torch tensor instead of numpy array in the whole process(which may not possible)?
And I believe keeping the grad of torch tensor is crucial, So please consider this.
Please explain how should I do the task.
Your answer much appreciated in regard.