Hello and thanks for taking the time,
As a first attempt to utilize libTorch, I am trying to build the following example made available here:
#include <torch/torch.h>
template <typename T>
void pretty_print(const std::string& info, T&& data) {
std::cout << info << std::endl;
std::cout << data << std::endl << std::endl;
}
int main(int /*argc*/, char* /*argv*/[]) {
// Use GPU when present, CPU otherwise.
torch::Device device(torch::kCPU);
if (torch::cuda::is_available()) {
device = torch::Device(torch::kCUDA);
std::cout << "CUDA is available! Training on GPU." << std::endl;
}
const size_t kSequenceLen = 1;
const size_t kInputDim = 1;
const size_t kHiddenDim = 5;
const size_t kOuputDim = 1;
show original
However, using the latest libTorch library I have downloaded recently, I get this error:
error: cannot convert ‘at::Tensor’ to ‘c10::optional<std::tuple<at::Tensor, at::Tensor> >’
for this line:
auto rnn_output = time_serie_detector->forward(i_tmp, s_tmp);
The API has clearly changed since 2020, but I am entirely unsure how to fix this issue without affecting correctness. Could somebody please advise me how to do it right?
Thanks in advance!
eqy
May 28, 2023, 6:43am
2
It looks like you would need to separate the hidden state into h_0 and c_0 like in the Python API rather than having it packed together in a single tensor:
torch::Tensor state = torch::zeros({2, kSequenceLen, kHiddenDim});
should probably become something like
torch::Tensor h_0 = torch::zeros({kSequenceLen, kHiddenDim});
torch::Tensor c_0 = torch::zeros({kSequenceLen, kHiddenDim});
And then the forward method should be passed a tuple of h_0, c_0 as the second argument e.g.,
auto rnn_output = time_serie_detector->forward(i_tmp, std::make_tuple(h_0, c_0));
Note that rnn_output has changed to be a tuple as well, so the prints should also be changed to e.g.,
pretty_print("rnn_output/output: ", std::get<0>(rnn_output));
pretty_print("rnn_output/state: ", std::get<0>(std::get<1>(rnn_output)));
For more details you might want to check this PR which I believe contains the API change, and I’m not 100% sure I got the indicies of the tuples correct
committed 12:48AM - 16 Mar 20 UTC
Summary:
This PR refactors RNN / GRU / LSTM layers in C++ API to exactly match t… he implementation in Python API.
**BC-breaking changes:**
- Instead of returning `RNNOutput`, RNN / GRU forward method now returns `std::tuple<Tensor, Tensor>`, and LSTM forward method now returns `std::tuple<Tensor, std::tuple<Tensor, Tensor>>`, matching Python API.
- RNN / LSTM / GRU forward method now accepts the same inputs (input tensor and optionally hidden state), matching Python API.
- RNN / LSTM / GRU layers now have `forward_with_packed_input` method which accepts `PackedSequence` as input and optionally hidden state, matching the `forward(PackedSequence, ...)` variant in Python API.
- RNN / LSTM / GRU layers no longer have these fields: `w_ih` / `w_hh` / `b_ih` / `b_hh`. Instead, to access the weights and biases of the gates, users should do e.g. `rnn->named_parameters()["weight_ih_l0"]`, which mirrors the Python API `rnn.weight_ih_l0`.
- In `RNNOptions`
- `tanh()` / `relu()` / `activation` are removed. Instead, `nonlinearity` is added which takes either `torch::kTanh` or `torch::kReLU`
- `layers` -> `num_layers`
- `with_bias` -> `bias`
- In `LSTMOptions`
- `layers` -> `num_layers`
- `with_bias` -> `bias`
- In `GRUOptions`
- `layers` -> `num_layers`
- `with_bias` -> `bias`
The majority of the changes in this PR focused on refactoring the implementations in `torch/csrc/api/src/nn/modules/rnn.cpp` to match the Python API. RNN tests are then changed to reflected the revised API design.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/34322
Differential Revision: D20458302
Pulled By: yf225
fbshipit-source-id: ffff2ae1ddb1c742c966956f6ad4d7fba03dc54d