# After torch::load model and predict, then got NaN

**URL:** <https://discuss.pytorch.org/t/after-torch-load-model-and-predict-then-got-nan/133142>\
**Category:** C++\
**Created:** [September 30, 2021, 2:23am UTC](https://discuss.pytorch.org/t/after-torch-load-model-and-predict-then-got-nan/133142 "2021-09-30T02:23:09Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![sudri](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sudri/32/66770_2.png) [@sudri](https://discuss.pytorch.org/u/sudri)\
**Post date:** [September 30, 2021, 2:23am UTC](https://discuss.pytorch.org/t/after-torch-load-model-and-predict-then-got-nan/133142/1 "2021-09-30T02:23:09Z")

</div>

When I was training and validating the model, the output was all normal. After training, I called `torch::save()` to save the model to a `.pt` file, and then called `torch::load()` to load the model from the file to make predictions. At this time, **the predicted value becomes NaN**. I have checked the predicted network structure and network data type(Double), input data and its data type(Double), they are all correct. I have been troubled by this problem for a few days, please help me.

_ **Network** _

```cpp
struct NetImpl : torch::nn::Module {
    NetImpl(int in_feature, int out_feature)
        : fc1(in_feature, 10),
        fc2(10, 100),
        fc3(100, 500),
        fc4(500, 250),
        fc5(250, 125),
        fc6(125, 70),
        fc7(70, out_feature)
    {
        register_module("fc1", fc1);
        register_module("fc2", fc2);
        register_module("fc3", fc3);
        register_module("fc4", fc4);
        register_module("fc5", fc5);
        register_module("fc6", fc6);
        register_module("fc7", fc7);
    }

    torch::Tensor forward(torch::Tensor x) {
        x = torch::leaky_relu(fc1->forward(x));
        x = torch::leaky_relu(fc2->forward(x));
        x = torch::leaky_relu(fc3->forward(x));
        x = torch::leaky_relu(fc4->forward(x));
        x = torch::leaky_relu(fc5->forward(x));
        x = torch::leaky_relu(fc6->forward(x));
        x = fc7->forward(x);
        return x;
    }
    torch::nn::Linear fc1, fc2, fc3, fc4, fc5, fc6, fc7;
};
TORCH_MODULE(Net);

```

**Save and load then got output as below**

```cpp
torch::save(model, model_path);
torch::load(model, model_path);
std::cout << model << std::endl;

```

> NetImpl(  
> (fc1): torch::nn::Linear(in\_features=5, out\_features=10, bias=true)  
> (fc2): torch::nn::Linear(in\_features=10, out\_features=100, bias=true)  
> (fc3): torch::nn::Linear(in\_features=100, out\_features=500, bias=true)  
> (fc4): torch::nn::Linear(in\_features=500, out\_features=250, bias=true)  
> (fc5): torch::nn::Linear(in\_features=250, out\_features=125, bias=true)  
> (fc6): torch::nn::Linear(in\_features=125, out\_features=70, bias=true)  
> (fc7): torch::nn::Linear(in\_features=70, out\_features=2, bias=true)  
> )

**Predict Method**

```cpp
template <typename DataLoader>
void predict(Net& model,
    DataLoader& data_loader,
    size_t dataset_size) {
    //torch::NoGradGuard no_grad;
    model->eval();
    model->zero_grad();
    auto loss_val = 0.0;
    for (auto& batch : *data_loader) {
        auto data = batch.data, target = batch.target;
        auto output = model->forward(data);
        std::cout << output << std::endl;
        loss_val += torch::mse_loss(output, target).template item<float>();
    }
    loss_val /= dataset_size;
    std::printf("\nTest set: Average loss: %.8f\n", loss_val);
}

```

**Result**

 ![20210930101752](https://discuss.pytorch.org/uploads/default/original/3X/b/d/bdef6f11ae924509bc4687527168b7192d388615.png)

**Model File**  
If necessary, I will upload it to Google Drive.

**Environment**  
libtorch stable 1.9.1  
libtorch preview(nightly)  
Microsoft Visual Studio 2019  
Windows 10

Any suggestions and help are greatly appreciated!

* * *

**UPDATE**  
When the tensor performs the first few forward operations, the output value produces several NaNs.

```cpp
 x = torch::leaky_relu(fc2->forward(x));
std::cout << x << std::endl;

```

If tensor continue to forward, all subsequent outputs will be NaN. I think this may be the cause of the problem, but how should it be solved?

 ![20210930130037](https://discuss.pytorch.org/uploads/default/original/3X/1/2/12b6547258a8058700ddba4a05e1646c50d9aa30.png)

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [September 30, 2021, 9:05am UTC](https://discuss.pytorch.org/t/after-torch-load-model-and-predict-then-got-nan/133142/2 "2021-09-30T09:05:37Z")

</div>

Could you check the output of `fc1` and its values?  
The printed values of `x = torch::leaky_relu(fc2->forward(x));` have a huge range so it would be interesting to see what ranges the input to the model has (min. and max. in particular) as well as the parameters of `fc1`.

---

<div class="post-metadata">

**Author:** ![sudri](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sudri/32/66770_2.png) [@sudri](https://discuss.pytorch.org/u/sudri)\
**Post date:** [September 30, 2021, 9:30am UTC](https://discuss.pytorch.org/t/after-torch-load-model-and-predict-then-got-nan/133142/3 "2021-09-30T09:30:02Z")

</div>

Hi, Thanks for help.

I made a small change in order to observe the change in value. The code and output results are shown below.

```cpp
std::cout << x << std::endl; // print normalized input data

x = fc1->forward(x);
std::cout << x << std::endl;

x = torch::leaky_relu(x);
std::cout << x << std::endl;

x = fc2->forward(x);
std::cout << x << std::endl;

```

 ![20210930172440](https://discuss.pytorch.org/uploads/default/original/3X/e/a/ead1f2026a846d04b6d0196e1eb9ea4c7bd1dc10.png)

I built the same network on PyTorch, but his output was normal, with no large or small values. It is worth mentioning that in LibTorch, I initialized the network to **Double type** by using code below, maybe this is a problem?

```cpp
model->to(device, torch::kDouble);

```

Looking forward to your reply, Thanks!

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [September 30, 2021, 10:33am UTC](https://discuss.pytorch.org/t/after-torch-load-model-and-predict-then-got-nan/133142/4 "2021-09-30T10:33:26Z")

</div>

Thanks the the update! No, `double` shouldn’t be a problem.  
So it seems that the `fc1` layer already creates these large values. Could you check the `weight` and `bias` of this layer and post it here, please?

---

<div class="post-metadata">

**Author:** ![sudri](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sudri/32/66770_2.png) [@sudri](https://discuss.pytorch.org/u/sudri)\
**Post date:** [September 30, 2021, 11:33am UTC](https://discuss.pytorch.org/t/after-torch-load-model-and-predict-then-got-nan/133142/5 "2021-09-30T11:33:52Z")

</div>

Thanks for your suggestion, I checked the weight and bias of fc1. As shown below, none of them are 0, but their range of values is indeed very large. For this situation, I have some confusion and don’t know what to do next.

 ![Snipaste_2021-09-30_20-03-26](https://discuss.pytorch.org/uploads/default/original/3X/d/8/d85b327b5776360432f47d30efaa28f9e18c1575.jpeg)

I save and load the model by using the following code.

```cpp
Net model = Net(in_feature, out_feature);
// some train and test code
torch::save(model, model_path); // model_path is like "xxx.pt"

// initialize new model
Net new_model = Net(in_feature, out_feature);
torch::load(new_model, model_path);
new_model->to(device, torch::kDouble);
// and predict...

```

Thank you very much and look forward to your reply!

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [September 30, 2021, 5:28pm UTC](https://discuss.pytorch.org/t/after-torch-load-model-and-predict-then-got-nan/133142/6 "2021-09-30T17:28:49Z")

</div>

So apparently something goes wrong during the saving or loading.  
Is it possible to load the model and/or parameters in Python? If so, are you seeing different parameters?

Could you also just run this check in a single code snippet:

- create the model
- check parameter stats
- save the model
- load the model right afterwards
- check parameters again

---

<div class="post-metadata">

**Author:** ![sudri](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sudri/32/66770_2.png) [@sudri](https://discuss.pytorch.org/u/sudri)\
**Post date:** [October 2, 2021, 2:10pm UTC](https://discuss.pytorch.org/t/after-torch-load-model-and-predict-then-got-nan/133142/7 "2021-10-02T14:10:21Z")

</div>

After trying again and again, I finally found the problem. Let me briefly describe the prerequisites and solutions.

Prerequisites

1. The model parameter type is Double
2. The input parameter is also of type Double
3. Therefore, the model saved by torch::save is also of Double type

Solutions  
After LibTorch saves the model, before loading the model, the network needs to be initialized first. **If you want to read an offline network file of type Double, you must first initialize the network to type Double, then load the model from file, which is very, very important.** The sample code is shown below.

```cpp
// Initialize the network
Net model = Net(in_feature, out_feature);
model->to(device, torch::kDouble);
// some code to train and validate and more...
torch::save(model, "test.pt");

// Load the model
Net new_model = Net(in_feature, out_feature);
new_model->to(device, torch::kDouble); // very very important before load
torch::load(new_model, "test.pt");

```

**In summary, whether it is to save the model or load the model, the data type must be consistent throughout the process.**

Thank you very much for all your suggestions and help!  
Have a good day!

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [October 2, 2021, 11:32pm UTC](https://discuss.pytorch.org/t/after-torch-load-model-and-predict-then-got-nan/133142/8 "2021-10-02T23:32:13Z")

</div>

Thanks a lot for the detailed explanation!  
I would consider this a nasty bug, as it’s causing silently wrong answers so thanks again for the debugging!  
I’ll create an issue to track and fix this.