# How are pretrained weights loaded in nn.LazyLinear layers

**URL:** <https://discuss.pytorch.org/t/how-are-pretrained-weights-loaded-in-nn-lazylinear-layers/222459>\
**Category:** complex\
**Created:** [August 18, 2025, 3:09pm UTC](https://discuss.pytorch.org/t/how-are-pretrained-weights-loaded-in-nn-lazylinear-layers/222459 "2025-08-18T15:09:09Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![gk-mixar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/gk-mixar/32/77994_2.png) [@gk-mixar](https://discuss.pytorch.org/u/gk-mixar)\
**Post date:** [August 18, 2025, 3:09pm UTC](https://discuss.pytorch.org/t/how-are-pretrained-weights-loaded-in-nn-lazylinear-layers/222459/1 "2025-08-18T15:09:09Z")

</div>

Hello in a project of mine I am trying to build a linear layer for which the input dim may change depending upon the dataset used.  
So I am using a nn.lazylinear.  
However after obtaining the stat\_dict after training whenever I try to load the model using the state\_dict I get issues with the keys :  
untimeError: Error(s) in loading state\_dict for SeamModule:

```
    Unexpected key(s) in state_dict: "model.enc_to_dec.weight", "model.enc_to_dec.bias". 

```

How do i properly load the trained Lazylinear weights ?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [August 18, 2025, 6:59pm UTC](https://discuss.pytorch.org/t/how-are-pretrained-weights-loaded-in-nn-lazylinear-layers/222459/2 "2025-08-18T18:59:31Z")

</div>

I’m unable to reproduce the issue by trying to load a `state_dict` into an uninitialized `Lazy*` model. Could you post a minimal code snippet to reproduce the issue, please?

Also, as a workaround you might need to perform the dry-run before loading the `state_dict` but it’s unclear to my if that’s indeed necessary as I don’t see any errors.

---

<div class="post-metadata">

**Author:** ![gk-mixar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/gk-mixar/32/77994_2.png) [@gk-mixar](https://discuss.pytorch.org/u/gk-mixar)\
**Post date:** [August 19, 2025, 10:06am UTC](https://discuss.pytorch.org/t/how-are-pretrained-weights-loaded-in-nn-lazylinear-layers/222459/3 "2025-08-19T10:06:59Z")

</div>

Yes, i tried training a model again from scratch using nn.LazyLinear. It seems to work fine when I later load checkpoints.  
I guess the error was coming up from a different part of the code.  
Thanks for the prompt reply though !