FSDP: can not use conv2d?

Hi! I meet this and meet a bug called:

Could anyone kindly help me? Thank you!!!

It seems PatchEmbeddingSTD is causing the error as apparently its .weight attribute is missing while a module expects it. Could you check where this module is coming from and how its self.weight attribute is initialized and used?

1 Like

I have printed all the parameters out:
For FSDP, it shows only(!) this:

For model without FSDP, it shows this:

And much more…

I am thinking maybe FSDP itself has some problem? Maybe it does not support embedding?

Thank you!!!

@Yanli_Zhao could you please take a look?

1 Like