# Loading weights without \`map\_location\` with \`strict=True\` doesnt throw error **URL:** \ **Category:** distributed\ **Created:** [October 24, 2026, 11:58pm UTC](https://discuss.pytorch.org/t/loading-weights-without-map-location-with-strict-true-doesnt-throw-error/225632 "2026-10-24T23:58:46Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Priyank\_Pathak](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/priyank_pathak/32/19974_2.png) [@Priyank\_Pathak](https://discuss.pytorch.org/u/Priyank_Pathak)\ **Post date:** [October 24, 2026, 11:58pm UTC](https://discuss.pytorch.org/t/loading-weights-without-map-location-with-strict-true-doesnt-throw-error/225632/1 "2026-10-24T23:58:46Z")
Suppose I’m running code with torch.nn.distributed on two GPUs. If the pre-trained weights were saved using `cuda:0`, then loading the same weights on different GPUs fails and doesn’t even throw error despite having `strict = True` ```auto checkpoint = torch.load(args.pretrain) checkpoint['model_state_dict']['ln_final.bias'].device # device(type='cuda', index=0) msg = model.load_state_dict(checkpoint['model_state_dict'], strict=True) # always return ``` If the above process is run via torch.nn.distributed on two GPUs, **cuda:0** and **cuda:1** , the models weights will still show `` for **both GPUs** but **no weights will be loaded for cuda:1** unless `checkpoint = torch.load(args.pretrain, map_location='cpu')` Took some time time to figure this out. Is this an expected behavior? ---
**Author:** ![aicost\_tools](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/b5e925/32.png) [@aicost\_tools](https://discuss.pytorch.org/u/aicost_tools)\ **Post date:** [October 25, 2026, 10:15am UTC](https://discuss.pytorch.org/t/loading-weights-without-map-location-with-strict-true-doesnt-throw-error/225632/2 "2026-10-25T10:15:01Z")
`strict=True` only checks that the key names line up, it doesn’t look at devices at all, so `` is expected here. `load_state_dict` copies the values into your existing parameters with `copy_`, which works across devices, so I’d double-check how you’re seeing that cuda:1 got no weights. What does bite you without `map_location` is that every rank deserializes the whole checkpoint onto cuda:0 first, which is why the DDP tutorial loads with `map_location` mapped to the local rank. I work on aicost.tools and mostly stare at the GPU bill side of things, but `map_location='cpu'` (or `f'cuda:{rank}'`) is the safe default either way. ---
**Author:** ![Priyank\_Pathak](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/priyank_pathak/32/19974_2.png) [@Priyank\_Pathak](https://discuss.pytorch.org/u/Priyank_Pathak)\ **Post date:** [October 25, 2026, 3:49pm UTC](https://discuss.pytorch.org/t/loading-weights-without-map-location-with-strict-true-doesnt-throw-error/225632/3 "2026-10-25T15:49:32Z")
> [@aicost\_tools](#): > > seeing that cuda:1 got no weights. I checked the values, values got updated on `cuda:0` but didnt on `cuda:1` Thank you, your explanation does make sense. I guess, I just assumed, `load_state_dict` to throw error on device incompatibility rather displaying `` without loading anything. ---
**Author:** ![DavidBPw](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/davidbpw/32/31128_2.png) [@DavidBPw](https://discuss.pytorch.org/u/DavidBPw)\ **Post date:** [October 26, 2026, 8:52am UTC](https://discuss.pytorch.org/t/loading-weights-without-map-location-with-strict-true-doesnt-throw-error/225632/4 "2026-10-26T08:52:14Z")
I also just encountered this issue. Trained models on GPU zero and when calculating additional metrics using the trained models only the runs on cuda:0 would perform as expected while the runs on cuda:1 would make random predictions. I only get expected results when explicitly mapping the weights to cuda:1