# Multi-gpu of model

**URL:** <https://discuss.pytorch.org/t/multi-gpu-of-model/33946>\
**Category:** nlp\
**Created:** [January 6, 2019, 3:22pm UTC](https://discuss.pytorch.org/t/multi-gpu-of-model/33946 "2019-01-06T15:22:45Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jiang\_ix](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/9e8a1a/32.png) [@jiang\_ix](https://discuss.pytorch.org/u/jiang_ix)\
**Post date:** [January 6, 2019, 3:22pm UTC](https://discuss.pytorch.org/t/multi-gpu-of-model/33946/1 "2019-01-06T15:22:45Z")

</div>

I’m implementing the seq2seq model with pytorch 0.4.1. For encoder, I want to save its parameters and tensors into cuda:0. For decoder, I want to save all its parameters and tensors into cuda:1. So can i do it? use encoder.to(‘cuda:0’), decoder.to(‘cuda:1’)？But I find the input tensor of encoder are not in gpu 0. Can I use a command to transfer the all parameters and tensors of encoder into gpu 0 ?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [January 6, 2019, 4:19pm UTC](https://discuss.pytorch.org/t/multi-gpu-of-model/33946/2 "2019-01-06T16:19:21Z")

</div>

The code snippets you’ve posted should already create the `encoder` on GPU0 and the `decoder` on GPU1.  
You would just have to make sure the input tensors are on the right device.  
Have a look at [this small example for model sharding](https://discuss.pytorch.org/t/split-single-model-in-multiple-gpus/13239/2?u=ptrblck).

---

<div class="post-metadata">

**Author:** ![jiang\_ix](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/9e8a1a/32.png) [@jiang\_ix](https://discuss.pytorch.org/u/jiang_ix)\
**Post date:** [January 7, 2019, 6:06am UTC](https://discuss.pytorch.org/t/multi-gpu-of-model/33946/3 "2019-01-07T06:06:26Z")

</div>

Yeah, I will see it and Thanks a lot