# Customized linear operation in neural network

**URL:** <https://discuss.pytorch.org/t/customized-linear-operation-in-neural-network/19996>\
**Category:** autograd\
**Created:** [June 20, 2018, 3:25pm UTC](https://discuss.pytorch.org/t/customized-linear-operation-in-neural-network/19996 "2018-06-20T15:25:57Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Shen](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/4da419/32.png) [@Shen](https://discuss.pytorch.org/u/Shen)\
**Post date:** [June 20, 2018, 3:25pm UTC](https://discuss.pytorch.org/t/customized-linear-operation-in-neural-network/19996/1 "2018-06-20T15:25:58Z")

</div>

Hi everyone, I’m trying to replace the linear operation in neural network with my own version (i.e. the operation **y = wx + b** ). By following [https://pytorch.org/docs/stable/notes/extending.html](https://pytorch.org/docs/stable/notes/extending.html), I created my own Linear class, and changed the **forward** function from _F.linear(input, self.weight, self.bias)_ to _customized\_linear(input, self.weight, self.bias)_ that has been defined somewhere else.

The problem is how to define this customized linear function. I first tried to detach **_input_** and **_self.weight_** tensors, and converted them to numpy. After adding some linear operations on them, convert the result back to tensor form. However, in this way the loss function got updated incorrectly. My guess is the detach operation break all histories (correct me if I’m wrong).

Then I tried to play with the tensor directly without going to numpy. I tried some simple code

```auto
def customized_linear(input, weight):
    input_row = len(input[:,0])
    weight_row = len(weight[:,0])    
    test = torch.zeros(input_row, weight_row)
    
    for i in range(0,input_row):
        for j in range(0,weight_row):
            test[i,j] = torch.matmul(input[i,:], weight[j,:])
    
    test.requires_grad_(True)        
    return test

```

But it always stopped working, and reported **Kernel died**.

How should I implement my customized linear operation? Please help!! Thanks!

---

<div class="post-metadata">

**Author:** ![richard](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/richard/32/1861_2.png) [@richard](https://discuss.pytorch.org/u/richard)\
**Post date:** [June 20, 2018, 4:33pm UTC](https://discuss.pytorch.org/t/customized-linear-operation-in-neural-network/19996/2 "2018-06-20T16:33:00Z")

</div>

By “kernel died”, do you mean a Jupyter notebook died?

---

<div class="post-metadata">

**Author:** ![Shen](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/4da419/32.png) [@Shen](https://discuss.pytorch.org/u/Shen)\
**Post date:** [June 21, 2018, 2:17am UTC](https://discuss.pytorch.org/t/customized-linear-operation-in-neural-network/19996/4 "2018-06-21T02:17:10Z")

</div>

Hi, that was reported by Spyder from Anaconda.

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [June 21, 2018, 8:55am UTC](https://discuss.pytorch.org/t/customized-linear-operation-in-neural-network/19996/5 "2018-06-21T08:55:56Z")

</div>

Could you run it in a terminal and see, what error you’ll get?  
The `IPython` kernel in Spyder often dies and is restarted immediately afterwards, which suppresses the error message.

---

<div class="post-metadata">

**Author:** ![Shen](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/4da419/32.png) [@Shen](https://discuss.pytorch.org/u/Shen)\
**Post date:** [June 21, 2018, 10:46am UTC](https://discuss.pytorch.org/t/customized-linear-operation-in-neural-network/19996/6 "2018-06-21T10:46:37Z")

</div>

I tried to run it in a terminal, the python simply stopped working after computing the 1st epoch. To be more specific, the python stopped working when it tried to compute the loss function for the 2nd epoch.

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [June 21, 2018, 11:15am UTC](https://discuss.pytorch.org/t/customized-linear-operation-in-neural-network/19996/7 "2018-06-21T11:15:51Z")

</div>

Could you try to create the custom function in a `nn.Module` like in the example and run it again?

---

<div class="post-metadata">

**Author:** ![Shen](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/4da419/32.png) [@Shen](https://discuss.pytorch.org/u/Shen)\
**Post date:** [June 21, 2018, 1:19pm UTC](https://discuss.pytorch.org/t/customized-linear-operation-in-neural-network/19996/8 "2018-06-21T13:19:10Z")

</div>

I tried, it ran successfully. I also tried to replace **LinearFunction.apply(input, self.weight, self.bias)** in forward function with **input.matmul(self.weight.t()) + self.bias**, it also worked. I think there is still something wrong with my implementation for **customized\_linear** function, but I cannot find the mistake.

---

<div class="post-metadata">

**Author:** ![Shen](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/4da419/32.png) [@Shen](https://discuss.pytorch.org/u/Shen)\
**Post date:** [June 21, 2018, 4:01pm UTC](https://discuss.pytorch.org/t/customized-linear-operation-in-neural-network/19996/9 "2018-06-21T16:01:29Z")

</div>

Sorry, I just realized it seems I misunderstood your question. When you say “create function in a nn.Module”, do you mean to implement my function directly in this module (e.g. Linear subclass), and call it in **forward** function?