# Question about implementation of complex Layers

**URL:** <https://discuss.pytorch.org/t/question-about-implementation-of-complex-layers/209467>\
**Category:** complex\
**Created:** [September 12, 2024, 11:50am UTC](https://discuss.pytorch.org/t/question-about-implementation-of-complex-layers/209467 "2024-09-12T11:50:51Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Timwolf01](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/timwolf01/32/72700_2.png) [@Timwolf01](https://discuss.pytorch.org/u/Timwolf01)\
**Post date:** [September 12, 2024, 11:50am UTC](https://discuss.pytorch.org/t/question-about-implementation-of-complex-layers/209467/1 "2024-09-12T11:50:51Z")

</div>

I have a question about the implementation of complex Layers, maybe its a duplicate, but havent found a similar.

In linear Layer the forward Implementation is x @ w.T + b, made with torch.addmm.  
With complex data, weights and biases it could be formed to:  
(x.real @ w.real.T + b.real) - (x.imag @ w.imag.T + b.imag) + 1j \* (x.real @ w.imag.T + b.imag + x.imag @ w.real.T + b.real)

But with nn.Linear(input\_shape, output\_shape, dtype=torch.complex64) it is:  
(x.real @ w.real.T + b.real) - (x.imag @ w.imag.T + 0) + 1j \* (x.real @ w.imag.T + b.imag + x.imag @ w.real.T + 0)

Why is the bias treated as 0 in the two subterms?  
Is this intendet and whats the purpose or benefit of it?

In most self-made complex pytorch code found online the upper version is used.

In other layers like nn.Conv2d it is the same thing

---

<div class="post-metadata">

**Author:** ![KFrank](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/32.png) [@KFrank](https://discuss.pytorch.org/u/KFrank)\
**Post date:** [September 15, 2024, 2:51am UTC](https://discuss.pytorch.org/t/question-about-implementation-of-complex-layers/209467/2 "2024-09-15T02:51:02Z")

</div>

Hi Tim!

> [@Timwolf01](#):
>
> With complex data, weights and biases it could be formed to:  
> (x.real @ w.real.T + b.real) - (x.imag @ w.imag.T + b.imag) + 1j \* (x.real @ w.imag.T + b.imag + x.imag @ w.real.T + b.real)

This just isn’t correct. Consider the case where everything, in  
particular the bias, `b`, is purely real. Your result, as written, will  
contain an incorrect imaginary term, `1j * b.real`.

> [@](#):
>
> But with nn.Linear(input\_shape, output\_shape, dtype=torch.complex64) it is:  
> (x.real @ w.real.T + b.real) - (x.imag @ w.imag.T + 0) + 1j \* (x.real @ w.imag.T + b.imag + x.imag @ w.real.T + 0)
> 
> Why is the bias treated as 0 in the two subterms?

The bias, `b`, is a purely additive term, so the real component  
of the result contains just `b.real` and the imaginary component  
of the result contains just `b.imag`. I don’t understand why you  
would expect `b.real` and `b.imag` to both show up in both the  
real and imaginary components of the result.

> [@](#):
>
> In most self-made complex pytorch code found online the upper version is used.

I find this hard to believe. But if it is true, then most self-made  
code found online is wrong.

Best.

K. Frank

---

<div class="post-metadata">

**Author:** ![Timwolf01](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/timwolf01/32/72700_2.png) [@Timwolf01](https://discuss.pytorch.org/u/Timwolf01)\
**Post date:** [September 16, 2024, 12:07pm UTC](https://discuss.pytorch.org/t/question-about-implementation-of-complex-layers/209467/3 "2024-09-16T12:07:33Z")

</div>

Hi K. Frank,

thanks for your clarification. I see my error now. I must have found the exact only 2 or 3 githubs where its wrong and made an error in my thoughts. Of course the bias is just added after the mutliplication.

For interested (and my rehabilitation 😅) the top google results for complex pytorch githubs make it like this:

- [repo 1](https://github.com/soumickmj/pytorch-complex/blob/f5d0c3511a3f3c7cb7f138ee7f42a699d3f8209a/torchcomplex/nn/functional.py#L73)
- [repo 2](https://github.com/wavefrontshaping/complexPyTorch/blob/636e1cc545be662f41e10808ff86f6daa2d18ce6/complexPyTorch/complexLayers.py#L34)

I can only post two links, but these are examples of the wrong made ones. I also found repos where its made correctly.

Bests  
Tim