Take a more closer look at the formulation of the problem involving logSoftmax(A(sum q_w) + b). Intuitively, it’s one way of gathering the contributions of the surrounding words. You may find this useful. Also spoiler alert for the solution I gave here.