Question about exercise sheet 3

Question about exercise sheet 3

بواسطة - مستخدم محذوف
عدد الردود: 2


Hello all,

I have a question about exercise sheet 3 | Semantic Retrieval - Representation Learning | Task 3:

Calculate the output of the last layer (softmax layer) for the current sentence in which Sam is the center word.

How do I calculate "logits" and "y"?


Thanks a lot for your answers :)

رداً على مستخدم محذوف

Re: Question about exercise sheet 3

بواسطة - Daniel Weller

Hi Alexander,

logits:
u1 = 0,0625*0,64 + -0,5875*1,4 + -0,5225*-0,83 + 0,2225*-0,23 = -0.4

softmax:
y1 = eu1 / eu1 + eu2 + eu3 + eu4 + eu5 + eu6 +eu7 + eu8
y1 = e-0,4 / e-0,4 + e1,6070 + ... + e0,8874 = 0,04

I found this page very helpful for understanding (point 3.2 Forward Propagation): 
https://thinkinfi.com/continuous-bag-of-words-cbow-single-word-model-how-it-works/?utm_content=cmp-true