Is the Attention mechanism (layer) implemented in the dnn module

asked 2019-03-31 03:34:46 -0600

siddhantsahu gravatar image

Wanted to check if the Attentionlayer operation implemented as a layer in the dnn module? Link to the "Attention is all you Need" paper https://papers.nips.cc/paper/7181-att...

edit retag flag offensive close merge delete

Comments

do you know any computer-vision related model using it ?

(imho it's more used for text, like in the mentioned paper)

berak gravatar imageberak ( 2019-03-31 03:45:03 -0600 )edit

The paper uses this attention mechanism for language task but it seems like it can be used for vision tasks as well.

siddhantsahu gravatar imagesiddhantsahu ( 2019-03-31 05:42:29 -0600 )edit
add a comment