One day I was reading some papers and realized that I did not know all the shapes of the Transformer architecture by heart.

So I drew it all out in Excalidraw to make sure I really understood.

Here’s a render below – I made this post so that I can always quickly refer back to it if I need to.

Note that the shapes may not be drawn to scale.