Ben Bolte

nlp

• Retentive Networks and RWKV September 16, 2023
  A short, hand-wavy explainer for the mathematical intuition behind faster attention mechanisms.
• Fast Attention Implementations June 29, 2023
  A reference collection of fast attention implementations.
• RWKV Language Model Math June 16, 2023
  In-depth explanation of the math behind the RWKV model, with PyTorch implementations, plus a discussion of numerical stability.
• Using Gensim Word2Vec Embeddings in Keras August 2, 2016
  A short post and script regarding using Gensim Word2Vec embeddings in Keras, with example code.
• Question Answering using Keras April 27, 2016
  An in-depth introduction to using Keras for language modeling; word embedding, recurrent and convolutional neural networks, attentional RNNs, and similarity metrics for vector embeddings.