- Published on
A short, hand-wavy explainer for the mathematical intuition behind faster attention mechanisms.
Nlp
- Published on
A reference collection of fast attention implementations.- Published on
In-depth explanation of the math behind the RWKV model, with PyTorch implementations, plus a discussion of numerical stability.- Published on
A short post and script regarding using Gensim Word2Vec embeddings in Keras, with example code.- Published on
An in-depth introduction to using Keras for language modeling; word embedding, recurrent and convolutional neural networks, attentional RNNs, and similarity metrics for vector embeddings.