All Posts
- Retentive Networks and RWKVSeptember 16, 2023
A short, hand-wavy explainer for the mathematical intuition behind faster attention mechanisms.
- Miscellaneous Azure NotesSeptember 14, 2023
Miscellaneous notes about various Azure-related things.
- Miscellaneous AWS NotesSeptember 13, 2023
Miscellaneous notes about various AWS-related things.
- Streaming ConvolutionsAugust 24, 2023
Working out the math for streaming convolutions.
- Diffusion verses Flow MatchingJuly 19, 2023
An accessible introduction to diffusion and flow matching models.
- Fast Attention ImplementationsJune 29, 2023
A reference collection of fast attention implementations.
- Starting a StartupJune 27, 2023
I left FAIR to start a startup a few weeks ago, and figured I should describe what we're actually doing.
- RWKV Language Model MathJune 16, 2023
Math behind the RWKV model, plus a discussion of numerical stability.
- Types of Electrical Connectors for BatteriesMarch 6, 2023
A collection of the different types of electrical connectors that might be relevant for robotics.
- Types of GearsMarch 6, 2023
A reference for the different types of gears that might be relevant for robotics applications.