It's funny to me that we need to research how to make LLMs work with smaller data structures, because typically you start with the smallest data structures and then figure out how making them larger can improve the software. Lo and behold, less bits can make faster more efficient LLMs.
https://www.youtube.com/watch?v=7hMoz9q4zv0