why does nobody talk about the fact that google's gemini models have a 2M context window but they ALWAYS go into a lobotomized death loop after 20%?
I've never reached 30% without it turning into a s...
I suspected that these context windows are not really fixed in their size but rather degrade in quality so the size is more a marketing term. Yeah, you can feed it 2million tokens for text search but not for meaningful "understanding".