-
Context windows, KV-cache, and why long prompts get expensive
Modern large language models can accept remarkably long context windows. It’s tempting to assume that if a model supports 128K, 200K, or even 1M tokens, you should simply keep appending more informati