Posts Tagged "sparse-attention"

A long-running agent works with 100,000 tokens of context or more, and the GPU memory that holds it is getting scarcer. Our paper, Adaptive DHiSparse, keeps each session's KV cache in host memory and lends it free GPU memory as load allows. On DeepSeek-V4.1-Flash with 500,000-token sessions, the same GPUs produced 1.8 times the output once memory ran out.