Posts by Inesh Chakrabarti
4-bit KV Caching in LMCache: Offloading Quantized KV Beyond HBM for Context-Heavy Agents on AMD MI355X
- 28 August 2026
As agents carry ever-longer context from turn to turn, the KV cache becomes the resource that runs out first. Two techniques ease that pressure from different angles: KV quantization makes each cached token smaller, while a hierarchical cache like LMCache spills cold KV to CPU DRAM. The two have mostly been developed independently.
Quark Support for HuggingFace Diffusers and SVDQuant
- 06 August 2026
Diffusion models are heavy on memory and compute: a single text-to-image call runs a large transformer or UNet dozens of times. Quantization — storing weights (and sometimes activations) in low precision — is one of the most effective ways to cut both the memory footprint and the latency of these models.
Productionizing TurboQuant on AMD GPUs for KV-Cache-Bound LLM Inference
- 11 June 2026
*The first three authors (Chakrabarti, Limpus, Rana) contributed equally to this work.