Nvidia's KV Cache Transform Coding (KVTC) compresses LLM key-value cache by 20x without model changes, cutting GPU memory costs and time-to-first-token by up to 8x for multi-turn AI applications.

MIT researchers developed Attention Matching, a KV cache compaction technique that compresses LLM memory by 50x in seconds — ...

Accelerating memory-dependent AI processes, Penguin's MemoryAI KV cache server increases memory capacity by integrating 3 TB of DDR5 main memory and up to eight 1 TB CXL Add-in Cards (AICs). Penguin ...

As AI workloads extend across nearly every technology sector, systems must move more data, use memory more efficiently, and respond more predictably than traditional design methodologies allow. These ...

How lossless data compression can reduce memory and power requirements. How ZeroPoint’s compression technology differs from the competition. One can never have enough memory, and one way to get more ...

The latest Area-51 desktop from Alienware centers around AMD’s Ryzen 7 9800X3D, an 8-core processor with 104MB of total cache designed for gaming workloads. Paired with an RTX 5080 graphics card, 64GB ...

A new technical paper titled “ARCANE: Adaptive RISC-V Cache Architecture for Near-memory Extensions” was published by researchers at Politecnico di Torino and EPFL. Abstract “Modern data-driven ...

Forbes contributors publish independent expert analyses and insights. This article discusses memory and chip and system design talks at the 2025 AI Infra Summit in Santa Clara, CA by Kove, Pliops and ...

In a computer, the entire memory can be separated into different levels based on access time and capacity. Figure 1 shows different levels in the memory hierarchy. Smaller and faster memories are kept ...

In the eighties, computer processors became faster and faster, while memory access times stagnated and hindered additional performance increases. Something had to be done to speed up memory access and ...