Huawei releases AI reasoning innovation technology UCM

On August 12th, Huawei officially released its AI reasoning innovation technology UCM (Reasoning Memory Data Manager). It is reported that as an inference acceleration kit centered on KV Cache, UCM integrates multiple types of cache acceleration algorithm tools to hierarchically manage KV Cache memory data generated during the inference process, expand the inference context window, achieve high throughput and low latency inference experience, and reduce the inference cost per token. This technology has taken the lead in piloting the application of intelligent financial AI reasoning acceleration in three major business scenarios of China UnionPay: "Customer Voice", "Marketing Planning", and "Office Assistant", and has achieved results. (People's Financial News)