Apple Trains Neural Networks to Speed Up RAG Retrieval
Apple researchers have developed a new approach to speed up maximum inner product search (MIPS), the core technology behind retrieval-augmented generation systems. Current methods treat all queries the same way and don’t account for the fact that search queries—typically short user questions—differ substantially from indexed content, which consists of long, curated documents. The researchers train small neural networks that learn efficient routing patterns for a specific database and query distribution, trading offline computational cost for faster and more accurate retrieval during deployment.
The approach leverages a mathematical insight: MIPS operates on support functions from convex analysis, which neural networks can approximate efficiently. Researchers developed two models—SupportNet, which learns cluster support functions to improve routing decisions, and KeyNet, which directly predicts the best-matching key vector. Both are trained on supervised examples drawn from actual query distributions, ensuring the learned models optimize for real-world usage patterns rather than generic cases.
The work, presented at ICML 2026, demonstrates that learning query-aware retrieval methods achieves better performance with reduced computational overhead compared to traditional clustering and quantization approaches. This has implications for scaling RAG systems and other retrieval-dependent applications to handle millions of keys at lower cost.
Read the full article
Continue reading at Hacker News →This is an AI-generated summary. Read the original for the full story.