ggml 1.0.0 Removes Deprecated APIs, Adds FP8 Support
ggml 1.0.0 removes deprecated public API elements and legacy DIAG_MASK operations, updating the RPC protocol to reject incompatible peers. The release adds configurable YaRN truncation and FP8 support, along with RPC protocol v9 for faster loading and graceful failure handling when endpoints are unreachable or handshakes fail.
Performance improvements extend across backends. CUDA and Metal gain few-row quantized matmuls and MoE fusion. Vulkan introduces a prefill FlashAttention path, SYCL gets a device-side MoE scheduler, and OpenCL adds decomposed Adreno FlashAttention. WebGPU and Hexagon receive new kernels and expanded operation coverage. Scheduler optimizations avoid uploading unused inputs and prefetch only data computed by earlier splits, while MoE caching can compute cache misses on the CPU and copy results back asynchronously.
Read the full article
Continue reading at Hacker News →This is an AI-generated summary. Read the original for the full story.