RC RANDOM CHAOS

ggml 1.0.0 Removes Deprecated APIs, Adds FP8 Support

· via Hacker News

Original source

Ggml v1.0.0

Hacker News →

ggml 1.0.0 removes deprecated public API elements and legacy DIAG_MASK operations, updating the RPC protocol to reject incompatible peers. The release adds configurable YaRN truncation and FP8 support, along with RPC protocol v9 for faster loading and graceful failure handling when endpoints are unreachable or handshakes fail.

Performance improvements extend across backends. CUDA and Metal gain few-row quantized matmuls and MoE fusion. Vulkan introduces a prefill FlashAttention path, SYCL gets a device-side MoE scheduler, and OpenCL adds decomposed Adreno FlashAttention. WebGPU and Hexagon receive new kernels and expanded operation coverage. Scheduler optimizations avoid uploading unused inputs and prefetch only data computed by earlier splits, while MoE caching can compute cache misses on the CPU and copy results back asynchronously.

Read the full article

Continue reading at Hacker News →

This is an AI-generated summary. Read the original for the full story.