RC RANDOM CHAOS

EmbeddingGemma 2 repurposed for speech-to-text

· via Hacker News

Original source

We turned EmbeddingGemma 2 into a speech-to-text model

Hacker News →

Researchers transformed Google’s EmbeddingGemma 2, a multimodal embedding model, into a speech-to-text system. The model processes an hour of audio in about 13 seconds on a base MacBook Air, with all processing done on-device. The conversion involved keeping the audio encoder unchanged while adding a small reader to map audio slices to text space, selecting the closest English word. A secondary model adds punctuation and capitalization. The system was trained for under two hours using openly licensed English speech data, with the base model never specifically trained for speech-to-text tasks. Performance is slightly lower than leading speech models but shows promise for future improvements.

Read the full article

Continue reading at Hacker News →

This is an AI-generated summary. Read the original for the full story.