RC RANDOM CHAOS

LLM Wrapper Extends Vision Model Capabilities

· via Hacker News

Original source

A single function Jev-like wrapper for LLMs, including vision models

Hacker News →

A developer created a Jev-like wrapper for LLMs, including vision models, leveraging token probabilities to extract insights. The approach involves writing specific prompts and adding JSON request parameters to a Chat Completions request, enabling the LLM API to return log probabilities for alternative tokens. This method works with vision models, allowing for flexible condition changes described in plain text. The developer’s example captures webcam frames, sends base64 JPEGs, and prints a table assessing various scene attributes. The solution is efficient, with Gemma 4 12B achieving around 1 frame per second on an RTX 3090.

Read the full article

Continue reading at Hacker News →

This is an AI-generated summary. Read the original for the full story.