LLM engineering
33 posts
The best terminal assistant has no AI
TERMy runs without an LLM to show where deterministic retrieval beats AI-assisted terminal work: bounded inputs, known answers, instant offline lookups.
Three model calls, finished before you blink
Qwen 3.8 27B at 1500 tokens/s on Cerebras is an orchestration budget, not a quality upgrade - spend it on validation and decomposition, not more agents.
The 150x model tax
Cost, latency, and quality trade against each other in LLM inference. How to route work, use serving-layer levers, and architect systems on the efficient frontier.
A smarter model would have leaked it too.
GitHub's AI agent leaked private repos not from a bug but a design failure. How two-plane architecture, scoped tokens, and deterministic validation stop it.
A meditation app shipped a switch statement as AI
Whether a product 'really uses AI' is unanswerable and beside the point. What predicts reliability is system design: validated inputs, constrained outputs, fallbacks.
Keep the hard part
AI doesn't erode your problem-solving skills-offloading the reasoning does. Any intelligence atrophies without use; the fix is design, not avoidance.
2026's AI failures aren't model failures
AI deployments fail at orchestration, not capability. Building validated pipelines around the model - not completing the task - is the real job.
The demo passed. Two weeks later, the queue filled.
Prompt engineering treats AI as magic. Reliable LLM systems come from validation, retries, fallbacks, and monitoring - not better wording.
Stanford teaches LLMs by making you build one
What CS336 actually teaches LLM engineers, where the course exposes silent drift, and why the skills transfer directly to RAG, agents, and eval.
Hy3 is quietly winning production
Hy3 is topping OpenRouter rankings with no public lineage. NovaMind breaks down what its dominance means for pipelines, automation, and team design.
Liquid AI's 8B-A1B drop rewrites inference math
Liquid AI's 8B-A1B MoE trained on 38T tokens shifts LLM inference economics. What it means for engineering pipelines and workforce planning.
One billion fire, eight billion sit in memory
Liquid AI's 8B-A1B MoE frees compute and latency, not memory. How to match sparse-model architecture to the real constraint in your deployment.