TERMy ditches LLMs for a rule-based NLU pipeline in ~1000 lines

The creator of the PJON networking protocol set out to build a natural-language terminal assistant so he could stop paying a monthly LLM bill for trivial chores like activating a virtualenv. His first attempts trained transformers from scratch on a decade-old gaming PC (a 4GB GTX 1050 Ti, i7-4790K, 16GB RAM), experimenting with NanoGPT-style models, flash attention, and even Mamba. The output was incoherent and prone to repeating itself, and a usable run would have needed a month of nonstop training. Switching to Ollama with open-weight models constrained to emit only shell commands proved too slow and unreliable on 4GB of VRAM.

The working design abandons machine learning entirely: no embeddings, no ML, no LLMs. It centers on NDF, a JSON dataset format where each object is a self-contained ‘atom of knowledge’ bundling example phrasings, the tool call to run, human-readable thinking traces, and a permission field. Adding capabilities is as simple as dropping a new JSON file into the dataset directory. Variable requests such as ‘create file test.txt’ are handled by template intents that match synonym-tagged vocabulary (create/make/generate/forge) and extract typed entities like filenames via regex.

The parser and runtime (FlintParser and FlintNPC) fit in roughly 1000 lines and are mirrored in Python for local use and JavaScript for the browser or Node. Requests flow through a cheapest-path pipeline: strip filler and expletives, run sentiment analysis, then try exact match, template match, and finally a probabilistic match using IDF and bag-of-words. The notable safety angle is that gating any potentially destructive command behind a ‘permission: ask’ step makes the tool inherently safer than a black-box model, and the whole approach is a deliberately minimalist, deterministic, auditable rebuttal to the assumption that every task needs a giant model behind it.