continual learning
2 posts
Article
Mini-AGI trains a model with no frozen base
mini-AGI trains as it reads, through the same path it answers on, so the weights keep moving and there is no single frozen version of the model to sign.
Article
Rewriting weights live on 8GB of VRAM
mini-AGI trains a byte-level MoE model from scratch on an 8GB GPU, keeping weights on disk and paging experts in, learning as it reads.