The Future of AI Might Be Tiny: Small Models Win Where Networks Fail

When a counterfeit-drug scanner failed during a 2019 Cape Town demo because its AI model lived on a server 14,000 km away, founder Adebayo Alonge had his engineers shrink the model to run entirely on an Android phone within two hours. That improvised fix became the blueprint for ‘small AI’ — specialized models with at most a few billion parameters that run locally on phones, Raspberry Pis, or cheap microcontrollers using only a few watts of power, no data center or reliable connectivity required. The stakes are concrete: a World Bank report found only 0.7 percent of internet users in the poorest countries have touched ChatGPT, versus a quarter in wealthy nations, so for much of the world the only viable AI is the kind that fits on a battery-powered edge device.

These models aren’t a different technology from frontier LLMs — they’re derived from them. Engineers ‘prune’ large models down to the parameters relevant to one task, ‘distill’ them by training a small model to mimic a big one, or reduce numerical precision so a model can run on 8-bit hardware; classification tasks are often trained on the small device from scratch. Real deployments already include disease detection on cashew-farming drones in India, ant and mosquito monitoring in Uruguay and elsewhere, and Arduino-based ECGs in parts of Brazil. Open-weight releases like Google’s Gemma and Alibaba’s Qwen make it easy to retrain a compact model on narrow, domain-specific data.

Two trends are accelerating this shift: hardware keeps getting more capable per watt — more than a third of smartphones shipped in 2025 could run generative AI, projected to cross 50 percent next year — and language models keep shrinking. Advocates frame small AI less as a stopgap for poor regions and more as the sustainable path forward, arguing that giant centralized models may become unaffordable without subsidies. Alonge’s bet is that AI’s future is ‘millions of small, precise models deployed at the edge,’ each solving one specific problem in one specific context, rather than a single colossal model in a distant data center.