Thirty years of weaponizing fork-exec

Li Chen’s spawn-template patch set gets about a 2% speedup on workloads that launch the same program over and over, and it will not be merged. The review it drew is more interesting than the code, because it turned into a plan to finally give Linux a real posix_spawn().

Start with the cost the patch is chasing. fork() creates a child as a copy of the parent, including its memory, and exec() then replaces that process image with a new program. On Linux the calls are clone() and execve(), but the shape is the one Unix has had since the beginning. The expensive case is the common one: a fork() immediately followed by an exec() copies the parent’s address space and then discards all of it. vfork() and other optimizations have chipped at that cost over the years without removing it.

Chen’s approach targets programs that run one executable repeatedly, such as a tool shelling out to Git again and again to read a repository. Instead of paying full setup on every launch, the program builds a template once and reuses it. spawn_template_create() takes an executable, specified by file descriptor or absolute path but not both, opens it, and caches the information the kernel would otherwise recompute each time. It returns a file descriptor for the template.

Per-invocation details go in a separate structure passed to spawn_template_spawn(): argv, envp, and an actions array describing changes to file descriptors and signal handling. Each action is a small struct with a type and arguments. To close file descriptor four in the child, you set the type to SPAWN_TEMPLATE_ACTION_CLOSE and fd to four. Other action types duplicate descriptors, open files, change the working directory, and adjust signal handling. Internally the call still runs something close to the normal fork()/exec() path, and Chen notes the usual checks for executing a file stay in place. The win comes entirely from the cached template.

Mateusz Guzik gave the most detailed review and said the entire fork + exec idiom “is terrible and needs to be retired”. His objection to the patch was that it optimizes the exec side and leaves fork() alone, and fork() is where most of the cost sits. Copying the current process is the wrong starting point; creating a pristine process, he argued, is the way to go.

Christian Brauner was warmer to the idea of a builder API for exec but wanted it built on pidfd rather than a new template object. His sketch: add an option to pidfd_open() that creates an empty process, then configure it through a series of pidfd_config() calls that set its environment, the image to execute, and the rest, the way fsconfig() configures a filesystem. The objective he cares about is letting someone implement posix_spawn() in user space on top of that interface. posix_spawn() is the standard replacement for the fork/exec pattern, and today’s implementations fake it by calling fork() and exec() under the covers. A native path would let it stop pretending.

Chen agreed the pidfd-based API was the better design and said future work would head that way. So there will be no spawn templates in the kernel. What may come out of it is the piece Linux has been missing: a way to build a process without first cloning the one you already have.