Interpreter Forking Research¶

What you'll learn¶

• What forking the interpreter means and why people do it
• The CPython source structure you'd touch
• How new opcodes are added
• Famous forks and experiments (GIL removal, subinterpreters)
• Why this is a last resort

What "forking the interpreter" means¶

A core question explored in Interpreter Forking: What "forking the interpreter" means.

CPython — the reference Python implementation — is a C program that reads your bytecode and executes it. Forking it means taking that C source, changing how the language works at its core, and building your own python binary. This is the deepest level of language hacking: below your code, below the bytecode, into the machine that runs it.

People fork CPython to experiment with language features, performance ideas, or research questions that can't be expressed from within Python. Most forks are experiments; a few graduate into CPython itself.

   your .py  →  bytecode  →  [ CPython eval loop (C) ]  →  result
                              ▲
                              └── forking modifies THIS layer

Seeing the layer you'd modify¶

Where interpreter behavior lives, so you know what forking touches.

You can inspect the bytecode the eval loop executes with dis — this is the interface between "your code" and "the interpreter's C". Runnable:

import dis

def add(a, b):
    return a + b

dis.dis(add)

Output (abridged, version-dependent):

  2   RESUME               0
  3   LOAD_FAST            a
      LOAD_FAST            b
      BINARY_OP            0 (+)
      RETURN_VALUE

Each line is an opcode the CPython eval loop handles in a giant C switch (historically in ceval.c). Forking the interpreter often means adding or changing opcodes in that loop. For example, adding a hypothetical BINARY_POWER_MOD opcode would mean: define it, emit it from the compiler, and handle it in the eval loop's C code.

The CPython source you'd touch¶

The files that implement the behavior you'd change.

A fork typically involves these parts of the CPython tree:

Adding a feature usually ripples through several of these: new syntax needs grammar + AST + compiler + eval loop changes, then a full rebuild.

Famous forks and experiments¶

PyPy, Stackless, and research interpreters.

Interpreter forking has a rich history — and some experiments became hugely important:

• No-GIL / free-threaded CPython (PEP 703) — a long-running fork removing the Global Interpreter Lock, now being merged into CPython as an experimental build. Possibly the most consequential fork in Python's history.
• Subinterpreters (PEP 554/734) — multiple isolated interpreters in one process, developed partly as experimental branches before landing in the stdlib.
• Cinder (Meta) — a performance-oriented CPython fork with a JIT and other optimizations, running Instagram.
• Pyston, Pyjion — JIT-adding forks/experiments.
• PyPy — not a CPython fork but an entirely separate implementation with a tracing JIT (see the Performance section).

Many CPython performance wins (the "Faster CPython" project, specializing adaptive interpreter) were prototyped as experimental interpreter modifications before shipping.

Why it's a last resort¶

A core question explored in Interpreter Forking: Why it's a last resort.

Practice exercises¶

1. Use dis on several functions (a loop, a comprehension, a with block) and map each to the opcodes it produces.
2. Read the CPython source layout online and note which files you'd change to add a new built-in opcode.
3. Explain, in terms of the eval loop, why removing the GIL is so invasive (touches object refcounting everywhere).
4. Research one CPython fork (Cinder or the no-GIL build) and summarize what it changed and why.
5. Argue why an import-time AST transform is preferable to a fork for adding a custom behavior to your own project.