How to build Hot Module Replacement in Python

Engineers need fast feedback to build software quickly. Almost everyone uses a hot-reloading development server, but for large Python projects even this approach eventually becomes painfully slow.

Fundamentally, hot-reloading can be slow because the entire Python server process is killed and restarted from scratch - even when only a single module has been changed. Django’s runserver, uvicorn, and gunicorn are all popular options which use this model for hot-reloading. For projects that can’t tolerate this kind of delay (which can reach a minute in extreme cases), building a dependency map can enable hot module replacement for near-instantaneous feedback.

Dependency Maps

A dependency map is a simple yet powerful structure, typically implemented as a flat object. Each key is a filepath, and it is mapped to the set of files which directly import from it (its dependents).

By repeatedly performing lookups in this map, you can obtain a closure, which is the set of all modules potentially affected by changes. This enables precise identification of what needs to be reloaded, significantly streamlining incremental updates.

The dependency map we will use is provided as a library by Tach. It is generated statically up-front, and is incrementally updated as files change. For other use cases, the map can be created statically by the tach map command-line tool.

Understanding Python Imports

When Python modules are imported, they're added to sys.modules, a global dictionary which also acts as a cache. Python checks this cache on subsequent import attempts, avoiding redundant file reads and compilations.

This caching mechanism also handles complex scenarios such as import cycles. Python allows modules to be partially initialized in sys.modules, enabling imports within cycles to resolve correctly despite the interdependencies.

Hot Module Replacement (HMR)

Implementing HMR in Python means replacing modules in sys.modules in the correct order, and avoiding a full restart. If moduleA imports moduleB and both modules have changed, when we reload moduleA, we need to make sure it does not get the old, cached version of moduleB.

Several well-known HMR implementations exist for frontend development, most notably in the React ecosystem. These implementations typically also have sophisticated runtime support on the client side, and specific accept and dispose handlers for modules to manage state during reloads.

In this example, we illustrate a simplified form of HMR, while acknowledging that robust state management would require a more sophisticated strategy.

Example: HMR for a WSGI Application

Using a dependency map, we can build a relatively simple HMR harness for WSGI applications. The final product looks like this:

Consider a large WSGI application composed of many interconnected Python modules. Normally, every change triggers a full restart, often taking several seconds. We will use an example directory, containing sleep statements and many inter-module dependencies to simulate this.

Using traditional auto-reloading:

‍But with HMR, the situation dramatically improves. By reading the closure from our dependency map, and pairing this with recorded import order, we can reload only what's necessary.

On this example, we've gone from 4.8s to essentially instant (~6ms) by replacing modules in-memory rather than doing a full process restart. The implementation of ReloadableWSGI fits comfortably in a single Python file, with a couple key snippets shown below.

‍

The last novel feature worth mentioning is cycle handling. This example uses a 'best-effort' approach by recording the initial order in which modules are loaded when the server starts up, and then using this to sort the closure generated from the dependency map. This allows reloading modules which form cycles by replicating an order which has already been shown to work. Further development would involve updating this initial order as modules are reloaded.

All the code for this example is available here, and the example server can be run with a single command!

Conclusion

Dependency mapping is a powerful technique for speeding up developer workflows, particularly in large Python codebases. Tach provides a fast, Rust-based implementation out-of-the-box, making it much easier to build systems like HMR or test selection.