[Python Jupyter Notebook] Hot live reload Python module within Jupyter Notebook

Reload Python module within Jupyter Notebook

All the code in a Jupyter Notebook is the code that runs in the same runtime. When you import the same module multiple times by import, Python’s package management mechanism will automatically ignore the subsequent imports, and always only use the first import.

There are three ways to reload module:

  • (Recommended) Use %autoreload magic;

  • Use importlib, But this has to import the that module again in each Cell;

  • Restart the entire Notebook. But this will cause all the variables in the current runtime to be lost.

Use %autoreload magic

These lines of code can only run normally in Jupyter:

1
2
3
4
5
# first load the autoreload extension, using the %load_ext magic.
%load_ext autoreload

# Reload all modules (except those excluded by %aimport) every time before executing the Python code typed.
%autoreload 2

When any cell runs, all modules imported will be reloaded once. This allows you to use the latest code every time.

See the following Usages to learn more.

Usages

The following magic commands are provided:

1
%autoreload

Reload all modules (except those excluded by %aimport) automatically now.

1
%autoreload 0

Disable automatic reloading.

1
%autoreload 1

Reload all modules imported with %aimport every time before executing the Python code typed.

1
%autoreload 2

Reload all modules (except those excluded by %aimport) every time before executing the Python code typed.

1
%aimport

List modules which are to be automatically imported or not to be imported.

1
%aimport foo

Import module ‘foo’ and mark it to be autoreloaded for %autoreload 1

1
%aimport -foo

Mark module ‘foo’ to not be autoreloaded.

Caveats

Reloading Python modules in a reliable way is in general difficult, and unexpected things may occur. %autoreload tries to work around common pitfalls by replacing function code objects and parts of classes previously in the module with new versions. This makes the following things to work:

  • Functions and classes imported via from xxx import foo are upgraded to new versions when xxx is reloaded.

  • Methods and properties of classes are upgraded on reload, so that calling c.foo() on an object c created before the reload causes the new code for foo to be executed.

Some of the known remaining caveats are:

  • Replacing code objects does not always succeed: changing a @property in a class to an ordinary method or a method to a member variable can cause problems (but in old objects only).

  • Functions that are removed (eg. via monkey-patching) from a module before it is reloaded are not upgraded.

  • C extension modules cannot be reloaded, and so cannot be autoreloaded.

Use importlib

The purpose of the importlib package is two-fold. One is to provide the implementation of the import statement (and thus, by extension, the __import__() function) in Python source code. This provides an implementation of import which is portable to any Python interpreter. This also provides an implementation which is easier to comprehend than one implemented in a programming language other than Python.

Two, the components to implement import are exposed in this package, making it easier for users to create their own custom objects (known generically as an importer) to participate in the import process.

1
2
import importlib
importlib.reload <Your Module>

This can be use between Python file and Jupyter Notebook, but has to import the that module again in each Cell.


References

[1] autoreload — IPython 3.2.1 documentation - https://ipython.org/ipython-doc/stable/config/extensions/autoreload.html

[2] Using autoreload to speed up IPython and Jupyter work - wrighters.io - https://www.wrighters.io/using-autoreload-to-speed-up-ipython-and-jupyter-work/

[3] importlib — The implementation of import — Python 3.9.5 documentation - https://docs.python.org/3/library/importlib.html

[4] Project Jupyter | Home - https://jupyter.org/