Developing extensions for Sphinx

Since many projects will need special features in their documentation, Sphinx is designed to be extensible on several levels.

This is what you can do in an extension: First, you can add new builders to support new output formats or actions on the parsed documents. Then, it is possible to register custom reStructuredText roles and directives, extending the markup. And finally, there are so-called “hook points” at strategic places throughout the build process, where an extension can register a hook and run specialized code.

An extension is simply a Python module. When an extension is loaded, Sphinx imports this module and executes its setup() function, which in turn notifies Sphinx of everything the extension offers – see the extension tutorial for examples.

The configuration file itself can be treated as an extension if it contains a setup() function. All other extensions to load must be listed in the extensions configuration value.

Discovery of builders by entry point

New in version 1.6.

Builder extensions can be discovered by means of entry points so that they do not have to be listed in the extensions configuration value.

Builder extensions should define an entry point in the sphinx.builders group. The name of the entry point needs to match your builder’s name attribute, which is the name passed to the sphinx-build -b option. The entry point value should equal the dotted name of the extension module. Here is an example of how an entry point for ‘mybuilder’ can be defined in the extension’s setup.py:

setup(
    # ...
    entry_points={
        'sphinx.builders': [
            'mybuilder = my.extension.module',
        ],
    }
)

Note that it is still necessary to register the builder using add_builder() in the extension’s setup() function.

Important objects

There are several key objects whose API you will use while writing an extension. These are:

Application

The application object (usually called app) is an instance of Sphinx. It controls most high-level functionality, such as the setup of extensions, event dispatching and producing output (logging).

If you have the environment object, the application is available as env.app.

Environment

The build environment object (usually called env) is an instance of BuildEnvironment. It is responsible for parsing the source documents, stores all metadata about the document collection and is serialized to disk after each build.

Its API provides methods to do with access to metadata, resolving references, etc. It can also be used by extensions to cache information that should persist for incremental rebuilds.

If you have the application or builder object, the environment is available as app.env or builder.env.

Builder

The builder object (usually called builder) is an instance of a specific subclass of Builder. Each builder class knows how to convert the parsed documents into an output format, or otherwise process them (e.g. check external links).

If you have the application object, the builder is available as app.builder.

Config

The config object (usually called config) provides the values of configuration values set in conf.py as attributes. It is an instance of Config.

The config is available as app.config or env.config.

To see an example of use of these objects, refer to Extension tutorials.

Build Phases

One thing that is vital in order to understand extension mechanisms is the way in which a Sphinx project is built: this works in several phases.

Phase 0: Initialization

In this phase, almost nothing of interest to us happens. The source directory is searched for source files, and extensions are initialized. Should a stored build environment exist, it is loaded, otherwise a new one is created.

Phase 1: Reading

In Phase 1, all source files (and on subsequent builds, those that are new or changed) are read and parsed. This is the phase where directives and roles are encountered by docutils, and the corresponding code is executed. The output of this phase is a doctree for each source file; that is a tree of docutils nodes. For document elements that aren’t fully known until all existing files are read, temporary nodes are created.

There are nodes provided by docutils, which are documented in the docutils documentation. Additional nodes are provided by Sphinx and documented here.

During reading, the build environment is updated with all meta- and cross reference data of the read documents, such as labels, the names of headings, described Python objects and index entries. This will later be used to replace the temporary nodes.

The parsed doctrees are stored on the disk, because it is not possible to hold all of them in memory.

Phase 2: Consistency checks

Some checking is done to ensure no surprises in the built documents.

Phase 3: Resolving

Now that the metadata and cross-reference data of all existing documents is known, all temporary nodes are replaced by nodes that can be converted into output using components called transforms. For example, links are created for object references that exist, and simple literal nodes are created for those that don’t.

Phase 4: Writing

This phase converts the resolved doctrees to the desired output format, such as HTML or LaTeX. This happens via a so-called docutils writer that visits the individual nodes of each doctree and produces some output in the process.

Note

Some builders deviate from this general build plan, for example, the builder that checks external links does not need anything more than the parsed doctrees and therefore does not have phases 2–4.

To see an example of application, refer to Developing a “TODO” extension.

Extension metadata

New in version 1.3.

The setup() function can return a dictionary. This is treated by Sphinx as metadata of the extension. Metadata keys currently recognized are:

  • 'version': a string that identifies the extension version. It is used for extension version requirement checking (see needs_extensions) and informational purposes. If not given, "unknown version" is substituted.

  • 'env_version': an integer that identifies the version of env data structure if the extension stores any data to environment. It is used to detect the data structure has been changed from last build. The extensions have to increment the version when data structure has changed. If not given, Sphinx considers the extension does not stores any data to environment.

  • 'parallel_read_safe': a boolean that specifies if parallel reading of source files can be used when the extension is loaded. It defaults to False, i.e. you have to explicitly specify your extension to be parallel-read-safe after checking that it is.

  • 'parallel_write_safe': a boolean that specifies if parallel writing of output files can be used when the extension is loaded. Since extensions usually don’t negatively influence the process, this defaults to True.