"TODO" 拡張機能の開発

このチュートリアルでは、 "Hello world" 拡張機能の開発 で作成した拡張機能よりもさらに包括的なものの作成を目指します。まさに、カスタム directive の書き方のガイドとなるため、カスタムノード、付加的設定値、カスタムイベントハンドラーを伴う様々なディレクティブが関わってきます。この目的を見据え、 ドキュメンテーション内のtodoエントリーを取り込み、それらを中心部に集める機能を持った todo 拡張を取り扱うことにします。これは、Sphinxで配布されている sphinxext.todo 拡張に類似したものになります。



To understand the design of this extension, refer to 重要なオブジェクト and ビルド・フェーズ.


  • A todo directive, containing some content that is marked with "TODO" and only shown in the output if a new config value is set. Todo entries should not be in the output by default.

  • A todolist directive that creates a list of all todo entries throughout the documentation.


  • todo, todolistと呼ばれる新しいディレクティブ

  • todo, todolistというディレクティブが使用された場合に、それを表現する慣習的な新しいドキュメントツリーのノード。もしも、新しいディレクティブが、既存のノードで表現可能なものだけを生成するのであれば、新しいノードを作成する必要はありません。

  • todo_include_todosという新しい設定値。設定値の名前は、一意性を保つために拡張名から始まる名前にしてください。この設定値はtodoのエントリーが、出力を行うかどうかを判断します

  • New event handlers: one for the doctree-resolved event, to replace the todo and todolist nodes, one for env-merge-info to merge intermediate results from parallel builds, and one for env-purge-doc (the reason for that will be covered later).


As with "Hello world" 拡張機能の開発, we will not be distributing this plugin via PyPI so once again we need a Sphinx project to call this from. You can use an existing project or create a new one using sphinx-quickstart.

ソース (source) と ビルド (build) のフォルダーをみなさんが分けて使用していることを想定します。 みなさんの拡張ファイルはご自身のプロジェクトのどこかのフォルダに入っていることでしょう。ここでは、次のようにします:

  1. source 内に _ext フォルダーを作成します

  2. Create a new Python file in the _ext folder called todo.py


└── source
    ├── _ext
    │   └── todo.py
    ├── _static
    ├── conf.py
    ├── somefolder
    ├── index.rst
    ├── somefile.rst
    └── someotherfile.rst


Open todo.py and paste the following code in it, all of which we will explain in detail shortly:

  1from docutils import nodes
  2from docutils.parsers.rst import Directive
  4from sphinx.locale import _
  5from sphinx.util.docutils import SphinxDirective
  8class todo(nodes.Admonition, nodes.Element):
  9    pass
 12class todolist(nodes.General, nodes.Element):
 13    pass
 16def visit_todo_node(self, node):
 17    self.visit_admonition(node)
 20def depart_todo_node(self, node):
 21    self.depart_admonition(node)
 24class TodolistDirective(Directive):
 26    def run(self):
 27        return [todolist('')]
 30class TodoDirective(SphinxDirective):
 32    # this enables content in the directive
 33    has_content = True
 35    def run(self):
 36        targetid = 'todo-%d' % self.env.new_serialno('todo')
 37        targetnode = nodes.target('', '', ids=[targetid])
 39        todo_node = todo('\n'.join(self.content))
 40        todo_node += nodes.title(_('Todo'), _('Todo'))
 41        self.state.nested_parse(self.content, self.content_offset, todo_node)
 43        if not hasattr(self.env, 'todo_all_todos'):
 44            self.env.todo_all_todos = []
 46        self.env.todo_all_todos.append({
 47            'docname': self.env.docname,
 48            'lineno': self.lineno,
 49            'todo': todo_node.deepcopy(),
 50            'target': targetnode,
 51        })
 53        return [targetnode, todo_node]
 56def purge_todos(app, env, docname):
 57    if not hasattr(env, 'todo_all_todos'):
 58        return
 60    env.todo_all_todos = [todo for todo in env.todo_all_todos
 61                          if todo['docname'] != docname]
 64def merge_todos(app, env, docnames, other):
 65    if not hasattr(env, 'todo_all_todos'):
 66        env.todo_all_todos = []
 67    if hasattr(other, 'todo_all_todos'):
 68        env.todo_all_todos.extend(other.todo_all_todos)
 71def process_todo_nodes(app, doctree, fromdocname):
 72    if not app.config.todo_include_todos:
 73        for node in doctree.traverse(todo):
 74            node.parent.remove(node)
 76    # Replace all todolist nodes with a list of the collected todos.
 77    # Augment each todo with a backlink to the original location.
 78    env = app.builder.env
 80    if not hasattr(env, 'todo_all_todos'):
 81        env.todo_all_todos = []
 83    for node in doctree.traverse(todolist):
 84        if not app.config.todo_include_todos:
 85            node.replace_self([])
 86            continue
 88        content = []
 90        for todo_info in env.todo_all_todos:
 91            para = nodes.paragraph()
 92            filename = env.doc2path(todo_info['docname'], base=None)
 93            description = (
 94                _('(The original entry is located in %s, line %d and can be found ') %
 95                (filename, todo_info['lineno']))
 96            para += nodes.Text(description, description)
 98            # Create a reference
 99            newnode = nodes.reference('', '')
100            innernode = nodes.emphasis(_('here'), _('here'))
101            newnode['refdocname'] = todo_info['docname']
102            newnode['refuri'] = app.builder.get_relative_uri(
103                fromdocname, todo_info['docname'])
104            newnode['refuri'] += '#' + todo_info['target']['refid']
105            newnode.append(innernode)
106            para += newnode
107            para += nodes.Text('.)', '.)')
109            # Insert into the todolist
110            content.append(todo_info['todo'])
111            content.append(para)
113        node.replace_self(content)
116def setup(app):
117    app.add_config_value('todo_include_todos', False, 'html')
119    app.add_node(todolist)
120    app.add_node(todo,
121                 html=(visit_todo_node, depart_todo_node),
122                 latex=(visit_todo_node, depart_todo_node),
123                 text=(visit_todo_node, depart_todo_node))
125    app.add_directive('todo', TodoDirective)
126    app.add_directive('todolist', TodolistDirective)
127    app.connect('doctree-resolved', process_todo_nodes)
128    app.connect('env-purge-doc', purge_todos)
129    app.connect('env-merge-info', merge_todos)
131    return {
132        'version': '0.1',
133        'parallel_read_safe': True,
134        'parallel_write_safe': True,
135    }

This is far more extensive extension than the one detailed in "Hello world" 拡張機能の開発, however, we will will look at each piece step-by-step to explain what's happening.

The node classes

Let's start with the node classes:

 1class todo(nodes.Admonition, nodes.Element):
 2    pass
 5class todolist(nodes.General, nodes.Element):
 6    pass
 9def visit_todo_node(self, node):
10    self.visit_admonition(node)
13def depart_todo_node(self, node):
14    self.depart_admonition(node)

ノードクラスは docutils.nodes の中で定義されているdocutilsの標準クラスを継承する以外には何もやる必要はありません。todonotewarningのように使用されなければならないため、Admonitionクラスを定義しています。todolistは単なる"一般"ノードです。


多くの拡張は独自のノードクラスを実装する必要はなく、 docutilsSphinx が提供するノードを使えば良いでしょう。


It is important to know that while you can extend Sphinx without leaving your conf.py, if you declare an inherited node right there, you'll hit an unobvious PickleError. So if something goes wrong, please make sure that you put inherited nodes into a separate Python module.

For more details, see:

The directive classes

A directive class is a class deriving usually from docutils.parsers.rst.Directive. The directive interface is also covered in detail in the docutils documentation; the important thing is that the class should have attributes that configure the allowed markup, and a run method that returns a list of nodes.

Looking first at the TodolistDirective directive:

1class TodolistDirective(Directive):
3    def run(self):
4        return [todolist('')]

It's very simple, creating and returning an instance of our todolist node class. The TodolistDirective directive itself has neither content nor arguments that need to be handled. That brings us to the TodoDirective directive:

 1class TodoDirective(SphinxDirective):
 3    # this enables content in the directive
 4    has_content = True
 6    def run(self):
 7        targetid = 'todo-%d' % self.env.new_serialno('todo')
 8        targetnode = nodes.target('', '', ids=[targetid])
10        todo_node = todo('\n'.join(self.content))
11        todo_node += nodes.title(_('Todo'), _('Todo'))
12        self.state.nested_parse(self.content, self.content_offset, todo_node)
14        if not hasattr(self.env, 'todo_all_todos'):
15            self.env.todo_all_todos = []
17        self.env.todo_all_todos.append({
18            'docname': self.env.docname,
19            'lineno': self.lineno,
20            'todo': todo_node.deepcopy(),
21            'target': targetnode,
22        })
24        return [targetnode, todo_node]

Several important things are covered here. First, as you can see, we're now subclassing the SphinxDirective helper class instead of the usual Directive class. This gives us access to the build environment instance using the self.env property. Without this, we'd have to use the rather convoluted self.state.document.settings.env. Then, to act as a link target (from TodolistDirective), the TodoDirective directive needs to return a target node in addition to the todo node. The target ID (in HTML, this will be the anchor name) is generated by using env.new_serialno which returns a new unique integer on each call and therefore leads to unique target names. The target node is instantiated without any text (the first two arguments).

On creating admonition node, the content body of the directive are parsed using self.state.nested_parse. The first argument gives the content body, and the second one gives content offset. The third argument gives the parent node of parsed result, in our case the todo node. Following this, the todo node is added to the environment. This is needed to be able to create a list of all todo entries throughout the documentation, in the place where the author puts a todolist directive. For this case, the environment attribute todo_all_todos is used (again, the name should be unique, so it is prefixed by the extension name). It does not exist when a new environment is created, so the directive must check and create it if necessary. Various information about the todo entry's location are stored along with a copy of the node.



| target node        |
| todo node          |
     | admonition title   |
     | paragraph          |
     | ...                |

The event handlers

Event handlers are one of Sphinx's most powerful features, providing a way to do hook into any part of the documentation process. There are many events provided by Sphinx itself, as detailed in the API guide, and we're going to use a subset of them here.

Let's look at the event handlers used in the above example. First, the one for the env-purge-doc event:

1def purge_todos(app, env, docname):
2    if not hasattr(env, 'todo_all_todos'):
3        return
5    env.todo_all_todos = [todo for todo in env.todo_all_todos
6                          if todo['docname'] != docname]

ソースファイルの中から情報を取り出し、環境の中に格納しましたが、これは永続化されます。そのため、ソースファイルが変更されると古い情報になってしまう可能性があります。そのため、それぞれのソースファイルを読み込む前に、環境の記録をクリアしています。 env-purge-doc イベントは、拡張機能の中でそのような作業を行うのに適した場所になります。ここではtodo_all_todosのリストの中の項目のうち、ドキュメントの名前(docname)がマッチしたものを削除しています。もしもドキュメント内のToDoが残っていたとしたら、パース時に重複して追加されてしまいます。

The next handler, for the env-merge-info event, is used during parallel builds. As during parallel builds all threads have their own env, there's multiple todo_all_todos lists that need to be merged:

1def merge_todos(app, env, docnames, other):
2    if not hasattr(env, 'todo_all_todos'):
3        env.todo_all_todos = []
4    if hasattr(other, 'todo_all_todos'):
5        env.todo_all_todos.extend(other.todo_all_todos)

The other handler belongs to the doctree-resolved event:

 1def process_todo_nodes(app, doctree, fromdocname):
 2    if not app.config.todo_include_todos:
 3        for node in doctree.traverse(todo):
 4            node.parent.remove(node)
 6    # Replace all todolist nodes with a list of the collected todos.
 7    # Augment each todo with a backlink to the original location.
 8    env = app.builder.env
10    if not hasattr(env, 'todo_all_todos'):
11        env.todo_all_todos = []
13    for node in doctree.traverse(todolist):
14        if not app.config.todo_include_todos:
15            node.replace_self([])
16            continue
18        content = []
20        for todo_info in env.todo_all_todos:
21            para = nodes.paragraph()
22            filename = env.doc2path(todo_info['docname'], base=None)
23            description = (
24                _('(The original entry is located in %s, line %d and can be found ') %
25                (filename, todo_info['lineno']))
26            para += nodes.Text(description, description)
28            # Create a reference
29            newnode = nodes.reference('', '')
30            innernode = nodes.emphasis(_('here'), _('here'))
31            newnode['refdocname'] = todo_info['docname']
32            newnode['refuri'] = app.builder.get_relative_uri(
33                fromdocname, todo_info['docname'])
34            newnode['refuri'] += '#' + todo_info['target']['refid']
35            newnode.append(innernode)
36            para += newnode
37            para += nodes.Text('.)', '.)')
39            # Insert into the todolist
40            content.append(todo_info['todo'])
41            content.append(para)
43        node.replace_self(content)

The doctree-resolved event is emitted at the end of phase 3 (resolving) and allows custom resolving to be done. The handler we have written for this event is a bit more involved. If the todo_include_todos config value (which we'll describe shortly) is false, all todo and todolist nodes are removed from the documents. If not, todo nodes just stay where and how they are. todolist nodes are replaced by a list of todo entries, complete with backlinks to the location where they come from. The list items are composed of the nodes from the todo entry and docutils nodes created on the fly: a paragraph for each entry, containing text that gives the location, and a link (reference node containing an italic node) with the backreference. The reference URI is built by sphinx.builders.Builder.get_relative_uri() which creates a suitable URI depending on the used builder, and appending the todo node's (the target's) ID as the anchor name.

setup 関数

As noted previously, the setup function is a requirement and is used to plug directives into Sphinx. However, we also use it to hook up the other parts of our extension. Let's look at our setup function:

 1def setup(app):
 2    app.add_config_value('todo_include_todos', False, 'html')
 4    app.add_node(todolist)
 5    app.add_node(todo,
 6                 html=(visit_todo_node, depart_todo_node),
 7                 latex=(visit_todo_node, depart_todo_node),
 8                 text=(visit_todo_node, depart_todo_node))
10    app.add_directive('todo', TodoDirective)
11    app.add_directive('todolist', TodolistDirective)
12    app.connect('doctree-resolved', process_todo_nodes)
13    app.connect('env-purge-doc', purge_todos)
14    app.connect('env-merge-info', merge_todos)
16    return {
17        'version': '0.1',
18        'parallel_read_safe': True,
19        'parallel_write_safe': True,
20    }

The calls in this function refer to the classes and functions we added earlier. What the individual calls do is the following:

  • add_config_value() メソッドはSphinxに対して新しい設定値である todo_include_todosを追加するように指示して、 conf.py の中に書けるようにします。このオプションのデフォルト値はFalseになります。また、この設定値がブーリアンの値を取るということもSphinxに知らせます。

    If the third argument was 'html', HTML documents would be full rebuild if the config value changed its value. This is needed for config values that influence reading (build phase 1 (reading)).

  • add_node() adds a new node class to the build system. It also can specify visitor functions for each supported output format. These visitor functions are needed when the new nodes stay until phase 4 (writing). Since the todolist node is always replaced in phase 3 (resolving), it doesn't need any.

  • add_directive() メソッドは、指定された名前とクラスから、新しいディレクティブを追加します。

  • 最後に、 connect() メソッドは、最初の引数に指定されたイベントの名前に対する、イベントハンドラを追加します。イベントハンドラの関数は、ドキュメントに関する引数をいくつか伴って呼び出されます。

With this, our extension is complete.

Using the extension

As before, we need to enable the extension by declaring it in our conf.py file. There are two steps necessary here:

  1. Add the _ext directory to the Python path using sys.path.append. This should be placed at the top of the file.

  2. Update or create the extensions list and add the extension file name to the list

In addition, we may wish to set the todo_include_todos config value. As noted above, this defaults to False but we can set it explicitly.


import os
import sys


extensions = ['todo']

todo_include_todos = False

You can now use the extension throughout your project. For example:

Hello, world

.. toctree::

Hello world. Below is the list of TODOs.

.. todolist::

Some intro text here...

.. todo:: Fix this

Some more text here...

.. todo:: Fix that

Because we have configured todo_include_todos to False, we won't actually see anything rendered for the todo and todolist directives. However, if we toggle this to true, we will see the output described previously.

Further reading

For more information, refer to the docutils documentation and Sphinx拡張機能の開発.