How to generate sphinx documentation for python code running in an embedded system
Sphinx is a python documentation generator, it was created to generate the Python documentation, so it is a very complete tool. This Python document generator uses reStructuredTest as its markup language.
This article will focus on generating sphinx documentation as automatic as possible, allowing to generate documentation out of your pydoc strings.
Creating the documentation directory
Suppose your example application:
APPDIR=$DEVDIR/myapps/example/src
1. Install sphinx
sudo apt-get install python-sphinx
2. Add the directory for documentation and the initial files.
mkdir $APPDIR/docs cd $APPDIR/docs sphinx-quickstart
The sphinx-quickstart command will ask you some questions. Most items can be left to their default value, but here we highlight some convenient ones to edit. The most important to specify is the autodoc, so be sure you answer y in that field.
> Project name: MyApp Project > Author name(s): RidgeRun Engineering LLC > Project version: 0.1 > autodoc: automatically insert docstrings from modules (y/N) [n]: y # <---- Important! > Create Makefile? (Y/n) [y]: y > Create Windows command file? (Y/n) [y]: n # Not necessary in Linux
Once you have finished, you should have a directory structure as follows:
$ tree $APPDIR/docs ../myapps/example/src/docs ├── _build ├── conf.py ├── index.rst ├── Makefile ├── _static └── _templates 3 directories, 3 files
You might want to commit this initial structure (to avoid from committing automatically generated files later).
Editing conf.py
The configuration file ($APPDIR/docs/conf.py) is very important to sphinx, it is a python file that will be executed prior to any documentation generation.
1. Edit sys.path
You need to include the necessary source files folders in the PATH to be able to generate the documentation from.
The first code to add is the one that obtains DEVDIR from the environment.
# If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. #sys.path.insert(0, os.path.abspath('.')) # This will include the necessary source files folders in the PATH to be able to generate the documentation from. devdir='' try: if os.environ['DEVDIR']: devdir = os.environ['DEVDIR'] except KeyError: print 'Unable to obtain $DEVDIR from the environment.' exit(-1)
If the python code you are creating is made to be run on the board, you might be safer if you just include your app's code in the path, and not include directories like /fs/fs/usr/lib/python2.7/site-packages to avoid import problems (more info on this later):
sys.path.insert(0, devdir + '/myapps/example/src')
If the python code you are creating was made for your host machine, then you might have to add lines like the following:
sys.path.insert(0, devdir + '/yourapp/src') sys.path.insert(0, devdir + '/bsp/local/lib/python2.7/site-packages')
2. Add some custom features (optional)
The autodoc_member_order allows you to override the default alphabetical ordering for members, to one that groups by their type, i.e. if they are methods, or attributes/properties, etc.
# Sort members by type autodoc_member_order = 'groupwise'
The init method (by default a special method and private) will not get documented. Sphinx offers a special_members directive to include special members in the documentation, but there are many other special members that you don't want documented. This hack will only get your __init__ method documented using autodoc-skip-member.
# Ensure that the __init__ method gets documented. def skip(app, what, name, obj, skip, options): if name == "__init__": return False return skip def setup(app): app.connect("autodoc-skip-member", skip)
3. Add exclusion patterns
If you want to omit generating documentation for some of the modules in your application's python code (for example everything under a tests/ directory), or a spi.py module that you only use as a library, then use the exclude_patterns directive.
# List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = ['_build', '**tests**', '**spi**']
4. Add logic to mock modules
Although you could include the board's python libraries in the PATH (directories like /fs/fs/usr/lib/python2.7/site-packages), it is not recommended because some modules that were cross-compiled will give you problems, or your host machine might be missing modules that are used in the python code for the target. For example:
ImportError: No module named smbus # Couldn't find the smbus module in the host ImportError: wrong ELF class: ELFCLASS32 # Sphinx found a C module (.so file), but it was cross-compiled
The following code snippet allows you to have a list of modules that you can mock for the purposes of generating documentation. In this example we are mocking the smbus and pygst modules.
import mock, sys MOCK_MODULES = ['smbus','pygst'] for mod_name in MOCK_MODULES: sys.modules[mod_name] = mock.Mock()
Note that you need python-mock:
sudo apt-get install python-mock
5. Add logic to mock calls
This is particularly important for python code that is made to run in your board. For example, your code might call a function read_reg to read a register from an FPGA using SPI; when sphinx (running in your host) executes this code for the purpose of generating documentation, it will fail.
The following code allows you to mock a call to a read_reg() function from the module examplemod:
import mox as mox import examplemod m = mox.Mox() m.StubOutWithMock(examplemod, 'read_reg')
Note that you need python-mox:
sudo apt-get install python-mox
Common problem: calling open()
It may be the case that your python code for the target calls open() to open a file that only exists in your target's file system (i.e /etc/version). This will also cause sphinx to fail because the file doesn't exist. The easy fix is to modify your code to first verify the file's existence. Something like:
import os if os.path.isfile(filename): # open file here else: # handle error here - in a way that sphinx doesn't crash
Edit index.rst
The main index.rst file can be modified to include your application modules automatically (watch out for identation).
Contents: .. toctree:: :maxdepth: 2 modules Indices and tables
You can also edit this file further for custom titles, descriptions, etc. using the rst markup language.
Generating documentation
Before starting to generate documentation, you might want to commit what you have done so far, so that automatically generated files after this point are not versioned. In the moment that you version a file, you will have to maintain it manually.
Generating .rst files
The .rst files are automatically generated using the sphinx-apidoc command.
sphinx-apidoc -o $APPDIR/docs $APPDIR
You can also add a target to your Makefile ($APPDIR/docs/Makefile):
rst: sphinx-apidoc -f -o . ..
Generating the html documentation
Now that you have everything in place, just run:
cd $APPDIR/docs make html
And optionally clean:
make clean
You can view your generated documentation:
firefox $APPDIR/docs/_build/html/index.html
And optionally also add a target to your Makefile to do this automatically:
view: firefox _build/html/index.html
See also
Restructured Text (reST) and Sphinx CheatSheet
Documenting Your Project Using Sphinx
How to use Sphinx's autodoc to document a class's __init__(self) method?
Sphinx - file not found - sys.path issue
Sphinx: how to exclude imports in automodule? - used to mock modules.
Mocking open(file_name) in unit tests - used to mock calls.