Sphinx is a python documentation generator, it was created to generate the Python documentation, so it is a very complete tool. It uses reStructuredTest as its markup language.

This article will focus on generating sphinx documentation as automatic as possible, allowing to generate documentation out of your pydoc strings.

Creating the documentation directory

Suppose your your example application:

APPDIR=$DEVDIR/myapps/example/src

1. Install sphinx

sudo apt-get install python-sphinx

2. Add the directory for documentation and the initial files.

mkdir $APPDIR/docs
cd $APPDIR/docs
sphinx-quickstart

The sphinx-quickstart command will ask you some questions. Most items can be left to their default value, but here we highlight some convenient ones to edit. The most important to specify is the autodoc, so be sure you answer y in that field.

> Project name: MyApp Project
> Author name(s): RidgeRun Engineering LLC
> Project version: 0.1
> autodoc: automatically insert docstrings from modules (y/N) [n]: y       # <---- Important!
> Create Makefile? (Y/n) [y]: y
> Create Windows command file? (Y/n) [y]: n                                # Not necessary in Linux

Once you have finished, you should have a directory structure as follows:

$ tree $APPDIR/docs
~/myapp/docs
├── _build
├── conf.py
├── index.rst
├── Makefile
├── _static
└── _templates
3 directories, 3 files

You might want to commit this initial structure (to avoid from committing automatically generated files later).

Editing conf.py

The configuration file ($APPDIR/docs/conf.py) is very important to sphinx, it is a python file that will be executed prior to any documentation generation.

1. Edit sys.path

You need to include the necessary source files folders in the PATH to be able to generate the documentation from.

The first code to add is the one that obtains DEVDIR from the environment.

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#sys.path.insert(0, os.path.abspath('.'))

# This will include the necessary source files folders in the PATH to be able to generate the documentation from.
devdir=''
try:
    if os.environ['DEVDIR']:
        devdir = os.environ['DEVDIR'] 
except KeyError:
    print 'Unable to obtain $DEVDIR from the environment.'
    exit(-1)

If the python code you are creating is made to be run on the board, you might be safer if you just include your app's code in the path, and not include directories like /fs/fs/usr/lib/python2.7/site-packages to avoid import problems (more info on this later):

sys.path.insert(0, devdir + '/myapps/example/src')

If the python code you are creating was made for your host machine, then you might have to add lines like the following:

sys.path.insert(0, devdir + '/yourapp/src')
sys.path.insert(0, devdir + '/bsp/local/lib/python2.7/site-packages')

2. Add some custom features (optional)

The autodoc_member_order allows you to override the default alphabetical ordering for members, to one that groups by their type, i.e. if they are methods, or attributes/properties, etc.

# Sort members by type
autodoc_member_order = 'groupwise'

The init method (by default a special method and private) will not get documented. Sphinx offers a special_members directive to include special members in the documentation, but there are many other special members that you don't want documented. This hack will only get your __init__ method documented using autodoc-skip-member.

# Ensure that the __init__ method gets documented.
def skip(app, what, name, obj, skip, options):
    if name == "__init__":
        return False
    return skip

def setup(app):
    app.connect("autodoc-skip-member", skip)

3. Add exclusion patterns

If you want to omit generating documentation for some of the modules in your application's python code (for example everything under a tests/ directory), or a spi.py module that you only use as a library, then use the exclude_patterns directive.

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['_build', '**tests**', '**spi**']

4. Add logic to mock modules

Although you could include the board's python libraries in the PATH (directories like /fs/fs/usr/lib/python2.7/site-packages), it is not recommended because some modules that were cross-compiled will give you problems, or your host machine might be missing modules that are used in the python code for the target. For example:

ImportError: No module named smbus          # Couldn't find the smbus module in the host
ImportError: wrong ELF class: ELFCLASS32    # Sphinx found a C module (.so file), but it was cross-compiled

The following code snippet allows you to have a list of modules that you can mock for the purposes of generating documentation. In this example we are mocking the smbus and pygst modules.

import mock, sys
MOCK_MODULES = ['smbus','pygst']
for mod_name in MOCK_MODULES:
    sys.modules[mod_name] = mock.Mock()

Note that you need python-mock:

sudo apt-get install python-mock

5. Add logic to mock calls

This is particularly important for python code that is made to run in your board. For example, your code might call a function read_reg to read a register from an FPGA using SPI; when sphinx (running in your host) executes this code for the purpose of generating documentation, it will fail.

The following code allows you to mock a call to a read_reg() function from the module examplemod:

import mox as mox 
import examplemod
m = mox.Mox()
m.StubOutWithMock(examplemod, 'read_reg')

Note that you need python-mox:

 sudo apt-get install python-mox

Common problem: calling open()

It may be the case that your python code for the target calls open() to open a file that only exists in your target's file system (i.e /etc/version). This will also cause sphinx to fail because the file doesn't exist. The easy fix is to modify your code to first verify the file's existence. Something like:

import os
if os.path.isfile(filename):
    # open file here
else:
    # handle error here - in a way that sphinx doesn't crash

Edit index.rst

The main index.rst file can be modified to include your application modules automatically (watch out for identation).

Contents:

.. toctree::
   :maxdepth: 2
   
   modules

Indices and tables

You can also edit this file further for custom titles, descriptions, etc. using the rst markup language.

Generating documentation

Before starting to generate documentation, you might want to commit what you have done so far, so that automatically generated files after this point are not versioned. In the moment that you version a file, you will have to maintain it manually.

Generating .rst files

The .rst files are automatically generated using the sphinx-apidoc command.

 sphinx-apidoc -o $APPDIR/docs $APPDIR

You can also add a target to your Makefile ($APPDIR/docs/Makefile):

rst:
  sphinx-apidoc -f -o . ..

Generating the html documentation

Now that you have everything in place, just run:

cd $APPDIR/docs
make html

And optionally clean: I

make clean

You can view your generated documentation:

firefox $APPDIR/docs/_build/html/index.html

And optionally also add a target to your Makefile to do this automatically:

view:
  firefox _build/html/index.html

References

Restructured Text (reST) and Sphinx CheatSheet

Documenting Your Project Using Sphinx

Module-specific markup

How to use Sphinx's autodoc to document a class's __init__(self) method?

Sphinx - file not found - sys.path issue

Sphinx: how to exclude imports in automodule? - used to mock modules.

Mocking open(file_name) in unit tests - used to mock calls.