here is the traceback

Hello,

I don't reproduce... thus hard to troubleshoot.

I tested from the latest commits in swh-docs (which got updated yesterday).
I cleaned up eventual remaining tryout files first (make distclean and make, cf. P184).

From the stacktrace you provided:

• regarding version, we seem to share the same package version (i use sphinx's debian unstable version 1.6.4).

ii  sphinx-doc                                            1.4.9-2                         all                             documentation generator for Python projects - documentation
ii  python3-recommonmark                                  0.4.0+ds-2                      all                             CommonMark utility for Docutils and Sphinx projects -- Python 3
ii  python3-recommonmark                                  0.4.0+ds-2                      all                             CommonMark utility for Docutils and Sphinx projects -- Python 3
ii  python3-sphinxcontrib.httpdomain                      1.3.0-1                         all                             Sphinx domain for HTTP APIs - Python 3.x
ii  python3-sphinx                                        1.6.4-2                         all                             documentation generator for Python projects (implemented in Python 3)

• it seems that somehow the error is not understanding the`mailheader` keyword (stacktrace: there is an error about a mailheader key which does not exist which triggers another error)...

swh-deposit does not use that keyword.

Ok, looking for that mailheader in swh-environment, i'm finding references in swh-web as sphinx keyword :mailheader:.
(http://www.sphinx-doc.org/en/1.6.4/markup/inline.html#role-mailheader)

As it works locally, that leads me to think that you may be missing something somehow.

As a note, I recall having seen a new dependency in the latest swh-docs commit regarding the configuration (there sphinx.ext.extlinks, apt-file search says debian's sphinx-docs package)...
I don't think it's related though (since i did not have it prior to running and everything was buiding fine).

Maybe running the cd ../../swh-web/docs; make html could show more?

apparently due to swh-deposit.

The initial sphinx output is misleading. Even though i don't reproduce the error, on my side (cf. P184), the sphinx message output seems to be scrambled when reaching swh-deposit.
Indeed, the percentage stops at swh-deposit (since computing stuff for it prints noise somehow... i don't see why though since i don't see any dangling print statement there...).

Cheers,

ahah

Something in the reasoning were not perfectly sound.
That made me check for any dangling pip3 installed software...
I did have sphinxcontrib-httpdomain-1.5.0 installed...

Removing it, i now reproduced the error:

python3 -c 'import sphinx.apidoc, sys; sys.exit(sphinx.apidoc.main(sys.argv))' --implicit-namespaces -o apidoc ..  . ../setup.py '../swh/*/tests' '../swh/*/tests/*' '../swh/*/*/tests/*' '../swh/*/*/*/tests/*' '../swh/*/migrations' '../swh/*/migrations/*' '../swh/*/*/migrations/*' '../swh/*/*/*/migrations/*' '../bin/*' '../utils/*'
Creating file apidoc/swh.rst.
Creating file apidoc/swh.web.rst.
#   sphinx.ext.extlinks (1.6.4) from /usr/lib/python3/dist-packages/sphinx/ext/extlinks.py
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/sphinxcontrib/httpdomain.py", line 589, in resolve_xref
    info = self.data[str(typ)][target]
KeyError: 'mailheader'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/sphinx/cmdline.py", line 306, in main
    app.build(opts.force_all, filenames)
  File "/usr/lib/python3/dist-packages/sphinx/application.py", line 339, in build
    self.builder.build_update()
  File "/usr/lib/python3/dist-packages/sphinx/builders/__init__.py", line 331, in build_update
    'out of date' % len(to_build))
  File "/usr/lib/python3/dist-packages/sphinx/builders/__init__.py", line 397, in build
    self.write(docnames, list(updated_docnames), method)
  File "/usr/lib/python3/dist-packages/sphinx/builders/__init__.py", line 434, in write
    self._write_serial(sorted(docnames))
  File "/usr/lib/python3/dist-packages/sphinx/builders/__init__.py", line 441, in _write_serial
    doctree = self.env.get_and_resolve_doctree(docname, self)
  File "/usr/lib/python3/dist-packages/sphinx/environment/__init__.py", line 910, in get_and_resolve_doctree
    self.apply_post_transforms(doctree, docname)
  File "/usr/lib/python3/dist-packages/sphinx/environment/__init__.py", line 957, in apply_post_transforms
    transformer.apply_transforms()
  File "/usr/lib/python3/dist-packages/sphinx/transforms/__init__.py", line 92, in apply_transforms
    Transformer.apply_transforms(self)
  File "/usr/lib/python3/dist-packages/docutils/transforms/__init__.py", line 171, in apply_transforms
    transform.apply(**kwargs)
  File "/usr/lib/python3/dist-packages/sphinx/transforms/post_transforms/__init__.py", line 89, in apply
    typ, target, node, contnode)
  File "/usr/lib/python3/dist-packages/sphinxcontrib/httpdomain.py", line 595, in resolve_xref
    return http_header_role(None, text, text, None, None)[0][0]
  File "/usr/lib/python3/dist-packages/sphinxcontrib/httpdomain.py", line 439, in http_header_role
    msg = inliner.reporter.error('%s is not unknown HTTP header' % header,
AttributeError: 'NoneType' object has no attribute 'reporter'

So the immediate workaround is to install a latest version of sphinxcontrib-httpdomain version (through pip3 which is at 1.5.0, debian is at 1.3.0)...
The more reasonable fix is to upgrade the python3-sphinxcontrib.httpdomain debian package.

Note that there seems to be something fishy in sphinx though.
I don't see in sphinx documentation the need for that library to use the :mailheader: keyword...

Cheers,

Your tryout doesn't show if you've updated your entire swh-environment or not. Have you?

Yes, i did but should have mentioned it.
The only way to be in sync as a team is to be on the latest (swh/puppet)-environment (reasonably true, all the more on week-end :)

In T810#14672, @ardumont wrote:

I did have sphinxcontrib-httpdomain-1.5.0 installed...
Removing it, i now reproduced the error:

Nice catch, so that't the issue.

An even saner fix would be to avoid depending on on sphinx >= 1.5, but will see with @anlambert whether he has strongs reasons for that dependencies or if it was just an overlook.

For the sake of completeness...

As it works locally, that leads me to think that you may be missing something somehow.

Well, that was the 'not perfectly sound' argument.
It could have been the other way around, that is i could have something more in my environment.
As a matter of fact, i did.

Note that there seems to be something fishy in sphinx though.
I don't see in sphinx documentation the need for that library to use the :mailheader: keyword...

No. It's not sphinx which is at fault, it's probably more the sphinxcontrib library...

It's probably the python3-sphinxcontrib.httpversion at version 1.3.0 which does not expect new keywords from a superior version of sphinx (1.6.4).
Indeed, the python3-sphinxcontrib.httpversion's setup.py requires statements is for Sphinx >= 1.0...
So 1.6.4 is ok but they probably did not account for new 'keywords' in there somehow...

Effectively I was using sphinxcontrib.httpdomain 1.5 installed trough pip so I did not catch the issue. But I also got mistaken by httpdomain doc which makes uses of the deprecated :mailheader: directive in the examples ...

I have also found another issue regarding the 'Link' header doc which is not present in httpdomain 1.3.