FileNotFoundError when autodocumenting highlightjs.py
Closed, MigratedEdits Locked
Actions

Assigned To

Authored By

	vlorentz
	Apr 13 2021, 4:49 PM

Description

Sphinx raises a bunch of those

WARNING: autodoc: failed to import module 'urls' from module 'swh.web'; the following exception was raised:
Traceback (most recent call last):
  File "/home/dev/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/sphinx/ext/autodoc/importer.py", line 71, in import_module
    return importlib.import_module(modname)
  File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/dev/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/web/urls.py", line 47, in <module>
    url(r"^browse/", include("swh.web.browse.urls")),
  File "/home/dev/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/django/urls/conf.py", line 34, in include
    urlconf_module = import_module(urlconf_module)
  File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/dev/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/web/browse/urls.py", line 11, in <module>
    import swh.web.browse.views.content  # noqa
  File "/home/dev/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/web/browse/views/content.py", line 18, in <module>
    from swh.web.browse.snapshot_context import get_snapshot_context
  File "/home/dev/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/web/browse/snapshot_context.py", line 27, in <module>
    from swh.web.browse.utils import (
  File "/home/dev/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/web/browse/utils.py", line 18, in <module>
    from swh.web.common import archive, highlightjs
  File "/home/dev/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/web/common/highlightjs.py", line 15, in <module>
    with open(str(find("json/highlightjs-languages.json")), "r") as _hljs_languages_file:
FileNotFoundError: [Errno 2] No such file or directory: 'None'

Revisions and Commits

rDDOC Development documentation
	D5530	rDDOC6a380d85e4c1 django_settings: Ensure swh-web can find its static files with autodoc

Related Objects
Search...

		Status	Assigned	Task
		Migrated	gitlab-migration	T2265 Building the documentation should not show any warning.
		Migrated	gitlab-migration	T3248 FileNotFoundError when autodocumenting highlightjs.py

Event Timeline

vlorentz triaged this task as Normal priority.Apr 13 2021, 4:49 PM

vlorentz created this task.

vlorentz added a parent task: T2265: Building the documentation should not show any warning..

vlorentz updated the task description. (Show Details)

I tried this:

diff --git a/swh/web/common/highlightjs.py b/swh/web/common/highlightjs.py
index 54661d62..b98c79a2 100644
--- a/swh/web/common/highlightjs.py
+++ b/swh/web/common/highlightjs.py
@@ -5,6 +5,7 @@
 
 import functools
 import json
+import pkg_resources
 from typing import Dict
 
 from pygments.lexers import get_all_lexers, get_lexer_for_filename
@@ -12,8 +13,17 @@
 
 from django.contrib.staticfiles.finders import find
 
-with open(str(find("json/highlightjs-languages.json")), "r") as _hljs_languages_file:
-    _hljs_languages_data = json.load(_hljs_languages_file)
+_hljs_filename = "json/highlightjs-languages.json"
+_hljs_path = find(_hljs_filename)
+
+if _hljs_path is None:
+    # Happens when ran outside Django, eg. by Sphinx
+    _hljs_languages_data = json.load(
+        pkg_resources.resource_string(__name__, _hljs_filename)
+    )
+else:
+    with open(str(_hljs_path), "r") as _hljs_languages_file:
+        _hljs_languages_data = json.load(_hljs_languages_file)
 
 # set of languages ids that can be highlighted by highlight.js library
 _hljs_languages = set(_hljs_languages_data["languages"])

but it errors with:

  File "/home/dev/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/web/common/highlightjs.py", line 22, in <module>
    pkg_resources.resource_string(__name__, _hljs_filename)
  File "/home/dev/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1143, in resource_string
    self, resource_name
  File "/home/dev/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1388, in get_resource_string
    return self._get(self._fn(self.module_path, resource_name))
  File "/home/dev/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1611, in _get
    with open(path, 'rb') as stream:
FileNotFoundError: [Errno 2] No such file or directory: '/home/dev/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/web/common/json/highlightjs-languages.json'

because the data is in /home/dev/swh-environment/swh-docs/.tox/sphinx-dev/share/swh/web/static/json/highlightjs-languages.json........

anlambert added a revision: D5530: django_settings: Ensure swh-web can find its static files with autodoc.Apr 14 2021, 9:01 PM

Proper fix is in D5530.

anlambert closed this task as Resolved by committing rDDOC6a380d85e4c1: django_settings: Ensure swh-web can find its static files with autodoc.Apr 15 2021, 10:51 AM

anlambert claimed this task.

anlambert added a commit: rDDOC6a380d85e4c1: django_settings: Ensure swh-web can find its static files with autodoc.