Page MenuHomeSoftware Heritage
Differential D6153

webapp/config: Bump content_display_max_size to 5 Mb
ClosedPublic
Actions

Authored by anlambert on Aug 27 2021, 3:14 PM.

Details

Reviewers
ardumont
Maniphest Tasks
T3454: Displaying "large" PDF raise a Javascript error
Summary

Previous value was 1 Mb, which seemed a bit low.

For instance, that pdf file of that Jupyter notebook whose size
is slightly greater than 1 Mb cannot be displayed by the webapp currently.

Related to T3454

Diff Detail

Repository
rSPSITE puppet-swh-site
Branch
staging
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 23236
Build 36255: arc lint + arc unit

Revision Contents
Changeset List

PathSize
data/
common/
2 lines

Diff 22279

View Options

data/common/common.yaml

Show First 20 Lines • Show All 240 Lines • ▼ Show 20 Linesclass MavenMapping(DictMapping, SingleFileMapping):
""" """
filename = b'pom.xml' filename = b'pom.xml'
mapping = CROSSWALK_TABLE['Java (Maven)'] mapping = CROSSWALK_TABLE['Java (Maven)']
def translate(self, content): def translate(self, content):
d = xmltodict.parse(content)['project'] d = xmltodict.parse(content)['project']
metadata = self.translate_dict(d, normalize=False) metadata = self.translate_dict(d, normalize=False)
metadata[SCHEMA_URI+'codeRepository'] = self.parse_repositories(d) metadata[SCHEMA_URI+'codeRepository'] = self.parse_repositories(d)
metadata[SCHEMA_URI+'license'] = self.parse_licenses(d)
return self.normalize_translation(metadata) return self.normalize_translation(metadata)
_default_repository = {'url': 'https://repo.maven.apache.org/maven2/'} _default_repository = {'url': 'https://repo.maven.apache.org/maven2/'}
def parse_repositories(self, d): def parse_repositories(self, d):
"""https://maven.apache.org/pom.html#Repositories""" """https://maven.apache.org/pom.html#Repositories"""
if 'repositories' not in d: if 'repositories' not in d:
return [self.parse_repository(d, self._default_repository)] return [self.parse_repository(d, self._default_repository)]
Show All 16 Lines def parse_repository(self, d, repo):
url = os.path.join(url, *d['groupId'].split('.')) url = os.path.join(url, *d['groupId'].split('.'))
if d['artifactId']: if d['artifactId']:
url = os.path.join(url, d['artifactId']) url = os.path.join(url, d['artifactId'])
return {"@id": url} return {"@id": url}
def normalize_groupId(self, id_): def normalize_groupId(self, id_):
return {"@id": id_} return {"@id": id_}
def parse_licenses(self, d):
"""https://maven.apache.org/pom.html#Licenses
zackUnsubmitted
Done
Inline Actions

What happens when multiple <license> elements are present?

It would be nice to have:

  • a dedicated test case for that, and
  • a description of the general semantics of how that is handled in the docstirng of this method
zack: What happens when multiple <license> elements are present? It would be nice to have: - a…
vlorentzAuthorUnsubmitted
Done
Inline Actions

This method always returns a list, so there's nothing interesting here. However, JSON-LD compaction happens later, so I added tests for that.

vlorentz: This method always returns a list, so there's nothing interesting here. However, JSON-LD…
zackUnsubmitted
Not Done
Inline Actions

The fact that I wanted documented (in the docstring, type annotation, whatever) was precisely that it always returns a list :-)

zack: The fact that I wanted documented (in the docstring, type annotation, whatever) was precisely…
vlorentzAuthorUnsubmitted
Done
Inline Actions

True. I added a task to remember to document all these methods further: T1403

vlorentz: True. I added a task to remember to document all these methods further: T1403
The origin XML has the form:
<licenses>
<license>
<name>Apache License, Version 2.0</name>
<url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
</license>
</licenses>
Which was translated to a dict by xmltodict and is given as `d`:
>>> from pprint import pprint
>>> d = {
... # ...
... "licenses": {
... "license": {
... "name": "Apache License, Version 2.0",
... "url":
... "https://www.apache.org/licenses/LICENSE-2.0.txt"
... }
... }
... }
>>> MavenMapping().parse_licenses(d)
[{'@id': 'https://www.apache.org/licenses/LICENSE-2.0.txt'}]
>>> d = {
... # ...
... "licenses": {
... "license": [
... {
... "name": "Apache License, Version 2.0",
... "url":
... "https://www.apache.org/licenses/LICENSE-2.0.txt"
... },
... {
... "name": "MIT License, ",
... "url": "https://opensource.org/licenses/MIT"
... }
... ]
... }
... }
>>> pprint(MavenMapping().parse_licenses(d))
[{'@id': 'https://www.apache.org/licenses/LICENSE-2.0.txt'},
{'@id': 'https://opensource.org/licenses/MIT'}]
or, if there are more than one license:
"""
licenses = d.get('licenses', {}).get('license', [])
if isinstance(licenses, dict):
licenses = [licenses]
return [{"@id": license['url']} for license in licenses]
def main(): def main():
raw_content = """{"name": "test_name", "unknown_term": "ut"}""" raw_content = """{"name": "test_name", "unknown_term": "ut"}"""
raw_content1 = b"""{"name": "test_name", raw_content1 = b"""{"name": "test_name",
"unknown_term": "ut", "unknown_term": "ut",
"prerequisites" :"packageXYZ"}""" "prerequisites" :"packageXYZ"}"""
result = MAPPINGS["NpmMapping"].translate(raw_content) result = MAPPINGS["NpmMapping"].translate(raw_content)
result1 = MAPPINGS["MavenMapping"].translate(raw_content1) result1 = MAPPINGS["MavenMapping"].translate(raw_content1)
print(result) print(result)
print(result1) print(result1)
if __name__ == "__main__": if __name__ == "__main__":
main() main()
About · Developer information · Legal