Page MenuHomeSoftware Heritage
Differential D4312

Add `Hg20BundleLoader` tests from json files
ClosedPublic
Actions

Authored by acezar on Oct 19 2020, 10:42 PM.

Details

Reviewers
douardda
Group Reviewers
Reviewers
Commits
rDLDHGbc32e1280cfd: Add `Hg20BundleLoader` tests from json files
Summary

Generated json files with swh/loader/mercurial/tests/data/build.py for
existing repositories and added them to Hg20BundleLoader tests.

Introduce LoaderChecker as a standardized way to test repositories
against json files.

Diff Detail

Unit TestsFailed
View All

TimeTest
170 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.mercurial.tests.test_identify::test_all
datadir = '/var/lib/jenkins/workspace/DLDHG/tests-on-diff/.tox/py3/lib/python3.7/site-packages/swh/loader/mercurial/tests/data' tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_all0')
166 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.mercurial.tests.test_identify::test_all_revisions
datadir = '/var/lib/jenkins/workspace/DLDHG/tests-on-diff/.tox/py3/lib/python3.7/site-packages/swh/loader/mercurial/tests/data' tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_all_revisions0')
177 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.mercurial.tests.test_identify::test_single_revision
datadir = '/var/lib/jenkins/workspace/DLDHG/tests-on-diff/.tox/py3/lib/python3.7/site-packages/swh/loader/mercurial/tests/data' tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_single_revision0')
2 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.mercurial.tests.test_converters.TestParseAuthorConverters::test_parse_author_2
1 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.mercurial.tests.test_converters.TestParseAuthorConverters::test_parse_author_3
View Full Test Results (3 Failed · 18 Passed)

Event Timeline

acezar created this revision.Oct 19 2020, 10:42 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptOct 19 2020, 10:42 PM
swh-public-ci added a comment.Oct 19 2020, 10:43 PM

Build has FAILED

Patch application report for D4312 (id=15241)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..db9a43d
Fast-forward
 setup.py                                          |   2 +
 swh/loader/mercurial/identify.py                  | 494 ++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py          | 187 ++++++++
 swh/loader/mercurial/tests/data/hello.json        |   1 +
 swh/loader/mercurial/tests/data/the-sandbox.json  |   1 +
 swh/loader/mercurial/tests/data/transplant.json   |   1 +
 swh/loader/mercurial/tests/dvcs_loader_checker.py | 132 ++++++
 swh/loader/mercurial/tests/test_identify.py       |  88 ++++
 swh/loader/mercurial/tests/test_loader.py         |  12 +
 9 files changed, 918 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/data/hello.json
 create mode 100644 swh/loader/mercurial/tests/data/the-sandbox.json
 create mode 100644 swh/loader/mercurial/tests/data/transplant.json
 create mode 100644 swh/loader/mercurial/tests/dvcs_loader_checker.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit db9a43d8c0b532a06c07fb357af81726e2f957a7
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:22:07 2020 +0200

    Add `Hg20BundleLoader` tests from json files
    
    Generated json files with `swh/loader/mercurial/tests/data/build.py` for
    existing repositories and added them to `Hg20BundleLoader` tests.

commit 4a92800d8ab7cd108ad621b6d9da1533c09886d3
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder

commit b543125f433ab3022cc80c5b76e3a2eab0162b61
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/63/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/63/console

Harbormaster failed remote builds in B16473: Diff 15241!Oct 19 2020, 10:43 PM
acezar added a parent revision: D4311: Add testing repository builder.Oct 19 2020, 10:47 PM
acezar added a child revision: D4313: Add new example repository generated from script.
douardda added a subscriber: douardda.Oct 22 2020, 11:52 AM

Would be nice to have a README file in tests/data explaining what these json files are and how to produce them.

douardda accepted this revision.Oct 22 2020, 12:32 PM

globally ok, but please add a README file as suggested in the previous comment

This revision is now accepted and ready to land.Oct 22 2020, 12:32 PM
acezar updated this revision to Diff 15407.Oct 27 2020, 10:19 AM
acezar edited the summary of this revision. (Show Details)

Updated commit message

swh-public-ci added a comment.Oct 27 2020, 10:20 AM

Build has FAILED

Patch application report for D4312 (id=15407)

Rebasing onto bd914dec39...

Current branch diff-target is up to date.
Changes applied before test
commit f5a7a0cdf21fdf9823cff33895883638bb495415
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:22:07 2020 +0200

    Add `Hg20BundleLoader` tests from json files
    
    Generated json files with `swh/loader/mercurial/tests/data/build.py` for
    existing repositories and added them to `Hg20BundleLoader` tests.
    
    Introduce `DVCSLoaderChecker` as a standardized way to test repositories
    against json files.

commit 5b0256683349ad892ffed749e7dc45d8b768e5fb
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder

commit c3d959fe1cf5bee461ff7366697f907a578d9288
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/69/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/69/console

Harbormaster failed remote builds in B16636: Diff 15407!Oct 27 2020, 10:20 AM
acezar updated this revision to Diff 15417.Oct 27 2020, 11:06 AM

Fighting arc

swh-public-ci added a comment.Oct 27 2020, 11:07 AM

Build has FAILED

Patch application report for D4312 (id=15417)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..f5a7a0c
Fast-forward
 setup.py                                          |   2 +
 swh/loader/mercurial/identify.py                  | 532 ++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py          | 187 ++++++++
 swh/loader/mercurial/tests/data/hello.json        |   1 +
 swh/loader/mercurial/tests/data/the-sandbox.json  |   1 +
 swh/loader/mercurial/tests/data/transplant.json   |   1 +
 swh/loader/mercurial/tests/dvcs_loader_checker.py | 132 ++++++
 swh/loader/mercurial/tests/test_identify.py       |  88 ++++
 swh/loader/mercurial/tests/test_loader.py         |  12 +
 9 files changed, 956 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/data/hello.json
 create mode 100644 swh/loader/mercurial/tests/data/the-sandbox.json
 create mode 100644 swh/loader/mercurial/tests/data/transplant.json
 create mode 100644 swh/loader/mercurial/tests/dvcs_loader_checker.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit f5a7a0cdf21fdf9823cff33895883638bb495415
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:22:07 2020 +0200

    Add `Hg20BundleLoader` tests from json files
    
    Generated json files with `swh/loader/mercurial/tests/data/build.py` for
    existing repositories and added them to `Hg20BundleLoader` tests.
    
    Introduce `DVCSLoaderChecker` as a standardized way to test repositories
    against json files.

commit 5b0256683349ad892ffed749e7dc45d8b768e5fb
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder

commit c3d959fe1cf5bee461ff7366697f907a578d9288
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/73/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/73/console

Harbormaster failed remote builds in B16644: Diff 15417!Oct 27 2020, 11:08 AM
acezar updated this revision to Diff 15439.Oct 27 2020, 3:58 PM

failing at arc again

swh-public-ci added a comment.Oct 27 2020, 3:59 PM

Build has FAILED

Patch application report for D4312 (id=15439)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..175611c
Fast-forward
 setup.py                                          |   2 +
 swh/loader/mercurial/identify.py                  | 532 ++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py          | 265 +++++++++++
 swh/loader/mercurial/tests/data/hello.json        |   1 +
 swh/loader/mercurial/tests/data/the-sandbox.json  |   1 +
 swh/loader/mercurial/tests/data/transplant.json   |   1 +
 swh/loader/mercurial/tests/dvcs_loader_checker.py | 139 ++++++
 swh/loader/mercurial/tests/test_identify.py       |  88 ++++
 swh/loader/mercurial/tests/test_loader.py         |  12 +
 9 files changed, 1041 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/data/hello.json
 create mode 100644 swh/loader/mercurial/tests/data/the-sandbox.json
 create mode 100644 swh/loader/mercurial/tests/data/transplant.json
 create mode 100644 swh/loader/mercurial/tests/dvcs_loader_checker.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit 175611cfa1d8cc15a5131a8db920193def3cd4fd
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:22:07 2020 +0200

    Add `Hg20BundleLoader` tests from json files
    
    Generated json files with `swh/loader/mercurial/tests/data/build.py` for
    existing repositories and added them to `Hg20BundleLoader` tests.
    
    Introduce `DVCSLoaderChecker` as a standardized way to test repositories
    against json files.

commit 838149f98d435fce7d809fbe1c324c4c42dc2ba7
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    This build script purpose is to create example repositories from bash scripts
    and extract assertion data from them into json files.
    
    Advantages:
    
        - the bash script documents the repository creation
        - automating creation allow easy repository update
        - automation extraction allow easier update of assertion data

commit c3d959fe1cf5bee461ff7366697f907a578d9288
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/78/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/78/console

Harbormaster failed remote builds in B16667: Diff 15439!Oct 27 2020, 3:59 PM
acezar updated this revision to Diff 15453.Oct 27 2020, 4:50 PM

Update

swh-public-ci added a comment.Oct 27 2020, 4:53 PM

Build has FAILED

Patch application report for D4312 (id=15453)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..c0d13d2
Fast-forward
 setup.py                                          |   2 +
 swh/loader/mercurial/identify.py                  | 532 ++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py          | 265 +++++++++++
 swh/loader/mercurial/tests/data/hello.json        |   1 +
 swh/loader/mercurial/tests/data/the-sandbox.json  |   1 +
 swh/loader/mercurial/tests/data/transplant.json   |   1 +
 swh/loader/mercurial/tests/dvcs_loader_checker.py | 144 ++++++
 swh/loader/mercurial/tests/test_identify.py       |  93 ++++
 swh/loader/mercurial/tests/test_loader.py         |  12 +
 9 files changed, 1051 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/data/hello.json
 create mode 100644 swh/loader/mercurial/tests/data/the-sandbox.json
 create mode 100644 swh/loader/mercurial/tests/data/transplant.json
 create mode 100644 swh/loader/mercurial/tests/dvcs_loader_checker.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit c0d13d25aeaf4658a10ef6c1330208332328205f
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:22:07 2020 +0200

    Add `Hg20BundleLoader` tests from json files
    
    Generated json files with `swh/loader/mercurial/tests/data/build.py` for
    existing repositories and added them to `Hg20BundleLoader` tests.
    
    Introduce `DVCSLoaderChecker` as a standardized way to test repositories
    against json files.

commit b4e053fdcdd4cd7cdaef915de79c89829186b067
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    This build script purpose is to create example repositories from bash scripts
    and extract assertion data from them into json files.
    
    Advantages:
    
        - the bash script documents the repository creation
        - automating creation allow easy repository update
        - automation extraction allow easier update of assertion data

commit e75b99287ba58bdbc6cb95703f11303c561a8f34
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/83/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/83/console

Harbormaster failed remote builds in B16681: Diff 15453!Oct 27 2020, 4:53 PM
acezar updated this revision to Diff 15786.Nov 10 2020, 2:53 PM

Update to make ci pass

swh-public-ci added a comment.Nov 10 2020, 2:55 PM

Build is green

Patch application report for D4312 (id=15786)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..e4eb5c2
Fast-forward
 requirements.txt                                  |   1 +
 setup.py                                          |   2 +
 swh/loader/mercurial/identify.py                  | 533 ++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py          | 265 +++++++++++
 swh/loader/mercurial/tests/data/hello.json        |   1 +
 swh/loader/mercurial/tests/data/the-sandbox.json  |   1 +
 swh/loader/mercurial/tests/data/transplant.json   |   1 +
 swh/loader/mercurial/tests/dvcs_loader_checker.py | 144 ++++++
 swh/loader/mercurial/tests/test_identify.py       |  93 ++++
 swh/loader/mercurial/tests/test_loader.py         |  12 +
 10 files changed, 1053 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/data/hello.json
 create mode 100644 swh/loader/mercurial/tests/data/the-sandbox.json
 create mode 100644 swh/loader/mercurial/tests/data/transplant.json
 create mode 100644 swh/loader/mercurial/tests/dvcs_loader_checker.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit e4eb5c21afc064812c692b6b36fbbbf1801b3552
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:22:07 2020 +0200

    Add `Hg20BundleLoader` tests from json files
    
    Generated json files with `swh/loader/mercurial/tests/data/build.py` for
    existing repositories and added them to `Hg20BundleLoader` tests.
    
    Introduce `DVCSLoaderChecker` as a standardized way to test repositories
    against json files.

commit ea7cc149edbee8778121fc37a099edaedc584f60
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    This build script purpose is to create example repositories from bash scripts
    and extract assertion data from them into json files.
    
    Advantages:
    
        - the bash script documents the repository creation
        - automating creation allow easy repository update
        - automation extraction allow easier update of assertion data

commit 00b709bae8c01854fa9eb37e2b5d9989fa68a17d
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/90/ for more details.

Harbormaster completed remote builds in B16997: Diff 15786.Nov 10 2020, 2:55 PM
acezar updated this revision to Diff 15862.Nov 12 2020, 11:52 PM

Change DVCSLoaderChecker to LoaderChecker.

As HgLoaderFromDisk will stop inherit from DVCSLoader intercepting
objects from the interface to make assertions as things progress is not feasible
anymore. LoaderChecker check hashes from storage instead.

swh-public-ci added a comment.Nov 12 2020, 11:53 PM

Build is green

Patch application report for D4312 (id=15862)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..bc32e12
Fast-forward
 requirements.txt                                 |   1 +
 setup.py                                         |   2 +
 swh/loader/mercurial/identify.py                 | 541 +++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py         | 265 +++++++++++
 swh/loader/mercurial/tests/data/hello.json       |   1 +
 swh/loader/mercurial/tests/data/the-sandbox.json |   1 +
 swh/loader/mercurial/tests/data/transplant.json  |   1 +
 swh/loader/mercurial/tests/loader_checker.py     |  74 ++++
 swh/loader/mercurial/tests/test_identify.py      |  74 ++++
 swh/loader/mercurial/tests/test_loader.py        |  12 +
 10 files changed, 972 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/data/hello.json
 create mode 100644 swh/loader/mercurial/tests/data/the-sandbox.json
 create mode 100644 swh/loader/mercurial/tests/data/transplant.json
 create mode 100644 swh/loader/mercurial/tests/loader_checker.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit bc32e1280cfd6a59df595cdcbcc2c2b51b3618aa
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:22:07 2020 +0200

    Add `Hg20BundleLoader` tests from json files
    
    Generated json files with `swh/loader/mercurial/tests/data/build.py` for
    existing repositories and added them to `Hg20BundleLoader` tests.
    
    Introduce `LoaderChecker` as a standardized way to test repositories
    against json files.

commit ff11f77f1b493bd1c8ed257e790ded8da276101c
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    This build script purpose is to create example repositories from bash scripts
    and extract assertion data from them into json files.
    
    Advantages:
    
        - the bash script documents the repository creation
        - automating creation allow easy repository update
        - automation extraction allow easier update of assertion data

commit a2e9cf16919a5f81a06f955a533a254a9b3c9689
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/93/ for more details.

Harbormaster completed remote builds in B17071: Diff 15862.Nov 12 2020, 11:53 PM
acezar edited the summary of this revision. (Show Details)Nov 12 2020, 11:56 PM
Closed by commit rDLDHGbc32e1280cfd: Add `Hg20BundleLoader` tests from json files (authored by acezar). · Explain WhyNov 27 2020, 5:29 PM
This revision was automatically updated to reflect the committed changes.
acezar added a commit: rDLDHGbc32e1280cfd: Add `Hg20BundleLoader` tests from json files.

Revision Contents
Changeset List

PathSize
2 lines
swh/
loader/
mercurial/
532 lines
tests/
data/
187 lines
1 line
1 line
1 line
132 lines
88 lines
12 lines

Diff 15407

View Options

setup.py

Show All 9 Lines
from swh.web.common.exc import BadInputExc from swh.web.common.exc import BadInputExc
from swh.web.common.origin_visits import get_origin_visits from swh.web.common.origin_visits import get_origin_visits
from swh.web.common.utils import reverse from swh.web.common.utils import reverse
from swh.web.api.apidoc import api_doc, format_docstring from swh.web.api.apidoc import api_doc, format_docstring
from swh.web.api.apiurls import api_route from swh.web.api.apiurls import api_route
from swh.web.api.views.utils import api_lookup from swh.web.api.views.utils import api_lookup
DOC_RETURN_ORIGIN = '''
:>json string origin_visits_url: link to in order to get information
about the visits for that origin
:>json string url: the origin canonical url
:>json string type: the type of software origin (deprecated value;
types are now associated to visits instead of origins)
:>json number id: the origin unique identifier (deprecated value;
you should only refer to origins based on their URL)
'''
DOC_RETURN_ORIGIN_ARRAY = \
DOC_RETURN_ORIGIN.replace(':>json', ':>jsonarr')
DOC_RETURN_ORIGIN_VISIT = '''
:>json string date: ISO representation of the visit date (in UTC)
:>json str origin: the origin canonical url
:>json string origin_url: link to get information about the origin
:>jsonarr string snapshot: the snapshot identifier of the visit
:>jsonarr string snapshot_url: link to
:http:get:`/api/1/snapshot/(snapshot_id)/` in order to get
information about the snapshot of the visit
:>json string status: status of the visit (either **full**,
**partial** or **ongoing**)
:>json number visit: the unique identifier of the visit
'''
DOC_RETURN_ORIGIN_VISIT_ARRAY = \
DOC_RETURN_ORIGIN_VISIT.replace(':>json', ':>jsonarr')
DOC_RETURN_ORIGIN_VISIT_ARRAY += '''
:>jsonarr number id: the unique identifier of the origin
:>jsonarr string origin_visit_url: link to
:http:get:`/api/1/origin/(origin_url)/visit/(visit_id)/`
in order to get information about the visit
'''
def _enrich_origin(origin): def _enrich_origin(origin):
if 'id' in origin: if 'id' in origin:
o = origin.copy() o = origin.copy()
o['origin_visits_url'] = reverse( o['origin_visits_url'] = reverse(
'api-1-origin-visits', url_args={'origin_id': origin['id']}) 'api-1-origin-visits', url_args={'origin_url': origin['url']})
return o return o
return origin return origin
def _enrich_origin_visit(origin_visit, *, def _enrich_origin_visit(origin_visit, *,
with_origin_url, with_origin_visit_url): with_origin_link, with_origin_visit_link):
ov = origin_visit.copy() ov = origin_visit.copy()
if with_origin_url: if with_origin_link:
ov['origin_url'] = reverse('api-1-origin', ov['origin_url'] = reverse('api-1-origin',
url_args={'origin_id': ov['origin']}) url_args={'origin_url': ov['origin']})
if with_origin_visit_url: if with_origin_visit_link:
ov['origin_visit_url'] = reverse('api-1-origin-visit', ov['origin_visit_url'] = reverse('api-1-origin-visit',
url_args={'origin_id': ov['origin'], url_args={'origin_url': ov['origin'],
'visit_id': ov['visit']}) 'visit_id': ov['visit']})
snapshot = ov['snapshot'] snapshot = ov['snapshot']
if snapshot: if snapshot:
ov['snapshot_url'] = reverse('api-1-snapshot', ov['snapshot_url'] = reverse('api-1-snapshot',
url_args={'snapshot_id': snapshot}) url_args={'snapshot_id': snapshot})
else: else:
ov['snapshot_url'] = None ov['snapshot_url'] = None
return ov return ov
@api_route(r'/origins/', 'api-1-origins') @api_route(r'/origins/', 'api-1-origins')
@api_doc('/origins/', noargs=True) @api_doc('/origins/', noargs=True)
@format_docstring() @format_docstring(return_origin_array=DOC_RETURN_ORIGIN_ARRAY)
def api_origins(request): def api_origins(request):
""" """
.. http:get:: /api/1/origins/ .. http:get:: /api/1/origins/
Get list of archived software origins. Get list of archived software origins.
Origins are sorted by ids before returning them. Origins are sorted by ids before returning them.
:query int origin_from: The first origin id that will be included :query int origin_from: The first origin id that will be included
in returned results (default to 1) in returned results (default to 1)
:query int origin_count: The maximum number of origins to return :query int origin_count: The maximum number of origins to return
(default to 100, can not exceed 10000) (default to 100, can not exceed 10000)
:>jsonarr number id: the origin unique identifier {return_origin_array}
:>jsonarr string origin_visits_url: link to in order to get information
about the visits for that origin
:>jsonarr string type: the type of software origin (possible values
are ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp``
or ``deposit``)
:>jsonarr string url: the origin canonical url
{common_headers} {common_headers}
{resheader_link} {resheader_link}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
Show All 15 Lines if len(results) > origin_count:
origin_from = results.pop()['id'] origin_from = results.pop()['id']
response['headers']['link-next'] = reverse( response['headers']['link-next'] = reverse(
'api-1-origins', 'api-1-origins',
query_params={'origin_from': origin_from, query_params={'origin_from': origin_from,
'origin_count': origin_count}) 'origin_count': origin_count})
return response return response
@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/', @api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/',
'api-1-origin') 'api-1-origin')
@api_route(r'/origin/(?P<origin_url>.+)/get/', 'api-1-origin')
@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
@api_doc('/origin/') @api_doc('/origin/')
@format_docstring() @format_docstring(return_origin=DOC_RETURN_ORIGIN)
def api_origin(request, origin_id=None, origin_type=None, origin_url=None): def api_origin(request, origin_id=None, origin_type=None, origin_url=None):
""" """
vlorentzAuthorUnsubmitted
Done
Inline Actions

Note the /get/, which is needed to disambiguate the URL.

Not very pretty, but that's the only solution I got.

vlorentz: Note the `/get/`, which is needed to disambiguate the URL. Not very pretty, but that's the…
anlambertUnsubmitted
Not Done
Inline Actions

Indeed, the order of url declarations in Django matters for disambiguation.

I managed to resolve the issue through the following diff

diff --git a/swh/web/api/urls.py b/swh/web/api/urls.py
index afbcaf44..3c3b99cd 100644
--- a/swh/web/api/urls.py
+++ b/swh/web/api/urls.py
@@ -6,8 +6,8 @@
 import swh.web.api.views.content # noqa
 import swh.web.api.views.directory # noqa
 import swh.web.api.views.identifiers # noqa
-import swh.web.api.views.origin # noqa
 import swh.web.api.views.origin_save # noqa
+import swh.web.api.views.origin # noqa
 import swh.web.api.views.person # noqa
 import swh.web.api.views.release # noqa
 import swh.web.api.views.revision # noqa
diff --git a/swh/web/api/views/origin.py b/swh/web/api/views/origin.py
index 0c70e485..cb54f0ca 100644
--- a/swh/web/api/views/origin.py
+++ b/swh/web/api/views/origin.py
@@ -129,103 +129,6 @@ def api_origins(request):
     return response
 
 
-@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/',
-           'api-1-origin')
-@api_route(r'/origin/(?P<origin_url>.+)/get/', 'api-1-origin')
-@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
-@api_doc('/origin/')
-@format_docstring(return_origin=DOC_RETURN_ORIGIN)
-def api_origin(request, origin_id=None, origin_type=None, origin_url=None):
-    """
-    .. http:get:: /api/1/origin/(origin_url)/get/
-
-        Get information about a software origin.
-
-        :param string origin_url: the origin url
-
-        {return_origin}
-
-        {common_headers}
-
-        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
-            :http:method:`options`
-
-        :statuscode 200: no error
-        :statuscode 404: requested origin can not be found in the archive
-
-        **Example:**
-
-        .. parsed-literal::
-
-            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
-
-    .. http:get:: /api/1/origin/(origin_id)/
-
-        Get information about a software origin.
-
-        :param int origin_id: a software origin identifier
-
-        {return_origin}
-
-        {common_headers}
-
-        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
-            :http:method:`options`
-
-        :statuscode 200: no error
-        :statuscode 404: requested origin can not be found in the archive
-
-        **Example:**
-
-        .. parsed-literal::
-
-            :swh_web_api:`origin/1/`
-
-    .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/
-
-        Get information about a software origin.
-
-        .. warning::
-
-            This endpoint is deprecated. You should use
-            :http:get:`/api/1/origin/(origin_url)/get/` instead.
-
-        :param string origin_type: the origin type (possible values are
-            ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
-            ``deposit``)
-        :param string origin_url: the origin url
-
-        {return_origin}
-
-        {common_headers}
-
-        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
-            :http:method:`options`
-
-        :statuscode 200: no error
-        :statuscode 404: requested origin can not be found in the archive
-
-        **Example:**
-
-        .. parsed-literal::
-
-            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
-    """
-    ori_dict = {
-        'id': int(origin_id) if origin_id else None,
-        'type': origin_type,
-        'url': origin_url
-    }
-    ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]}
-    error_msg = 'Origin %s not found.' % \
-        (ori_dict.get('id') or ori_dict['url'])
-
-    return api_lookup(
-        service.lookup_origin, ori_dict,
-        notfound_msg=error_msg,
-        enrich_fn=_enrich_origin)
-
-
 @api_route(r'/origin/search/(?P<url_pattern>.+)/',
            'api-1-origin-search')
 @api_doc('/origin/search/')
@@ -569,3 +472,100 @@ def api_origin_intrinsic_metadata(request, origin_type, origin_url):
         service.lookup_origin_intrinsic_metadata, ori_dict,
         notfound_msg=error_msg,
         enrich_fn=_enrich_origin)
+
+
+@api_route(r'/origin/(?P<origin_url>.+)/', 'api-1-origin')
+@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
+@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/',
+           'api-1-origin')
+@api_doc('/origin/')
+@format_docstring(return_origin=DOC_RETURN_ORIGIN)
+def api_origin(request, origin_id=None, origin_type=None, origin_url=None):
+    """
+    .. http:get:: /api/1/origin/(origin_url)/
+
+        Get information about a software origin.
+
+        :param string origin_url: the origin url
+
+        {return_origin}
+
+        {common_headers}
+
+        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
+            :http:method:`options`
+
+        :statuscode 200: no error
+        :statuscode 404: requested origin can not be found in the archive
+
+        **Example:**
+
+        .. parsed-literal::
+
+            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
+
+    .. http:get:: /api/1/origin/(origin_id)/
+
+        Get information about a software origin.
+
+        :param int origin_id: a software origin identifier
+
+        {return_origin}
+
+        {common_headers}
+
+        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
+            :http:method:`options`
+
+        :statuscode 200: no error
+        :statuscode 404: requested origin can not be found in the archive
+
+        **Example:**
+
+        .. parsed-literal::
+
+            :swh_web_api:`origin/1/`
+
+    .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/
+
+        Get information about a software origin.
+
+        .. warning::
+
+            This endpoint is deprecated. You should use
+            :http:get:`/api/1/origin/(origin_url)/get/` instead.
+
+        :param string origin_type: the origin type (possible values are
+            ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
+            ``deposit``)
+        :param string origin_url: the origin url
+
+        {return_origin}
+
+        {common_headers}
+
+        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
+            :http:method:`options`
+
+        :statuscode 200: no error
+        :statuscode 404: requested origin can not be found in the archive
+
+        **Example:**
+
+        .. parsed-literal::
+
+            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
+    """
+    ori_dict = {
+        'id': int(origin_id) if origin_id else None,
+        'type': origin_type,
+        'url': origin_url
+    }
+    ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]}
+    error_msg = 'Origin %s not found.' % \
+        (ori_dict.get('id') or ori_dict['url'])
+
+    return api_lookup(
+        service.lookup_origin, ori_dict,
+        notfound_msg=error_msg,
+        enrich_fn=_enrich_origin)
\ No newline at end of file
anlambert: Indeed, the order of url declarations in Django matters for disambiguation. I managed to…
.. http:get:: /api/1/origin/(origin_url)/get/
Get information about a software origin.
:param string origin_url: the origin url
{return_origin}
{common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options`
:statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive
**Example:**
.. parsed-literal::
:swh_web_api:`origin/git/url/https://github.com/python/cpython/`
.. http:get:: /api/1/origin/(origin_id)/ .. http:get:: /api/1/origin/(origin_id)/
Get information about a software origin. Get information about a software origin.
:param int origin_id: a software origin identifier :param int origin_id: a software origin identifier
:>json number id: the origin unique identifier {return_origin}
:>json string origin_visits_url: link to in order to get information
about the visits for that origin
:>json string type: the type of software origin (possible values are
``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
``deposit``)
:>json string url: the origin canonical url
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive :statuscode 404: requested origin can not be found in the archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/1/` :swh_web_api:`origin/1/`
.. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/ .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/
Get information about a software origin. Get information about a software origin.
anlambertUnsubmitted
Not Done
Inline Actions

The warning directive is currently not parsed by the apidoc module.

This must be implemented in order to add the deprecation notice to the endpoint descriptions.

anlambert: The warning directive is currently not parsed by the `apidoc` module. This must be implemented…
.. warning::
This endpoint is deprecated. You should use
:http:get:`/api/1/origin/(origin_url)/get/` instead.
:param string origin_type: the origin type (possible values are :param string origin_type: the origin type (possible values are
``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
``deposit``) ``deposit``)
:param string origin_url: the origin url :param string origin_url: the origin url
:>json number id: the origin unique identifier {return_origin}
:>json string origin_visits_url: link to in order to get information
about the visits for that origin
:>json string type: the type of software origin
:>json string url: the origin canonical url
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive :statuscode 404: requested origin can not be found in the archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/git/url/https://github.com/python/cpython/` :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
""" """
ori_dict = { ori_dict = {
'id': int(origin_id) if origin_id else None, 'id': int(origin_id) if origin_id else None,
'type': origin_type, 'type': origin_type,
'url': origin_url 'url': origin_url
} }
ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]} ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]}
if 'id' in ori_dict: error_msg = 'Origin %s not found.' % \
error_msg = 'Origin with id %s not found.' % ori_dict['id'] (ori_dict.get('id') or ori_dict['url'])
else:
error_msg = 'Origin with type %s and URL %s not found' % (
ori_dict['type'], ori_dict['url'])
return api_lookup( return api_lookup(
service.lookup_origin, ori_dict, service.lookup_origin, ori_dict,
notfound_msg=error_msg, notfound_msg=error_msg,
enrich_fn=_enrich_origin) enrich_fn=_enrich_origin)
@api_route(r'/origin/search/(?P<url_pattern>.+)/', @api_route(r'/origin/search/(?P<url_pattern>.+)/',
'api-1-origin-search') 'api-1-origin-search')
@api_doc('/origin/search/') @api_doc('/origin/search/')
@format_docstring() @format_docstring(return_origin_array=DOC_RETURN_ORIGIN_ARRAY)
def api_origin_search(request, url_pattern): def api_origin_search(request, url_pattern):
""" """
.. http:get:: /api/1/origin/search/(url_pattern)/ .. http:get:: /api/1/origin/search/(url_pattern)/
Search for software origins whose urls contain a provided string Search for software origins whose urls contain a provided string
pattern or match a provided regular expression. pattern or match a provided regular expression.
The search is performed in a case insensitive way. The search is performed in a case insensitive way.
:param string url_pattern: a string pattern or a regular expression :param string url_pattern: a string pattern or a regular expression
:query int offset: the number of found origins to skip before returning :query int offset: the number of found origins to skip before returning
results results
:query int limit: the maximum number of found origins to return :query int limit: the maximum number of found origins to return
:query boolean regexp: if true, consider provided pattern as a regular :query boolean regexp: if true, consider provided pattern as a regular
expression and search origins whose urls match it expression and search origins whose urls match it
:query boolean with_visit: if true, only return origins with at least :query boolean with_visit: if true, only return origins with at least
one visit by Software heritage one visit by Software heritage
:>jsonarr number id: the origin unique identifier {return_origin_array}
:>jsonarr string origin_visits_url: link to in order to get information
about the visits for that origin
:>jsonarr string type: the type of software origin
:>jsonarr string url: the origin canonical url
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
Show All 31 Linesdef api_origin_search(request, url_pattern):
}) })
return result return result
@api_route(r'/origin/metadata-search/', @api_route(r'/origin/metadata-search/',
'api-1-origin-metadata-search') 'api-1-origin-metadata-search')
@api_doc('/origin/metadata-search/', noargs=True, need_params=True) @api_doc('/origin/metadata-search/', noargs=True, need_params=True)
@format_docstring() @format_docstring(return_origin_array=DOC_RETURN_ORIGIN_ARRAY)
def api_origin_metadata_search(request): def api_origin_metadata_search(request):
""" """
.. http:get:: /api/1/origin/metadata-search/ .. http:get:: /api/1/origin/metadata-search/
Search for software origins whose metadata (expressed as a Search for software origins whose metadata (expressed as a
JSON-LD/CodeMeta dictionary) match the provided criteria. JSON-LD/CodeMeta dictionary) match the provided criteria.
For now, only full-text search on this dictionary is supported. For now, only full-text search on this dictionary is supported.
:query str fulltext: a string that will be matched against origin :query str fulltext: a string that will be matched against origin
metadata; results are ranked and ordered starting with the best metadata; results are ranked and ordered starting with the best
ones. ones.
:query int limit: the maximum number of found origins to return :query int limit: the maximum number of found origins to return
(bounded to 100) (bounded to 100)
:>jsonarr number origin_id: the origin unique identifier {return_origin_array}
:>jsonarr dict metadata: metadata of the origin (as a
JSON-LD/CodeMeta dictionary)
:>jsonarr string from_revision: the revision used to extract these
metadata (the current HEAD or one of the former HEADs)
:>jsonarr dict tool: the tool used to extract these metadata
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
Show All 12 Linesdef api_origin_metadata_search(request):
results = api_lookup(service.search_origin_metadata, fulltext, limit) results = api_lookup(service.search_origin_metadata, fulltext, limit)
return { return {
'results': results, 'results': results,
} }
@api_route(r'/origin/(?P<origin_url>.*)/visits/', 'api-1-origin-visits')
@api_route(r'/origin/(?P<origin_id>[0-9]+)/visits/', 'api-1-origin-visits') @api_route(r'/origin/(?P<origin_id>[0-9]+)/visits/', 'api-1-origin-visits')
@api_doc('/origin/visits/') @api_doc('/origin/visits/')
@format_docstring() @format_docstring(
def api_origin_visits(request, origin_id): return_origin_visit_array=DOC_RETURN_ORIGIN_VISIT_ARRAY)
def api_origin_visits(request, origin_id=None, origin_url=None):
""" """
.. http:get:: /api/1/origin/(origin_url)/visits/
Get information about all visits of a software origin.
Visits are returned sorted in descending order according
to their date.
:param str origin_url: a software origin URL
:query int per_page: specify the number of visits to list, for
pagination purposes
:query int last_visit: visit to start listing from, for pagination
purposes
{common_headers}
{resheader_link}
{return_origin_visit_array}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options`
:statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive
**Example:**
.. parsed-literal::
:swh_web_api:`origin/https://github.com/hylang/hy/visits/`
.. http:get:: /api/1/origin/(origin_id)/visits/ .. http:get:: /api/1/origin/(origin_id)/visits/
Get information about all visits of a software origin. Get information about all visits of a software origin.
Visits are returned sorted in descending order according Visits are returned sorted in descending order according
to their date. to their date.
:param int origin_id: a software origin identifier :param int origin_id: a software origin identifier
:query int per_page: specify the number of visits to list, for :query int per_page: specify the number of visits to list, for
pagination purposes pagination purposes
:query int last_visit: visit to start listing from, for pagination :query int last_visit: visit to start listing from, for pagination
purposes purposes
{common_headers} {common_headers}
{resheader_link} {resheader_link}
:>jsonarr string date: ISO representation of the visit date (in UTC) {return_origin_visit_array}
:>jsonarr number id: the unique identifier of the origin
:>jsonarr string origin_visit_url: link to
:http:get:`/api/1/origin/(origin_id)/visit/(visit_id)/` in order to
get information about the visit
:>jsonarr string snapshot: the snapshot identifier of the visit
:>jsonarr string snapshot_url: link to
:http:get:`/api/1/snapshot/(snapshot_id)/` in order to get
information about the snapshot of the visit
:>jsonarr string status: status of the visit (either **full**,
**partial** or **ongoing**)
:>jsonarr number visit: the unique identifier of the visit
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive :statuscode 404: requested origin can not be found in the archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/1/visits/` :swh_web_api:`origin/1/visits/`
""" """
result = {} result = {}
origin_id = int(origin_id) if origin_url:
origin_query = {'url': origin_url}
notfound_msg = 'No origin {} found'.format(origin_url)
url_args_next = {'origin_url': origin_url}
else:
origin_query = {'id': int(origin_id)}
notfound_msg = 'No origin {} found'.format(origin_id)
url_args_next = {'origin_id': origin_id}
per_page = int(request.query_params.get('per_page', '10')) per_page = int(request.query_params.get('per_page', '10'))
last_visit = request.query_params.get('last_visit') last_visit = request.query_params.get('last_visit')
if last_visit: if last_visit:
last_visit = int(last_visit) last_visit = int(last_visit)
def _lookup_origin_visits( def _lookup_origin_visits(
origin_id, last_visit=last_visit, per_page=per_page): origin_query, last_visit=last_visit, per_page=per_page):
all_visits = get_origin_visits({'id': origin_id}) all_visits = get_origin_visits(origin_query)
all_visits.reverse() all_visits.reverse()
visits = [] visits = []
if not last_visit: if not last_visit:
visits = all_visits[:per_page] visits = all_visits[:per_page]
else: else:
for i, v in enumerate(all_visits): for i, v in enumerate(all_visits):
if v['visit'] == last_visit: if v['visit'] == last_visit:
visits = all_visits[i+1:i+1+per_page] visits = all_visits[i+1:i+1+per_page]
break break
for v in visits: for v in visits:
yield v yield v
results = api_lookup(_lookup_origin_visits, origin_id, results = api_lookup(_lookup_origin_visits, origin_query,
notfound_msg='No origin {} found'.format(origin_id), notfound_msg=notfound_msg,
enrich_fn=partial(_enrich_origin_visit, enrich_fn=partial(_enrich_origin_visit,
with_origin_url=False, with_origin_link=False,
with_origin_visit_url=True)) with_origin_visit_link=True))
if results: if results:
nb_results = len(results) nb_results = len(results)
if nb_results == per_page: if nb_results == per_page:
new_last_visit = results[-1]['visit'] new_last_visit = results[-1]['visit']
query_params = {} query_params = {}
query_params['last_visit'] = new_last_visit query_params['last_visit'] = new_last_visit
if request.query_params.get('per_page'): if request.query_params.get('per_page'):
query_params['per_page'] = per_page query_params['per_page'] = per_page
result['headers'] = { result['headers'] = {
'link-next': reverse('api-1-origin-visits', 'link-next': reverse('api-1-origin-visits',
url_args={'origin_id': origin_id}, url_args=url_args_next,
query_params=query_params) query_params=query_params)
} }
result.update({ result.update({
'results': results 'results': results
}) })
return result return result
@api_route(r'/origin/(?P<origin_url>.*)/visit/(?P<visit_id>[0-9]+)/',
'api-1-origin-visit')
@api_route(r'/origin/(?P<origin_id>[0-9]+)/visit/(?P<visit_id>[0-9]+)/', @api_route(r'/origin/(?P<origin_id>[0-9]+)/visit/(?P<visit_id>[0-9]+)/',
'api-1-origin-visit') 'api-1-origin-visit')
@api_doc('/origin/visit/') @api_doc('/origin/visit/')
@format_docstring() @format_docstring(return_origin_visit=DOC_RETURN_ORIGIN_VISIT)
def api_origin_visit(request, origin_id, visit_id): def api_origin_visit(request, visit_id, origin_url=None, origin_id=None):
""" """
.. http:get:: /api/1/origin/(origin_url)/visit/(visit_id)/
Get information about a specific visit of a software origin.
:param str origin_url: a software origin URL
:param int visit_id: a visit identifier
{common_headers}
{return_origin_visit}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options`
:statuscode 200: no error
:statuscode 404: requested origin or visit can not be found in the
archive
**Example:**
.. parsed-literal::
:swh_web_api:`origin/https://github.com/hylang/hy/visit/1/`
.. http:get:: /api/1/origin/(origin_id)/visit/(visit_id)/ .. http:get:: /api/1/origin/(origin_id)/visit/(visit_id)/
Get information about a specific visit of a software origin. Get information about a specific visit of a software origin.
:param int origin_id: a software origin identifier :param int origin_id: a software origin identifier
:param int visit_id: a visit identifier :param int visit_id: a visit identifier
{common_headers} {common_headers}
:>json string date: ISO representation of the visit date (in UTC) {return_origin_visit}
:>json number origin: the origin unique identifier
:>json string origin_url: link to get information about the origin
:>jsonarr string snapshot: the snapshot identifier of the visit
:>jsonarr string snapshot_url: link to
:http:get:`/api/1/snapshot/(snapshot_id)/` in order to get
information about the snapshot of the visit
:>json string status: status of the visit (either **full**,
**partial** or **ongoing**)
:>json number visit: the unique identifier of the visit
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin or visit can not be found in the :statuscode 404: requested origin or visit can not be found in the
archive archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/1500/visit/1/` :swh_web_api:`origin/1500/visit/1/`
""" """
if not origin_url:
origin_url = service.lookup_origin({'id': int(origin_id)})['url']
return api_lookup( return api_lookup(
service.lookup_origin_visit, int(origin_id), int(visit_id), service.lookup_origin_visit, origin_url, int(visit_id),
notfound_msg=('No visit {} for origin {} found' notfound_msg=('No visit {} for origin {} found'
.format(visit_id, origin_id)), .format(visit_id, origin_url)),
enrich_fn=partial(_enrich_origin_visit, enrich_fn=partial(_enrich_origin_visit,
with_origin_url=True, with_origin_link=True,
with_origin_visit_url=False)) with_origin_visit_link=False))
@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)' @api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)'
'/intrinsic-metadata', 'api-origin-intrinsic-metadata') '/intrinsic-metadata', 'api-origin-intrinsic-metadata')
@api_doc('/origin/intrinsic-metadata/') @api_doc('/origin/intrinsic-metadata/')
@format_docstring() @format_docstring()
def api_origin_intrinsic_metadata(request, origin_type, origin_url): def api_origin_intrinsic_metadata(request, origin_type, origin_url):
""" """
Show All 20 Lines .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/intrinsic-metadata
:swh_web_api:`origin/git/url/https://github.com/python/cpython/intrinsic-metadata` :swh_web_api:`origin/git/url/https://github.com/python/cpython/intrinsic-metadata`
""" # noqa """ # noqa
ori_dict = { ori_dict = {
'type': origin_type, 'type': origin_type,
'url': origin_url 'url': origin_url
} }
error_msg = 'Origin with type %s and URL %s not found' % ( error_msg = 'Origin with URL %s not found' % ori_dict['url']
ori_dict['type'], ori_dict['url'])
return api_lookup( return api_lookup(
service.lookup_origin_intrinsic_metadata, ori_dict, service.lookup_origin_intrinsic_metadata, ori_dict,
notfound_msg=error_msg, notfound_msg=error_msg,
enrich_fn=_enrich_origin) enrich_fn=_enrich_origin)
View Options

swh/loader/mercurial/identify.py

Show All 9 Lines
from swh.web.common.exc import BadInputExc from swh.web.common.exc import BadInputExc
from swh.web.common.origin_visits import get_origin_visits from swh.web.common.origin_visits import get_origin_visits
from swh.web.common.utils import reverse from swh.web.common.utils import reverse
from swh.web.api.apidoc import api_doc, format_docstring from swh.web.api.apidoc import api_doc, format_docstring
from swh.web.api.apiurls import api_route from swh.web.api.apiurls import api_route
from swh.web.api.views.utils import api_lookup from swh.web.api.views.utils import api_lookup
DOC_RETURN_ORIGIN = '''
:>json string origin_visits_url: link to in order to get information
about the visits for that origin
:>json string url: the origin canonical url
:>json string type: the type of software origin (deprecated value;
types are now associated to visits instead of origins)
:>json number id: the origin unique identifier (deprecated value;
you should only refer to origins based on their URL)
'''
DOC_RETURN_ORIGIN_ARRAY = \
DOC_RETURN_ORIGIN.replace(':>json', ':>jsonarr')
DOC_RETURN_ORIGIN_VISIT = '''
:>json string date: ISO representation of the visit date (in UTC)
:>json str origin: the origin canonical url
:>json string origin_url: link to get information about the origin
:>jsonarr string snapshot: the snapshot identifier of the visit
:>jsonarr string snapshot_url: link to
:http:get:`/api/1/snapshot/(snapshot_id)/` in order to get
information about the snapshot of the visit
:>json string status: status of the visit (either **full**,
**partial** or **ongoing**)
:>json number visit: the unique identifier of the visit
'''
DOC_RETURN_ORIGIN_VISIT_ARRAY = \
DOC_RETURN_ORIGIN_VISIT.replace(':>json', ':>jsonarr')
DOC_RETURN_ORIGIN_VISIT_ARRAY += '''
:>jsonarr number id: the unique identifier of the origin
:>jsonarr string origin_visit_url: link to
:http:get:`/api/1/origin/(origin_url)/visit/(visit_id)/`
in order to get information about the visit
'''
def _enrich_origin(origin): def _enrich_origin(origin):
if 'id' in origin: if 'id' in origin:
o = origin.copy() o = origin.copy()
o['origin_visits_url'] = reverse( o['origin_visits_url'] = reverse(
'api-1-origin-visits', url_args={'origin_id': origin['id']}) 'api-1-origin-visits', url_args={'origin_url': origin['url']})
return o return o
return origin return origin
def _enrich_origin_visit(origin_visit, *, def _enrich_origin_visit(origin_visit, *,
with_origin_url, with_origin_visit_url): with_origin_link, with_origin_visit_link):
ov = origin_visit.copy() ov = origin_visit.copy()
if with_origin_url: if with_origin_link:
ov['origin_url'] = reverse('api-1-origin', ov['origin_url'] = reverse('api-1-origin',
url_args={'origin_id': ov['origin']}) url_args={'origin_url': ov['origin']})
if with_origin_visit_url: if with_origin_visit_link:
ov['origin_visit_url'] = reverse('api-1-origin-visit', ov['origin_visit_url'] = reverse('api-1-origin-visit',
url_args={'origin_id': ov['origin'], url_args={'origin_url': ov['origin'],
'visit_id': ov['visit']}) 'visit_id': ov['visit']})
snapshot = ov['snapshot'] snapshot = ov['snapshot']
if snapshot: if snapshot:
ov['snapshot_url'] = reverse('api-1-snapshot', ov['snapshot_url'] = reverse('api-1-snapshot',
url_args={'snapshot_id': snapshot}) url_args={'snapshot_id': snapshot})
else: else:
ov['snapshot_url'] = None ov['snapshot_url'] = None
return ov return ov
@api_route(r'/origins/', 'api-1-origins') @api_route(r'/origins/', 'api-1-origins')
@api_doc('/origins/', noargs=True) @api_doc('/origins/', noargs=True)
@format_docstring() @format_docstring(return_origin_array=DOC_RETURN_ORIGIN_ARRAY)
def api_origins(request): def api_origins(request):
""" """
.. http:get:: /api/1/origins/ .. http:get:: /api/1/origins/
Get list of archived software origins. Get list of archived software origins.
Origins are sorted by ids before returning them. Origins are sorted by ids before returning them.
:query int origin_from: The first origin id that will be included :query int origin_from: The first origin id that will be included
in returned results (default to 1) in returned results (default to 1)
:query int origin_count: The maximum number of origins to return :query int origin_count: The maximum number of origins to return
(default to 100, can not exceed 10000) (default to 100, can not exceed 10000)
:>jsonarr number id: the origin unique identifier {return_origin_array}
:>jsonarr string origin_visits_url: link to in order to get information
about the visits for that origin
:>jsonarr string type: the type of software origin (possible values
are ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp``
or ``deposit``)
:>jsonarr string url: the origin canonical url
{common_headers} {common_headers}
{resheader_link} {resheader_link}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
Show All 15 Lines if len(results) > origin_count:
origin_from = results.pop()['id'] origin_from = results.pop()['id']
response['headers']['link-next'] = reverse( response['headers']['link-next'] = reverse(
'api-1-origins', 'api-1-origins',
query_params={'origin_from': origin_from, query_params={'origin_from': origin_from,
'origin_count': origin_count}) 'origin_count': origin_count})
return response return response
@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/', @api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/',
'api-1-origin') 'api-1-origin')
@api_route(r'/origin/(?P<origin_url>.+)/get/', 'api-1-origin')
@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
@api_doc('/origin/') @api_doc('/origin/')
@format_docstring() @format_docstring(return_origin=DOC_RETURN_ORIGIN)
def api_origin(request, origin_id=None, origin_type=None, origin_url=None): def api_origin(request, origin_id=None, origin_type=None, origin_url=None):
""" """
vlorentzAuthorUnsubmitted
Done
Inline Actions

Note the /get/, which is needed to disambiguate the URL.

Not very pretty, but that's the only solution I got.

vlorentz: Note the `/get/`, which is needed to disambiguate the URL. Not very pretty, but that's the…
anlambertUnsubmitted
Not Done
Inline Actions

Indeed, the order of url declarations in Django matters for disambiguation.

I managed to resolve the issue through the following diff

diff --git a/swh/web/api/urls.py b/swh/web/api/urls.py
index afbcaf44..3c3b99cd 100644
--- a/swh/web/api/urls.py
+++ b/swh/web/api/urls.py
@@ -6,8 +6,8 @@
 import swh.web.api.views.content # noqa
 import swh.web.api.views.directory # noqa
 import swh.web.api.views.identifiers # noqa
-import swh.web.api.views.origin # noqa
 import swh.web.api.views.origin_save # noqa
+import swh.web.api.views.origin # noqa
 import swh.web.api.views.person # noqa
 import swh.web.api.views.release # noqa
 import swh.web.api.views.revision # noqa
diff --git a/swh/web/api/views/origin.py b/swh/web/api/views/origin.py
index 0c70e485..cb54f0ca 100644
--- a/swh/web/api/views/origin.py
+++ b/swh/web/api/views/origin.py
@@ -129,103 +129,6 @@ def api_origins(request):
     return response
 
 
-@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/',
-           'api-1-origin')
-@api_route(r'/origin/(?P<origin_url>.+)/get/', 'api-1-origin')
-@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
-@api_doc('/origin/')
-@format_docstring(return_origin=DOC_RETURN_ORIGIN)
-def api_origin(request, origin_id=None, origin_type=None, origin_url=None):
-    """
-    .. http:get:: /api/1/origin/(origin_url)/get/
-
-        Get information about a software origin.
-
-        :param string origin_url: the origin url
-
-        {return_origin}
-
-        {common_headers}
-
-        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
-            :http:method:`options`
-
-        :statuscode 200: no error
-        :statuscode 404: requested origin can not be found in the archive
-
-        **Example:**
-
-        .. parsed-literal::
-
-            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
-
-    .. http:get:: /api/1/origin/(origin_id)/
-
-        Get information about a software origin.
-
-        :param int origin_id: a software origin identifier
-
-        {return_origin}
-
-        {common_headers}
-
-        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
-            :http:method:`options`
-
-        :statuscode 200: no error
-        :statuscode 404: requested origin can not be found in the archive
-
-        **Example:**
-
-        .. parsed-literal::
-
-            :swh_web_api:`origin/1/`
-
-    .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/
-
-        Get information about a software origin.
-
-        .. warning::
-
-            This endpoint is deprecated. You should use
-            :http:get:`/api/1/origin/(origin_url)/get/` instead.
-
-        :param string origin_type: the origin type (possible values are
-            ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
-            ``deposit``)
-        :param string origin_url: the origin url
-
-        {return_origin}
-
-        {common_headers}
-
-        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
-            :http:method:`options`
-
-        :statuscode 200: no error
-        :statuscode 404: requested origin can not be found in the archive
-
-        **Example:**
-
-        .. parsed-literal::
-
-            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
-    """
-    ori_dict = {
-        'id': int(origin_id) if origin_id else None,
-        'type': origin_type,
-        'url': origin_url
-    }
-    ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]}
-    error_msg = 'Origin %s not found.' % \
-        (ori_dict.get('id') or ori_dict['url'])
-
-    return api_lookup(
-        service.lookup_origin, ori_dict,
-        notfound_msg=error_msg,
-        enrich_fn=_enrich_origin)
-
-
 @api_route(r'/origin/search/(?P<url_pattern>.+)/',
            'api-1-origin-search')
 @api_doc('/origin/search/')
@@ -569,3 +472,100 @@ def api_origin_intrinsic_metadata(request, origin_type, origin_url):
         service.lookup_origin_intrinsic_metadata, ori_dict,
         notfound_msg=error_msg,
         enrich_fn=_enrich_origin)
+
+
+@api_route(r'/origin/(?P<origin_url>.+)/', 'api-1-origin')
+@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
+@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/',
+           'api-1-origin')
+@api_doc('/origin/')
+@format_docstring(return_origin=DOC_RETURN_ORIGIN)
+def api_origin(request, origin_id=None, origin_type=None, origin_url=None):
+    """
+    .. http:get:: /api/1/origin/(origin_url)/
+
+        Get information about a software origin.
+
+        :param string origin_url: the origin url
+
+        {return_origin}
+
+        {common_headers}
+
+        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
+            :http:method:`options`
+
+        :statuscode 200: no error
+        :statuscode 404: requested origin can not be found in the archive
+
+        **Example:**
+
+        .. parsed-literal::
+
+            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
+
+    .. http:get:: /api/1/origin/(origin_id)/
+
+        Get information about a software origin.
+
+        :param int origin_id: a software origin identifier
+
+        {return_origin}
+
+        {common_headers}
+
+        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
+            :http:method:`options`
+
+        :statuscode 200: no error
+        :statuscode 404: requested origin can not be found in the archive
+
+        **Example:**
+
+        .. parsed-literal::
+
+            :swh_web_api:`origin/1/`
+
+    .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/
+
+        Get information about a software origin.
+
+        .. warning::
+
+            This endpoint is deprecated. You should use
+            :http:get:`/api/1/origin/(origin_url)/get/` instead.
+
+        :param string origin_type: the origin type (possible values are
+            ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
+            ``deposit``)
+        :param string origin_url: the origin url
+
+        {return_origin}
+
+        {common_headers}
+
+        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
+            :http:method:`options`
+
+        :statuscode 200: no error
+        :statuscode 404: requested origin can not be found in the archive
+
+        **Example:**
+
+        .. parsed-literal::
+
+            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
+    """
+    ori_dict = {
+        'id': int(origin_id) if origin_id else None,
+        'type': origin_type,
+        'url': origin_url
+    }
+    ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]}
+    error_msg = 'Origin %s not found.' % \
+        (ori_dict.get('id') or ori_dict['url'])
+
+    return api_lookup(
+        service.lookup_origin, ori_dict,
+        notfound_msg=error_msg,
+        enrich_fn=_enrich_origin)
\ No newline at end of file
anlambert: Indeed, the order of url declarations in Django matters for disambiguation. I managed to…
.. http:get:: /api/1/origin/(origin_url)/get/
Get information about a software origin.
:param string origin_url: the origin url
{return_origin}
{common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options`
:statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive
**Example:**
.. parsed-literal::
:swh_web_api:`origin/git/url/https://github.com/python/cpython/`
.. http:get:: /api/1/origin/(origin_id)/ .. http:get:: /api/1/origin/(origin_id)/
Get information about a software origin. Get information about a software origin.
:param int origin_id: a software origin identifier :param int origin_id: a software origin identifier
:>json number id: the origin unique identifier {return_origin}
:>json string origin_visits_url: link to in order to get information
about the visits for that origin
:>json string type: the type of software origin (possible values are
``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
``deposit``)
:>json string url: the origin canonical url
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive :statuscode 404: requested origin can not be found in the archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/1/` :swh_web_api:`origin/1/`
.. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/ .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/
Get information about a software origin. Get information about a software origin.
anlambertUnsubmitted
Not Done
Inline Actions

The warning directive is currently not parsed by the apidoc module.

This must be implemented in order to add the deprecation notice to the endpoint descriptions.

anlambert: The warning directive is currently not parsed by the `apidoc` module. This must be implemented…
.. warning::
This endpoint is deprecated. You should use
:http:get:`/api/1/origin/(origin_url)/get/` instead.
:param string origin_type: the origin type (possible values are :param string origin_type: the origin type (possible values are
``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
``deposit``) ``deposit``)
:param string origin_url: the origin url :param string origin_url: the origin url
:>json number id: the origin unique identifier {return_origin}
:>json string origin_visits_url: link to in order to get information
about the visits for that origin
:>json string type: the type of software origin
:>json string url: the origin canonical url
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive :statuscode 404: requested origin can not be found in the archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/git/url/https://github.com/python/cpython/` :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
""" """
ori_dict = { ori_dict = {
'id': int(origin_id) if origin_id else None, 'id': int(origin_id) if origin_id else None,
'type': origin_type, 'type': origin_type,
'url': origin_url 'url': origin_url
} }
ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]} ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]}
if 'id' in ori_dict: error_msg = 'Origin %s not found.' % \
error_msg = 'Origin with id %s not found.' % ori_dict['id'] (ori_dict.get('id') or ori_dict['url'])
else:
error_msg = 'Origin with type %s and URL %s not found' % (
ori_dict['type'], ori_dict['url'])
return api_lookup( return api_lookup(
service.lookup_origin, ori_dict, service.lookup_origin, ori_dict,
notfound_msg=error_msg, notfound_msg=error_msg,
enrich_fn=_enrich_origin) enrich_fn=_enrich_origin)
@api_route(r'/origin/search/(?P<url_pattern>.+)/', @api_route(r'/origin/search/(?P<url_pattern>.+)/',
'api-1-origin-search') 'api-1-origin-search')
@api_doc('/origin/search/') @api_doc('/origin/search/')
@format_docstring() @format_docstring(return_origin_array=DOC_RETURN_ORIGIN_ARRAY)
def api_origin_search(request, url_pattern): def api_origin_search(request, url_pattern):
""" """
.. http:get:: /api/1/origin/search/(url_pattern)/ .. http:get:: /api/1/origin/search/(url_pattern)/
Search for software origins whose urls contain a provided string Search for software origins whose urls contain a provided string
pattern or match a provided regular expression. pattern or match a provided regular expression.
The search is performed in a case insensitive way. The search is performed in a case insensitive way.
:param string url_pattern: a string pattern or a regular expression :param string url_pattern: a string pattern or a regular expression
:query int offset: the number of found origins to skip before returning :query int offset: the number of found origins to skip before returning
results results
:query int limit: the maximum number of found origins to return :query int limit: the maximum number of found origins to return
:query boolean regexp: if true, consider provided pattern as a regular :query boolean regexp: if true, consider provided pattern as a regular
expression and search origins whose urls match it expression and search origins whose urls match it
:query boolean with_visit: if true, only return origins with at least :query boolean with_visit: if true, only return origins with at least
one visit by Software heritage one visit by Software heritage
:>jsonarr number id: the origin unique identifier {return_origin_array}
:>jsonarr string origin_visits_url: link to in order to get information
about the visits for that origin
:>jsonarr string type: the type of software origin
:>jsonarr string url: the origin canonical url
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
Show All 31 Linesdef api_origin_search(request, url_pattern):
}) })
return result return result
@api_route(r'/origin/metadata-search/', @api_route(r'/origin/metadata-search/',
'api-1-origin-metadata-search') 'api-1-origin-metadata-search')
@api_doc('/origin/metadata-search/', noargs=True, need_params=True) @api_doc('/origin/metadata-search/', noargs=True, need_params=True)
@format_docstring() @format_docstring(return_origin_array=DOC_RETURN_ORIGIN_ARRAY)
def api_origin_metadata_search(request): def api_origin_metadata_search(request):
""" """
.. http:get:: /api/1/origin/metadata-search/ .. http:get:: /api/1/origin/metadata-search/
Search for software origins whose metadata (expressed as a Search for software origins whose metadata (expressed as a
JSON-LD/CodeMeta dictionary) match the provided criteria. JSON-LD/CodeMeta dictionary) match the provided criteria.
For now, only full-text search on this dictionary is supported. For now, only full-text search on this dictionary is supported.
:query str fulltext: a string that will be matched against origin :query str fulltext: a string that will be matched against origin
metadata; results are ranked and ordered starting with the best metadata; results are ranked and ordered starting with the best
ones. ones.
:query int limit: the maximum number of found origins to return :query int limit: the maximum number of found origins to return
(bounded to 100) (bounded to 100)
:>jsonarr number origin_id: the origin unique identifier {return_origin_array}
:>jsonarr dict metadata: metadata of the origin (as a
JSON-LD/CodeMeta dictionary)
:>jsonarr string from_revision: the revision used to extract these
metadata (the current HEAD or one of the former HEADs)
:>jsonarr dict tool: the tool used to extract these metadata
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
Show All 12 Linesdef api_origin_metadata_search(request):
results = api_lookup(service.search_origin_metadata, fulltext, limit) results = api_lookup(service.search_origin_metadata, fulltext, limit)
return { return {
'results': results, 'results': results,
} }
@api_route(r'/origin/(?P<origin_url>.*)/visits/', 'api-1-origin-visits')
@api_route(r'/origin/(?P<origin_id>[0-9]+)/visits/', 'api-1-origin-visits') @api_route(r'/origin/(?P<origin_id>[0-9]+)/visits/', 'api-1-origin-visits')
@api_doc('/origin/visits/') @api_doc('/origin/visits/')
@format_docstring() @format_docstring(
def api_origin_visits(request, origin_id): return_origin_visit_array=DOC_RETURN_ORIGIN_VISIT_ARRAY)
def api_origin_visits(request, origin_id=None, origin_url=None):
""" """
.. http:get:: /api/1/origin/(origin_url)/visits/
Get information about all visits of a software origin.
Visits are returned sorted in descending order according
to their date.
:param str origin_url: a software origin URL
:query int per_page: specify the number of visits to list, for
pagination purposes
:query int last_visit: visit to start listing from, for pagination
purposes
{common_headers}
{resheader_link}
{return_origin_visit_array}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options`
:statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive
**Example:**
.. parsed-literal::
:swh_web_api:`origin/https://github.com/hylang/hy/visits/`
.. http:get:: /api/1/origin/(origin_id)/visits/ .. http:get:: /api/1/origin/(origin_id)/visits/
Get information about all visits of a software origin. Get information about all visits of a software origin.
Visits are returned sorted in descending order according Visits are returned sorted in descending order according
to their date. to their date.
:param int origin_id: a software origin identifier :param int origin_id: a software origin identifier
:query int per_page: specify the number of visits to list, for :query int per_page: specify the number of visits to list, for
pagination purposes pagination purposes
:query int last_visit: visit to start listing from, for pagination :query int last_visit: visit to start listing from, for pagination
purposes purposes
{common_headers} {common_headers}
{resheader_link} {resheader_link}
:>jsonarr string date: ISO representation of the visit date (in UTC) {return_origin_visit_array}
:>jsonarr number id: the unique identifier of the origin
:>jsonarr string origin_visit_url: link to
:http:get:`/api/1/origin/(origin_id)/visit/(visit_id)/` in order to
get information about the visit
:>jsonarr string snapshot: the snapshot identifier of the visit
:>jsonarr string snapshot_url: link to
:http:get:`/api/1/snapshot/(snapshot_id)/` in order to get
information about the snapshot of the visit
:>jsonarr string status: status of the visit (either **full**,
**partial** or **ongoing**)
:>jsonarr number visit: the unique identifier of the visit
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive :statuscode 404: requested origin can not be found in the archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/1/visits/` :swh_web_api:`origin/1/visits/`
""" """
result = {} result = {}
origin_id = int(origin_id) if origin_url:
origin_query = {'url': origin_url}
notfound_msg = 'No origin {} found'.format(origin_url)
url_args_next = {'origin_url': origin_url}
else:
origin_query = {'id': int(origin_id)}
notfound_msg = 'No origin {} found'.format(origin_id)
url_args_next = {'origin_id': origin_id}
per_page = int(request.query_params.get('per_page', '10')) per_page = int(request.query_params.get('per_page', '10'))
last_visit = request.query_params.get('last_visit') last_visit = request.query_params.get('last_visit')
if last_visit: if last_visit:
last_visit = int(last_visit) last_visit = int(last_visit)
def _lookup_origin_visits( def _lookup_origin_visits(
origin_id, last_visit=last_visit, per_page=per_page): origin_query, last_visit=last_visit, per_page=per_page):
all_visits = get_origin_visits({'id': origin_id}) all_visits = get_origin_visits(origin_query)
all_visits.reverse() all_visits.reverse()
visits = [] visits = []
if not last_visit: if not last_visit:
visits = all_visits[:per_page] visits = all_visits[:per_page]
else: else:
for i, v in enumerate(all_visits): for i, v in enumerate(all_visits):
if v['visit'] == last_visit: if v['visit'] == last_visit:
visits = all_visits[i+1:i+1+per_page] visits = all_visits[i+1:i+1+per_page]
break break
for v in visits: for v in visits:
yield v yield v
results = api_lookup(_lookup_origin_visits, origin_id, results = api_lookup(_lookup_origin_visits, origin_query,
notfound_msg='No origin {} found'.format(origin_id), notfound_msg=notfound_msg,
enrich_fn=partial(_enrich_origin_visit, enrich_fn=partial(_enrich_origin_visit,
with_origin_url=False, with_origin_link=False,
with_origin_visit_url=True)) with_origin_visit_link=True))
if results: if results:
nb_results = len(results) nb_results = len(results)
if nb_results == per_page: if nb_results == per_page:
new_last_visit = results[-1]['visit'] new_last_visit = results[-1]['visit']
query_params = {} query_params = {}
query_params['last_visit'] = new_last_visit query_params['last_visit'] = new_last_visit
if request.query_params.get('per_page'): if request.query_params.get('per_page'):
query_params['per_page'] = per_page query_params['per_page'] = per_page
result['headers'] = { result['headers'] = {
'link-next': reverse('api-1-origin-visits', 'link-next': reverse('api-1-origin-visits',
url_args={'origin_id': origin_id}, url_args=url_args_next,
query_params=query_params) query_params=query_params)
} }
result.update({ result.update({
'results': results 'results': results
}) })
return result return result
@api_route(r'/origin/(?P<origin_url>.*)/visit/(?P<visit_id>[0-9]+)/',
'api-1-origin-visit')
@api_route(r'/origin/(?P<origin_id>[0-9]+)/visit/(?P<visit_id>[0-9]+)/', @api_route(r'/origin/(?P<origin_id>[0-9]+)/visit/(?P<visit_id>[0-9]+)/',
'api-1-origin-visit') 'api-1-origin-visit')
@api_doc('/origin/visit/') @api_doc('/origin/visit/')
@format_docstring() @format_docstring(return_origin_visit=DOC_RETURN_ORIGIN_VISIT)
def api_origin_visit(request, origin_id, visit_id): def api_origin_visit(request, visit_id, origin_url=None, origin_id=None):
""" """
.. http:get:: /api/1/origin/(origin_url)/visit/(visit_id)/
Get information about a specific visit of a software origin.
:param str origin_url: a software origin URL
:param int visit_id: a visit identifier
{common_headers}
{return_origin_visit}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options`
:statuscode 200: no error
:statuscode 404: requested origin or visit can not be found in the
archive
**Example:**
.. parsed-literal::
:swh_web_api:`origin/https://github.com/hylang/hy/visit/1/`
.. http:get:: /api/1/origin/(origin_id)/visit/(visit_id)/ .. http:get:: /api/1/origin/(origin_id)/visit/(visit_id)/
Get information about a specific visit of a software origin. Get information about a specific visit of a software origin.
:param int origin_id: a software origin identifier :param int origin_id: a software origin identifier
:param int visit_id: a visit identifier :param int visit_id: a visit identifier
{common_headers} {common_headers}
:>json string date: ISO representation of the visit date (in UTC) {return_origin_visit}
:>json number origin: the origin unique identifier
:>json string origin_url: link to get information about the origin
:>jsonarr string snapshot: the snapshot identifier of the visit
:>jsonarr string snapshot_url: link to
:http:get:`/api/1/snapshot/(snapshot_id)/` in order to get
information about the snapshot of the visit
:>json string status: status of the visit (either **full**,
**partial** or **ongoing**)
:>json number visit: the unique identifier of the visit
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin or visit can not be found in the :statuscode 404: requested origin or visit can not be found in the
archive archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/1500/visit/1/` :swh_web_api:`origin/1500/visit/1/`
""" """
if not origin_url:
origin_url = service.lookup_origin({'id': int(origin_id)})['url']
return api_lookup( return api_lookup(
service.lookup_origin_visit, int(origin_id), int(visit_id), service.lookup_origin_visit, origin_url, int(visit_id),
notfound_msg=('No visit {} for origin {} found' notfound_msg=('No visit {} for origin {} found'
.format(visit_id, origin_id)), .format(visit_id, origin_url)),
enrich_fn=partial(_enrich_origin_visit, enrich_fn=partial(_enrich_origin_visit,
with_origin_url=True, with_origin_link=True,
with_origin_visit_url=False)) with_origin_visit_link=False))
@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)' @api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)'
'/intrinsic-metadata', 'api-origin-intrinsic-metadata') '/intrinsic-metadata', 'api-origin-intrinsic-metadata')
@api_doc('/origin/intrinsic-metadata/') @api_doc('/origin/intrinsic-metadata/')
@format_docstring() @format_docstring()
def api_origin_intrinsic_metadata(request, origin_type, origin_url): def api_origin_intrinsic_metadata(request, origin_type, origin_url):
""" """
Show All 20 Lines .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/intrinsic-metadata
:swh_web_api:`origin/git/url/https://github.com/python/cpython/intrinsic-metadata` :swh_web_api:`origin/git/url/https://github.com/python/cpython/intrinsic-metadata`
""" # noqa """ # noqa
ori_dict = { ori_dict = {
'type': origin_type, 'type': origin_type,
'url': origin_url 'url': origin_url
} }
error_msg = 'Origin with type %s and URL %s not found' % ( error_msg = 'Origin with URL %s not found' % ori_dict['url']
ori_dict['type'], ori_dict['url'])
return api_lookup( return api_lookup(
service.lookup_origin_intrinsic_metadata, ori_dict, service.lookup_origin_intrinsic_metadata, ori_dict,
notfound_msg=error_msg, notfound_msg=error_msg,
enrich_fn=_enrich_origin) enrich_fn=_enrich_origin)
View Options

swh/loader/mercurial/tests/data/build.py

Show All 9 Lines
from swh.web.common.exc import BadInputExc from swh.web.common.exc import BadInputExc
from swh.web.common.origin_visits import get_origin_visits from swh.web.common.origin_visits import get_origin_visits
from swh.web.common.utils import reverse from swh.web.common.utils import reverse
from swh.web.api.apidoc import api_doc, format_docstring from swh.web.api.apidoc import api_doc, format_docstring
from swh.web.api.apiurls import api_route from swh.web.api.apiurls import api_route
from swh.web.api.views.utils import api_lookup from swh.web.api.views.utils import api_lookup
DOC_RETURN_ORIGIN = '''
:>json string origin_visits_url: link to in order to get information
about the visits for that origin
:>json string url: the origin canonical url
:>json string type: the type of software origin (deprecated value;
types are now associated to visits instead of origins)
:>json number id: the origin unique identifier (deprecated value;
you should only refer to origins based on their URL)
'''
DOC_RETURN_ORIGIN_ARRAY = \
DOC_RETURN_ORIGIN.replace(':>json', ':>jsonarr')
DOC_RETURN_ORIGIN_VISIT = '''
:>json string date: ISO representation of the visit date (in UTC)
:>json str origin: the origin canonical url
:>json string origin_url: link to get information about the origin
:>jsonarr string snapshot: the snapshot identifier of the visit
:>jsonarr string snapshot_url: link to
:http:get:`/api/1/snapshot/(snapshot_id)/` in order to get
information about the snapshot of the visit
:>json string status: status of the visit (either **full**,
**partial** or **ongoing**)
:>json number visit: the unique identifier of the visit
'''
DOC_RETURN_ORIGIN_VISIT_ARRAY = \
DOC_RETURN_ORIGIN_VISIT.replace(':>json', ':>jsonarr')
DOC_RETURN_ORIGIN_VISIT_ARRAY += '''
:>jsonarr number id: the unique identifier of the origin
:>jsonarr string origin_visit_url: link to
:http:get:`/api/1/origin/(origin_url)/visit/(visit_id)/`
in order to get information about the visit
'''
def _enrich_origin(origin): def _enrich_origin(origin):
if 'id' in origin: if 'id' in origin:
o = origin.copy() o = origin.copy()
o['origin_visits_url'] = reverse( o['origin_visits_url'] = reverse(
'api-1-origin-visits', url_args={'origin_id': origin['id']}) 'api-1-origin-visits', url_args={'origin_url': origin['url']})
return o return o
return origin return origin
def _enrich_origin_visit(origin_visit, *, def _enrich_origin_visit(origin_visit, *,
with_origin_url, with_origin_visit_url): with_origin_link, with_origin_visit_link):
ov = origin_visit.copy() ov = origin_visit.copy()
if with_origin_url: if with_origin_link:
ov['origin_url'] = reverse('api-1-origin', ov['origin_url'] = reverse('api-1-origin',
url_args={'origin_id': ov['origin']}) url_args={'origin_url': ov['origin']})
if with_origin_visit_url: if with_origin_visit_link:
ov['origin_visit_url'] = reverse('api-1-origin-visit', ov['origin_visit_url'] = reverse('api-1-origin-visit',
url_args={'origin_id': ov['origin'], url_args={'origin_url': ov['origin'],
'visit_id': ov['visit']}) 'visit_id': ov['visit']})
snapshot = ov['snapshot'] snapshot = ov['snapshot']
if snapshot: if snapshot:
ov['snapshot_url'] = reverse('api-1-snapshot', ov['snapshot_url'] = reverse('api-1-snapshot',
url_args={'snapshot_id': snapshot}) url_args={'snapshot_id': snapshot})
else: else:
ov['snapshot_url'] = None ov['snapshot_url'] = None
return ov return ov
@api_route(r'/origins/', 'api-1-origins') @api_route(r'/origins/', 'api-1-origins')
@api_doc('/origins/', noargs=True) @api_doc('/origins/', noargs=True)
@format_docstring() @format_docstring(return_origin_array=DOC_RETURN_ORIGIN_ARRAY)
def api_origins(request): def api_origins(request):
""" """
.. http:get:: /api/1/origins/ .. http:get:: /api/1/origins/
Get list of archived software origins. Get list of archived software origins.
Origins are sorted by ids before returning them. Origins are sorted by ids before returning them.
:query int origin_from: The first origin id that will be included :query int origin_from: The first origin id that will be included
in returned results (default to 1) in returned results (default to 1)
:query int origin_count: The maximum number of origins to return :query int origin_count: The maximum number of origins to return
(default to 100, can not exceed 10000) (default to 100, can not exceed 10000)
:>jsonarr number id: the origin unique identifier {return_origin_array}
:>jsonarr string origin_visits_url: link to in order to get information
about the visits for that origin
:>jsonarr string type: the type of software origin (possible values
are ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp``
or ``deposit``)
:>jsonarr string url: the origin canonical url
{common_headers} {common_headers}
{resheader_link} {resheader_link}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
Show All 15 Lines if len(results) > origin_count:
origin_from = results.pop()['id'] origin_from = results.pop()['id']
response['headers']['link-next'] = reverse( response['headers']['link-next'] = reverse(
'api-1-origins', 'api-1-origins',
query_params={'origin_from': origin_from, query_params={'origin_from': origin_from,
'origin_count': origin_count}) 'origin_count': origin_count})
return response return response
@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/', @api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/',
'api-1-origin') 'api-1-origin')
@api_route(r'/origin/(?P<origin_url>.+)/get/', 'api-1-origin')
@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
@api_doc('/origin/') @api_doc('/origin/')
@format_docstring() @format_docstring(return_origin=DOC_RETURN_ORIGIN)
def api_origin(request, origin_id=None, origin_type=None, origin_url=None): def api_origin(request, origin_id=None, origin_type=None, origin_url=None):
""" """
vlorentzAuthorUnsubmitted
Done
Inline Actions

Note the /get/, which is needed to disambiguate the URL.

Not very pretty, but that's the only solution I got.

vlorentz: Note the `/get/`, which is needed to disambiguate the URL. Not very pretty, but that's the…
anlambertUnsubmitted
Not Done
Inline Actions

Indeed, the order of url declarations in Django matters for disambiguation.

I managed to resolve the issue through the following diff

diff --git a/swh/web/api/urls.py b/swh/web/api/urls.py
index afbcaf44..3c3b99cd 100644
--- a/swh/web/api/urls.py
+++ b/swh/web/api/urls.py
@@ -6,8 +6,8 @@
 import swh.web.api.views.content # noqa
 import swh.web.api.views.directory # noqa
 import swh.web.api.views.identifiers # noqa
-import swh.web.api.views.origin # noqa
 import swh.web.api.views.origin_save # noqa
+import swh.web.api.views.origin # noqa
 import swh.web.api.views.person # noqa
 import swh.web.api.views.release # noqa
 import swh.web.api.views.revision # noqa
diff --git a/swh/web/api/views/origin.py b/swh/web/api/views/origin.py
index 0c70e485..cb54f0ca 100644
--- a/swh/web/api/views/origin.py
+++ b/swh/web/api/views/origin.py
@@ -129,103 +129,6 @@ def api_origins(request):
     return response
 
 
-@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/',
-           'api-1-origin')
-@api_route(r'/origin/(?P<origin_url>.+)/get/', 'api-1-origin')
-@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
-@api_doc('/origin/')
-@format_docstring(return_origin=DOC_RETURN_ORIGIN)
-def api_origin(request, origin_id=None, origin_type=None, origin_url=None):
-    """
-    .. http:get:: /api/1/origin/(origin_url)/get/
-
-        Get information about a software origin.
-
-        :param string origin_url: the origin url
-
-        {return_origin}
-
-        {common_headers}
-
-        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
-            :http:method:`options`
-
-        :statuscode 200: no error
-        :statuscode 404: requested origin can not be found in the archive
-
-        **Example:**
-
-        .. parsed-literal::
-
-            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
-
-    .. http:get:: /api/1/origin/(origin_id)/
-
-        Get information about a software origin.
-
-        :param int origin_id: a software origin identifier
-
-        {return_origin}
-
-        {common_headers}
-
-        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
-            :http:method:`options`
-
-        :statuscode 200: no error
-        :statuscode 404: requested origin can not be found in the archive
-
-        **Example:**
-
-        .. parsed-literal::
-
-            :swh_web_api:`origin/1/`
-
-    .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/
-
-        Get information about a software origin.
-
-        .. warning::
-
-            This endpoint is deprecated. You should use
-            :http:get:`/api/1/origin/(origin_url)/get/` instead.
-
-        :param string origin_type: the origin type (possible values are
-            ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
-            ``deposit``)
-        :param string origin_url: the origin url
-
-        {return_origin}
-
-        {common_headers}
-
-        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
-            :http:method:`options`
-
-        :statuscode 200: no error
-        :statuscode 404: requested origin can not be found in the archive
-
-        **Example:**
-
-        .. parsed-literal::
-
-            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
-    """
-    ori_dict = {
-        'id': int(origin_id) if origin_id else None,
-        'type': origin_type,
-        'url': origin_url
-    }
-    ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]}
-    error_msg = 'Origin %s not found.' % \
-        (ori_dict.get('id') or ori_dict['url'])
-
-    return api_lookup(
-        service.lookup_origin, ori_dict,
-        notfound_msg=error_msg,
-        enrich_fn=_enrich_origin)
-
-
 @api_route(r'/origin/search/(?P<url_pattern>.+)/',
            'api-1-origin-search')
 @api_doc('/origin/search/')
@@ -569,3 +472,100 @@ def api_origin_intrinsic_metadata(request, origin_type, origin_url):
         service.lookup_origin_intrinsic_metadata, ori_dict,
         notfound_msg=error_msg,
         enrich_fn=_enrich_origin)
+
+
+@api_route(r'/origin/(?P<origin_url>.+)/', 'api-1-origin')
+@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
+@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/',
+           'api-1-origin')
+@api_doc('/origin/')
+@format_docstring(return_origin=DOC_RETURN_ORIGIN)
+def api_origin(request, origin_id=None, origin_type=None, origin_url=None):
+    """
+    .. http:get:: /api/1/origin/(origin_url)/
+
+        Get information about a software origin.
+
+        :param string origin_url: the origin url
+
+        {return_origin}
+
+        {common_headers}
+
+        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
+            :http:method:`options`
+
+        :statuscode 200: no error
+        :statuscode 404: requested origin can not be found in the archive
+
+        **Example:**
+
+        .. parsed-literal::
+
+            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
+
+    .. http:get:: /api/1/origin/(origin_id)/
+
+        Get information about a software origin.
+
+        :param int origin_id: a software origin identifier
+
+        {return_origin}
+
+        {common_headers}
+
+        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
+            :http:method:`options`
+
+        :statuscode 200: no error
+        :statuscode 404: requested origin can not be found in the archive
+
+        **Example:**
+
+        .. parsed-literal::
+
+            :swh_web_api:`origin/1/`
+
+    .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/
+
+        Get information about a software origin.
+
+        .. warning::
+
+            This endpoint is deprecated. You should use
+            :http:get:`/api/1/origin/(origin_url)/get/` instead.
+
+        :param string origin_type: the origin type (possible values are
+            ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
+            ``deposit``)
+        :param string origin_url: the origin url
+
+        {return_origin}
+
+        {common_headers}
+
+        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
+            :http:method:`options`
+
+        :statuscode 200: no error
+        :statuscode 404: requested origin can not be found in the archive
+
+        **Example:**
+
+        .. parsed-literal::
+
+            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
+    """
+    ori_dict = {
+        'id': int(origin_id) if origin_id else None,
+        'type': origin_type,
+        'url': origin_url
+    }
+    ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]}
+    error_msg = 'Origin %s not found.' % \
+        (ori_dict.get('id') or ori_dict['url'])
+
+    return api_lookup(
+        service.lookup_origin, ori_dict,
+        notfound_msg=error_msg,
+        enrich_fn=_enrich_origin)
\ No newline at end of file
anlambert: Indeed, the order of url declarations in Django matters for disambiguation. I managed to…
.. http:get:: /api/1/origin/(origin_url)/get/
Get information about a software origin.
:param string origin_url: the origin url
{return_origin}
{common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options`
:statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive
**Example:**
.. parsed-literal::
:swh_web_api:`origin/git/url/https://github.com/python/cpython/`
.. http:get:: /api/1/origin/(origin_id)/ .. http:get:: /api/1/origin/(origin_id)/
Get information about a software origin. Get information about a software origin.
:param int origin_id: a software origin identifier :param int origin_id: a software origin identifier
:>json number id: the origin unique identifier {return_origin}
:>json string origin_visits_url: link to in order to get information
about the visits for that origin
:>json string type: the type of software origin (possible values are
``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
``deposit``)
:>json string url: the origin canonical url
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive :statuscode 404: requested origin can not be found in the archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/1/` :swh_web_api:`origin/1/`
.. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/ .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/
Get information about a software origin. Get information about a software origin.
anlambertUnsubmitted
Not Done
Inline Actions

The warning directive is currently not parsed by the apidoc module.

This must be implemented in order to add the deprecation notice to the endpoint descriptions.

anlambert: The warning directive is currently not parsed by the `apidoc` module. This must be implemented…
.. warning::
This endpoint is deprecated. You should use
:http:get:`/api/1/origin/(origin_url)/get/` instead.
:param string origin_type: the origin type (possible values are :param string origin_type: the origin type (possible values are
``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
``deposit``) ``deposit``)
:param string origin_url: the origin url :param string origin_url: the origin url
:>json number id: the origin unique identifier {return_origin}
:>json string origin_visits_url: link to in order to get information
about the visits for that origin
:>json string type: the type of software origin
:>json string url: the origin canonical url
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive :statuscode 404: requested origin can not be found in the archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/git/url/https://github.com/python/cpython/` :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
""" """
ori_dict = { ori_dict = {
'id': int(origin_id) if origin_id else None, 'id': int(origin_id) if origin_id else None,
'type': origin_type, 'type': origin_type,
'url': origin_url 'url': origin_url
} }
ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]} ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]}
if 'id' in ori_dict: error_msg = 'Origin %s not found.' % \
error_msg = 'Origin with id %s not found.' % ori_dict['id'] (ori_dict.get('id') or ori_dict['url'])
else:
error_msg = 'Origin with type %s and URL %s not found' % (
ori_dict['type'], ori_dict['url'])
return api_lookup( return api_lookup(
service.lookup_origin, ori_dict, service.lookup_origin, ori_dict,
notfound_msg=error_msg, notfound_msg=error_msg,
enrich_fn=_enrich_origin) enrich_fn=_enrich_origin)
@api_route(r'/origin/search/(?P<url_pattern>.+)/', @api_route(r'/origin/search/(?P<url_pattern>.+)/',
'api-1-origin-search') 'api-1-origin-search')
@api_doc('/origin/search/') @api_doc('/origin/search/')
@format_docstring() @format_docstring(return_origin_array=DOC_RETURN_ORIGIN_ARRAY)
def api_origin_search(request, url_pattern): def api_origin_search(request, url_pattern):
""" """
.. http:get:: /api/1/origin/search/(url_pattern)/ .. http:get:: /api/1/origin/search/(url_pattern)/
Search for software origins whose urls contain a provided string Search for software origins whose urls contain a provided string
pattern or match a provided regular expression. pattern or match a provided regular expression.
The search is performed in a case insensitive way. The search is performed in a case insensitive way.
:param string url_pattern: a string pattern or a regular expression :param string url_pattern: a string pattern or a regular expression
:query int offset: the number of found origins to skip before returning :query int offset: the number of found origins to skip before returning
results results
:query int limit: the maximum number of found origins to return :query int limit: the maximum number of found origins to return
:query boolean regexp: if true, consider provided pattern as a regular :query boolean regexp: if true, consider provided pattern as a regular
expression and search origins whose urls match it expression and search origins whose urls match it
:query boolean with_visit: if true, only return origins with at least :query boolean with_visit: if true, only return origins with at least
one visit by Software heritage one visit by Software heritage
:>jsonarr number id: the origin unique identifier {return_origin_array}
:>jsonarr string origin_visits_url: link to in order to get information
about the visits for that origin
:>jsonarr string type: the type of software origin
:>jsonarr string url: the origin canonical url
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
Show All 31 Linesdef api_origin_search(request, url_pattern):
}) })
return result return result
@api_route(r'/origin/metadata-search/', @api_route(r'/origin/metadata-search/',
'api-1-origin-metadata-search') 'api-1-origin-metadata-search')
@api_doc('/origin/metadata-search/', noargs=True, need_params=True) @api_doc('/origin/metadata-search/', noargs=True, need_params=True)
@format_docstring() @format_docstring(return_origin_array=DOC_RETURN_ORIGIN_ARRAY)
def api_origin_metadata_search(request): def api_origin_metadata_search(request):
""" """
.. http:get:: /api/1/origin/metadata-search/ .. http:get:: /api/1/origin/metadata-search/
Search for software origins whose metadata (expressed as a Search for software origins whose metadata (expressed as a
JSON-LD/CodeMeta dictionary) match the provided criteria. JSON-LD/CodeMeta dictionary) match the provided criteria.
For now, only full-text search on this dictionary is supported. For now, only full-text search on this dictionary is supported.
:query str fulltext: a string that will be matched against origin :query str fulltext: a string that will be matched against origin
metadata; results are ranked and ordered starting with the best metadata; results are ranked and ordered starting with the best
ones. ones.
:query int limit: the maximum number of found origins to return :query int limit: the maximum number of found origins to return
(bounded to 100) (bounded to 100)
:>jsonarr number origin_id: the origin unique identifier {return_origin_array}
:>jsonarr dict metadata: metadata of the origin (as a
JSON-LD/CodeMeta dictionary)
:>jsonarr string from_revision: the revision used to extract these
metadata (the current HEAD or one of the former HEADs)
:>jsonarr dict tool: the tool used to extract these metadata
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
Show All 12 Linesdef api_origin_metadata_search(request):
results = api_lookup(service.search_origin_metadata, fulltext, limit) results = api_lookup(service.search_origin_metadata, fulltext, limit)
return { return {
'results': results, 'results': results,
} }
@api_route(r'/origin/(?P<origin_url>.*)/visits/', 'api-1-origin-visits')
@api_route(r'/origin/(?P<origin_id>[0-9]+)/visits/', 'api-1-origin-visits') @api_route(r'/origin/(?P<origin_id>[0-9]+)/visits/', 'api-1-origin-visits')
@api_doc('/origin/visits/') @api_doc('/origin/visits/')
@format_docstring() @format_docstring(
def api_origin_visits(request, origin_id): return_origin_visit_array=DOC_RETURN_ORIGIN_VISIT_ARRAY)
def api_origin_visits(request, origin_id=None, origin_url=None):
""" """
.. http:get:: /api/1/origin/(origin_url)/visits/
Get information about all visits of a software origin.
Visits are returned sorted in descending order according
to their date.
:param str origin_url: a software origin URL
:query int per_page: specify the number of visits to list, for
pagination purposes
:query int last_visit: visit to start listing from, for pagination
purposes
{common_headers}
{resheader_link}
{return_origin_visit_array}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options`
:statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive
**Example:**
.. parsed-literal::
:swh_web_api:`origin/https://github.com/hylang/hy/visits/`
.. http:get:: /api/1/origin/(origin_id)/visits/ .. http:get:: /api/1/origin/(origin_id)/visits/
Get information about all visits of a software origin. Get information about all visits of a software origin.
Visits are returned sorted in descending order according Visits are returned sorted in descending order according
to their date. to their date.
:param int origin_id: a software origin identifier :param int origin_id: a software origin identifier
:query int per_page: specify the number of visits to list, for :query int per_page: specify the number of visits to list, for
pagination purposes pagination purposes
:query int last_visit: visit to start listing from, for pagination :query int last_visit: visit to start listing from, for pagination
purposes purposes
{common_headers} {common_headers}
{resheader_link} {resheader_link}
:>jsonarr string date: ISO representation of the visit date (in UTC) {return_origin_visit_array}
:>jsonarr number id: the unique identifier of the origin
:>jsonarr string origin_visit_url: link to
:http:get:`/api/1/origin/(origin_id)/visit/(visit_id)/` in order to
get information about the visit
:>jsonarr string snapshot: the snapshot identifier of the visit
:>jsonarr string snapshot_url: link to
:http:get:`/api/1/snapshot/(snapshot_id)/` in order to get
information about the snapshot of the visit
:>jsonarr string status: status of the visit (either **full**,
**partial** or **ongoing**)
:>jsonarr number visit: the unique identifier of the visit
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive :statuscode 404: requested origin can not be found in the archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/1/visits/` :swh_web_api:`origin/1/visits/`
""" """
result = {} result = {}
origin_id = int(origin_id) if origin_url:
origin_query = {'url': origin_url}
notfound_msg = 'No origin {} found'.format(origin_url)
url_args_next = {'origin_url': origin_url}
else:
origin_query = {'id': int(origin_id)}
notfound_msg = 'No origin {} found'.format(origin_id)
url_args_next = {'origin_id': origin_id}
per_page = int(request.query_params.get('per_page', '10')) per_page = int(request.query_params.get('per_page', '10'))
last_visit = request.query_params.get('last_visit') last_visit = request.query_params.get('last_visit')
if last_visit: if last_visit:
last_visit = int(last_visit) last_visit = int(last_visit)
def _lookup_origin_visits( def _lookup_origin_visits(
origin_id, last_visit=last_visit, per_page=per_page): origin_query, last_visit=last_visit, per_page=per_page):
all_visits = get_origin_visits({'id': origin_id}) all_visits = get_origin_visits(origin_query)
all_visits.reverse() all_visits.reverse()
visits = [] visits = []
if not last_visit: if not last_visit:
visits = all_visits[:per_page] visits = all_visits[:per_page]
else: else:
for i, v in enumerate(all_visits): for i, v in enumerate(all_visits):
if v['visit'] == last_visit: if v['visit'] == last_visit:
visits = all_visits[i+1:i+1+per_page] visits = all_visits[i+1:i+1+per_page]
break break
for v in visits: for v in visits:
yield v yield v
results = api_lookup(_lookup_origin_visits, origin_id, results = api_lookup(_lookup_origin_visits, origin_query,
notfound_msg='No origin {} found'.format(origin_id), notfound_msg=notfound_msg,
enrich_fn=partial(_enrich_origin_visit, enrich_fn=partial(_enrich_origin_visit,
with_origin_url=False, with_origin_link=False,
with_origin_visit_url=True)) with_origin_visit_link=True))
if results: if results:
nb_results = len(results) nb_results = len(results)
if nb_results == per_page: if nb_results == per_page:
new_last_visit = results[-1]['visit'] new_last_visit = results[-1]['visit']
query_params = {} query_params = {}
query_params['last_visit'] = new_last_visit query_params['last_visit'] = new_last_visit
if request.query_params.get('per_page'): if request.query_params.get('per_page'):
query_params['per_page'] = per_page query_params['per_page'] = per_page
result['headers'] = { result['headers'] = {
'link-next': reverse('api-1-origin-visits', 'link-next': reverse('api-1-origin-visits',
url_args={'origin_id': origin_id}, url_args=url_args_next,
query_params=query_params) query_params=query_params)
} }
result.update({ result.update({
'results': results 'results': results
}) })
return result return result
@api_route(r'/origin/(?P<origin_url>.*)/visit/(?P<visit_id>[0-9]+)/',
'api-1-origin-visit')
@api_route(r'/origin/(?P<origin_id>[0-9]+)/visit/(?P<visit_id>[0-9]+)/', @api_route(r'/origin/(?P<origin_id>[0-9]+)/visit/(?P<visit_id>[0-9]+)/',
'api-1-origin-visit') 'api-1-origin-visit')
@api_doc('/origin/visit/') @api_doc('/origin/visit/')
@format_docstring() @format_docstring(return_origin_visit=DOC_RETURN_ORIGIN_VISIT)
def api_origin_visit(request, origin_id, visit_id): def api_origin_visit(request, visit_id, origin_url=None, origin_id=None):
""" """
.. http:get:: /api/1/origin/(origin_url)/visit/(visit_id)/
Get information about a specific visit of a software origin.
:param str origin_url: a software origin URL
:param int visit_id: a visit identifier
{common_headers}
{return_origin_visit}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options`
:statuscode 200: no error
:statuscode 404: requested origin or visit can not be found in the
archive
**Example:**
.. parsed-literal::
:swh_web_api:`origin/https://github.com/hylang/hy/visit/1/`
.. http:get:: /api/1/origin/(origin_id)/visit/(visit_id)/ .. http:get:: /api/1/origin/(origin_id)/visit/(visit_id)/
Get information about a specific visit of a software origin. Get information about a specific visit of a software origin.
:param int origin_id: a software origin identifier :param int origin_id: a software origin identifier
:param int visit_id: a visit identifier :param int visit_id: a visit identifier
{common_headers} {common_headers}
:>json string date: ISO representation of the visit date (in UTC) {return_origin_visit}
:>json number origin: the origin unique identifier
:>json string origin_url: link to get information about the origin
:>jsonarr string snapshot: the snapshot identifier of the visit
:>jsonarr string snapshot_url: link to
:http:get:`/api/1/snapshot/(snapshot_id)/` in order to get
information about the snapshot of the visit
:>json string status: status of the visit (either **full**,
**partial** or **ongoing**)
:>json number visit: the unique identifier of the visit
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin or visit can not be found in the :statuscode 404: requested origin or visit can not be found in the
archive archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/1500/visit/1/` :swh_web_api:`origin/1500/visit/1/`
""" """
if not origin_url:
origin_url = service.lookup_origin({'id': int(origin_id)})['url']
return api_lookup( return api_lookup(
service.lookup_origin_visit, int(origin_id), int(visit_id), service.lookup_origin_visit, origin_url, int(visit_id),
notfound_msg=('No visit {} for origin {} found' notfound_msg=('No visit {} for origin {} found'
.format(visit_id, origin_id)), .format(visit_id, origin_url)),
enrich_fn=partial(_enrich_origin_visit, enrich_fn=partial(_enrich_origin_visit,
with_origin_url=True, with_origin_link=True,
with_origin_visit_url=False)) with_origin_visit_link=False))
@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)' @api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)'
'/intrinsic-metadata', 'api-origin-intrinsic-metadata') '/intrinsic-metadata', 'api-origin-intrinsic-metadata')
@api_doc('/origin/intrinsic-metadata/') @api_doc('/origin/intrinsic-metadata/')
@format_docstring() @format_docstring()
def api_origin_intrinsic_metadata(request, origin_type, origin_url): def api_origin_intrinsic_metadata(request, origin_type, origin_url):
""" """
Show All 20 Lines .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/intrinsic-metadata
:swh_web_api:`origin/git/url/https://github.com/python/cpython/intrinsic-metadata` :swh_web_api:`origin/git/url/https://github.com/python/cpython/intrinsic-metadata`
""" # noqa """ # noqa
ori_dict = { ori_dict = {
'type': origin_type, 'type': origin_type,
'url': origin_url 'url': origin_url
} }
error_msg = 'Origin with type %s and URL %s not found' % ( error_msg = 'Origin with URL %s not found' % ori_dict['url']
ori_dict['type'], ori_dict['url'])
return api_lookup( return api_lookup(
service.lookup_origin_intrinsic_metadata, ori_dict, service.lookup_origin_intrinsic_metadata, ori_dict,
notfound_msg=error_msg, notfound_msg=error_msg,
enrich_fn=_enrich_origin) enrich_fn=_enrich_origin)
View Options

swh/loader/mercurial/tests/data/hello.json

Show All 9 Lines
from swh.web.common.exc import BadInputExc from swh.web.common.exc import BadInputExc
from swh.web.common.origin_visits import get_origin_visits from swh.web.common.origin_visits import get_origin_visits
from swh.web.common.utils import reverse from swh.web.common.utils import reverse
from swh.web.api.apidoc import api_doc, format_docstring from swh.web.api.apidoc import api_doc, format_docstring
from swh.web.api.apiurls import api_route from swh.web.api.apiurls import api_route
from swh.web.api.views.utils import api_lookup from swh.web.api.views.utils import api_lookup
DOC_RETURN_ORIGIN = '''
:>json string origin_visits_url: link to in order to get information
about the visits for that origin
:>json string url: the origin canonical url
:>json string type: the type of software origin (deprecated value;
types are now associated to visits instead of origins)
:>json number id: the origin unique identifier (deprecated value;
you should only refer to origins based on their URL)
'''
DOC_RETURN_ORIGIN_ARRAY = \
DOC_RETURN_ORIGIN.replace(':>json', ':>jsonarr')
DOC_RETURN_ORIGIN_VISIT = '''
:>json string date: ISO representation of the visit date (in UTC)
:>json str origin: the origin canonical url
:>json string origin_url: link to get information about the origin
:>jsonarr string snapshot: the snapshot identifier of the visit
:>jsonarr string snapshot_url: link to
:http:get:`/api/1/snapshot/(snapshot_id)/` in order to get
information about the snapshot of the visit
:>json string status: status of the visit (either **full**,
**partial** or **ongoing**)
:>json number visit: the unique identifier of the visit
'''
DOC_RETURN_ORIGIN_VISIT_ARRAY = \
DOC_RETURN_ORIGIN_VISIT.replace(':>json', ':>jsonarr')
DOC_RETURN_ORIGIN_VISIT_ARRAY += '''
:>jsonarr number id: the unique identifier of the origin
:>jsonarr string origin_visit_url: link to
:http:get:`/api/1/origin/(origin_url)/visit/(visit_id)/`
in order to get information about the visit
'''
def _enrich_origin(origin): def _enrich_origin(origin):
if 'id' in origin: if 'id' in origin:
o = origin.copy() o = origin.copy()
o['origin_visits_url'] = reverse( o['origin_visits_url'] = reverse(
'api-1-origin-visits', url_args={'origin_id': origin['id']}) 'api-1-origin-visits', url_args={'origin_url': origin['url']})
return o return o
return origin return origin
def _enrich_origin_visit(origin_visit, *, def _enrich_origin_visit(origin_visit, *,
with_origin_url, with_origin_visit_url): with_origin_link, with_origin_visit_link):
ov = origin_visit.copy() ov = origin_visit.copy()
if with_origin_url: if with_origin_link:
ov['origin_url'] = reverse('api-1-origin', ov['origin_url'] = reverse('api-1-origin',
url_args={'origin_id': ov['origin']}) url_args={'origin_url': ov['origin']})
if with_origin_visit_url: if with_origin_visit_link:
ov['origin_visit_url'] = reverse('api-1-origin-visit', ov['origin_visit_url'] = reverse('api-1-origin-visit',
url_args={'origin_id': ov['origin'], url_args={'origin_url': ov['origin'],
'visit_id': ov['visit']}) 'visit_id': ov['visit']})
snapshot = ov['snapshot'] snapshot = ov['snapshot']
if snapshot: if snapshot:
ov['snapshot_url'] = reverse('api-1-snapshot', ov['snapshot_url'] = reverse('api-1-snapshot',
url_args={'snapshot_id': snapshot}) url_args={'snapshot_id': snapshot})
else: else:
ov['snapshot_url'] = None ov['snapshot_url'] = None
return ov return ov
@api_route(r'/origins/', 'api-1-origins') @api_route(r'/origins/', 'api-1-origins')
@api_doc('/origins/', noargs=True) @api_doc('/origins/', noargs=True)
@format_docstring() @format_docstring(return_origin_array=DOC_RETURN_ORIGIN_ARRAY)
def api_origins(request): def api_origins(request):
""" """
.. http:get:: /api/1/origins/ .. http:get:: /api/1/origins/
Get list of archived software origins. Get list of archived software origins.
Origins are sorted by ids before returning them. Origins are sorted by ids before returning them.
:query int origin_from: The first origin id that will be included :query int origin_from: The first origin id that will be included
in returned results (default to 1) in returned results (default to 1)
:query int origin_count: The maximum number of origins to return :query int origin_count: The maximum number of origins to return
(default to 100, can not exceed 10000) (default to 100, can not exceed 10000)
:>jsonarr number id: the origin unique identifier {return_origin_array}
:>jsonarr string origin_visits_url: link to in order to get information
about the visits for that origin
:>jsonarr string type: the type of software origin (possible values
are ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp``
or ``deposit``)
:>jsonarr string url: the origin canonical url
{common_headers} {common_headers}
{resheader_link} {resheader_link}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
Show All 15 Lines if len(results) > origin_count:
origin_from = results.pop()['id'] origin_from = results.pop()['id']
response['headers']['link-next'] = reverse( response['headers']['link-next'] = reverse(
'api-1-origins', 'api-1-origins',
query_params={'origin_from': origin_from, query_params={'origin_from': origin_from,
'origin_count': origin_count}) 'origin_count': origin_count})
return response return response
@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/', @api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/',
'api-1-origin') 'api-1-origin')
@api_route(r'/origin/(?P<origin_url>.+)/get/', 'api-1-origin')
@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
@api_doc('/origin/') @api_doc('/origin/')
@format_docstring() @format_docstring(return_origin=DOC_RETURN_ORIGIN)
def api_origin(request, origin_id=None, origin_type=None, origin_url=None): def api_origin(request, origin_id=None, origin_type=None, origin_url=None):
""" """
vlorentzAuthorUnsubmitted
Done
Inline Actions

Note the /get/, which is needed to disambiguate the URL.

Not very pretty, but that's the only solution I got.

vlorentz: Note the `/get/`, which is needed to disambiguate the URL. Not very pretty, but that's the…
anlambertUnsubmitted
Not Done
Inline Actions

Indeed, the order of url declarations in Django matters for disambiguation.

I managed to resolve the issue through the following diff

diff --git a/swh/web/api/urls.py b/swh/web/api/urls.py
index afbcaf44..3c3b99cd 100644
--- a/swh/web/api/urls.py
+++ b/swh/web/api/urls.py
@@ -6,8 +6,8 @@
 import swh.web.api.views.content # noqa
 import swh.web.api.views.directory # noqa
 import swh.web.api.views.identifiers # noqa
-import swh.web.api.views.origin # noqa
 import swh.web.api.views.origin_save # noqa
+import swh.web.api.views.origin # noqa
 import swh.web.api.views.person # noqa
 import swh.web.api.views.release # noqa
 import swh.web.api.views.revision # noqa
diff --git a/swh/web/api/views/origin.py b/swh/web/api/views/origin.py
index 0c70e485..cb54f0ca 100644
--- a/swh/web/api/views/origin.py
+++ b/swh/web/api/views/origin.py
@@ -129,103 +129,6 @@ def api_origins(request):
     return response
 
 
-@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/',
-           'api-1-origin')
-@api_route(r'/origin/(?P<origin_url>.+)/get/', 'api-1-origin')
-@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
-@api_doc('/origin/')
-@format_docstring(return_origin=DOC_RETURN_ORIGIN)
-def api_origin(request, origin_id=None, origin_type=None, origin_url=None):
-    """
-    .. http:get:: /api/1/origin/(origin_url)/get/
-
-        Get information about a software origin.
-
-        :param string origin_url: the origin url
-
-        {return_origin}
-
-        {common_headers}
-
-        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
-            :http:method:`options`
-
-        :statuscode 200: no error
-        :statuscode 404: requested origin can not be found in the archive
-
-        **Example:**
-
-        .. parsed-literal::
-
-            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
-
-    .. http:get:: /api/1/origin/(origin_id)/
-
-        Get information about a software origin.
-
-        :param int origin_id: a software origin identifier
-
-        {return_origin}
-
-        {common_headers}
-
-        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
-            :http:method:`options`
-
-        :statuscode 200: no error
-        :statuscode 404: requested origin can not be found in the archive
-
-        **Example:**
-
-        .. parsed-literal::
-
-            :swh_web_api:`origin/1/`
-
-    .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/
-
-        Get information about a software origin.
-
-        .. warning::
-
-            This endpoint is deprecated. You should use
-            :http:get:`/api/1/origin/(origin_url)/get/` instead.
-
-        :param string origin_type: the origin type (possible values are
-            ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
-            ``deposit``)
-        :param string origin_url: the origin url
-
-        {return_origin}
-
-        {common_headers}
-
-        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
-            :http:method:`options`
-
-        :statuscode 200: no error
-        :statuscode 404: requested origin can not be found in the archive
-
-        **Example:**
-
-        .. parsed-literal::
-
-            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
-    """
-    ori_dict = {
-        'id': int(origin_id) if origin_id else None,
-        'type': origin_type,
-        'url': origin_url
-    }
-    ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]}
-    error_msg = 'Origin %s not found.' % \
-        (ori_dict.get('id') or ori_dict['url'])
-
-    return api_lookup(
-        service.lookup_origin, ori_dict,
-        notfound_msg=error_msg,
-        enrich_fn=_enrich_origin)
-
-
 @api_route(r'/origin/search/(?P<url_pattern>.+)/',
            'api-1-origin-search')
 @api_doc('/origin/search/')
@@ -569,3 +472,100 @@ def api_origin_intrinsic_metadata(request, origin_type, origin_url):
         service.lookup_origin_intrinsic_metadata, ori_dict,
         notfound_msg=error_msg,
         enrich_fn=_enrich_origin)
+
+
+@api_route(r'/origin/(?P<origin_url>.+)/', 'api-1-origin')
+@api_route(r'/origin/(?P<origin_id>[0-9]+)/', 'api-1-origin')
+@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)/',
+           'api-1-origin')
+@api_doc('/origin/')
+@format_docstring(return_origin=DOC_RETURN_ORIGIN)
+def api_origin(request, origin_id=None, origin_type=None, origin_url=None):
+    """
+    .. http:get:: /api/1/origin/(origin_url)/
+
+        Get information about a software origin.
+
+        :param string origin_url: the origin url
+
+        {return_origin}
+
+        {common_headers}
+
+        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
+            :http:method:`options`
+
+        :statuscode 200: no error
+        :statuscode 404: requested origin can not be found in the archive
+
+        **Example:**
+
+        .. parsed-literal::
+
+            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
+
+    .. http:get:: /api/1/origin/(origin_id)/
+
+        Get information about a software origin.
+
+        :param int origin_id: a software origin identifier
+
+        {return_origin}
+
+        {common_headers}
+
+        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
+            :http:method:`options`
+
+        :statuscode 200: no error
+        :statuscode 404: requested origin can not be found in the archive
+
+        **Example:**
+
+        .. parsed-literal::
+
+            :swh_web_api:`origin/1/`
+
+    .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/
+
+        Get information about a software origin.
+
+        .. warning::
+
+            This endpoint is deprecated. You should use
+            :http:get:`/api/1/origin/(origin_url)/get/` instead.
+
+        :param string origin_type: the origin type (possible values are
+            ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
+            ``deposit``)
+        :param string origin_url: the origin url
+
+        {return_origin}
+
+        {common_headers}
+
+        **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
+            :http:method:`options`
+
+        :statuscode 200: no error
+        :statuscode 404: requested origin can not be found in the archive
+
+        **Example:**
+
+        .. parsed-literal::
+
+            :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
+    """
+    ori_dict = {
+        'id': int(origin_id) if origin_id else None,
+        'type': origin_type,
+        'url': origin_url
+    }
+    ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]}
+    error_msg = 'Origin %s not found.' % \
+        (ori_dict.get('id') or ori_dict['url'])
+
+    return api_lookup(
+        service.lookup_origin, ori_dict,
+        notfound_msg=error_msg,
+        enrich_fn=_enrich_origin)
\ No newline at end of file
anlambert: Indeed, the order of url declarations in Django matters for disambiguation. I managed to…
.. http:get:: /api/1/origin/(origin_url)/get/
Get information about a software origin.
:param string origin_url: the origin url
{return_origin}
{common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options`
:statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive
**Example:**
.. parsed-literal::
:swh_web_api:`origin/git/url/https://github.com/python/cpython/`
.. http:get:: /api/1/origin/(origin_id)/ .. http:get:: /api/1/origin/(origin_id)/
Get information about a software origin. Get information about a software origin.
:param int origin_id: a software origin identifier :param int origin_id: a software origin identifier
:>json number id: the origin unique identifier {return_origin}
:>json string origin_visits_url: link to in order to get information
about the visits for that origin
:>json string type: the type of software origin (possible values are
``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
``deposit``)
:>json string url: the origin canonical url
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive :statuscode 404: requested origin can not be found in the archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/1/` :swh_web_api:`origin/1/`
.. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/ .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/
Get information about a software origin. Get information about a software origin.
anlambertUnsubmitted
Not Done
Inline Actions

The warning directive is currently not parsed by the apidoc module.

This must be implemented in order to add the deprecation notice to the endpoint descriptions.

anlambert: The warning directive is currently not parsed by the `apidoc` module. This must be implemented…
.. warning::
This endpoint is deprecated. You should use
:http:get:`/api/1/origin/(origin_url)/get/` instead.
:param string origin_type: the origin type (possible values are :param string origin_type: the origin type (possible values are
``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or ``git``, ``svn``, ``hg``, ``deb``, ``pypi``, ``npm``, ``ftp`` or
``deposit``) ``deposit``)
:param string origin_url: the origin url :param string origin_url: the origin url
:>json number id: the origin unique identifier {return_origin}
:>json string origin_visits_url: link to in order to get information
about the visits for that origin
:>json string type: the type of software origin
:>json string url: the origin canonical url
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive :statuscode 404: requested origin can not be found in the archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/git/url/https://github.com/python/cpython/` :swh_web_api:`origin/git/url/https://github.com/python/cpython/`
""" """
ori_dict = { ori_dict = {
'id': int(origin_id) if origin_id else None, 'id': int(origin_id) if origin_id else None,
'type': origin_type, 'type': origin_type,
'url': origin_url 'url': origin_url
} }
ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]} ori_dict = {k: v for k, v in ori_dict.items() if ori_dict[k]}
if 'id' in ori_dict: error_msg = 'Origin %s not found.' % \
error_msg = 'Origin with id %s not found.' % ori_dict['id'] (ori_dict.get('id') or ori_dict['url'])
else:
error_msg = 'Origin with type %s and URL %s not found' % (
ori_dict['type'], ori_dict['url'])
return api_lookup( return api_lookup(
service.lookup_origin, ori_dict, service.lookup_origin, ori_dict,
notfound_msg=error_msg, notfound_msg=error_msg,
enrich_fn=_enrich_origin) enrich_fn=_enrich_origin)
@api_route(r'/origin/search/(?P<url_pattern>.+)/', @api_route(r'/origin/search/(?P<url_pattern>.+)/',
'api-1-origin-search') 'api-1-origin-search')
@api_doc('/origin/search/') @api_doc('/origin/search/')
@format_docstring() @format_docstring(return_origin_array=DOC_RETURN_ORIGIN_ARRAY)
def api_origin_search(request, url_pattern): def api_origin_search(request, url_pattern):
""" """
.. http:get:: /api/1/origin/search/(url_pattern)/ .. http:get:: /api/1/origin/search/(url_pattern)/
Search for software origins whose urls contain a provided string Search for software origins whose urls contain a provided string
pattern or match a provided regular expression. pattern or match a provided regular expression.
The search is performed in a case insensitive way. The search is performed in a case insensitive way.
:param string url_pattern: a string pattern or a regular expression :param string url_pattern: a string pattern or a regular expression
:query int offset: the number of found origins to skip before returning :query int offset: the number of found origins to skip before returning
results results
:query int limit: the maximum number of found origins to return :query int limit: the maximum number of found origins to return
:query boolean regexp: if true, consider provided pattern as a regular :query boolean regexp: if true, consider provided pattern as a regular
expression and search origins whose urls match it expression and search origins whose urls match it
:query boolean with_visit: if true, only return origins with at least :query boolean with_visit: if true, only return origins with at least
one visit by Software heritage one visit by Software heritage
:>jsonarr number id: the origin unique identifier {return_origin_array}
:>jsonarr string origin_visits_url: link to in order to get information
about the visits for that origin
:>jsonarr string type: the type of software origin
:>jsonarr string url: the origin canonical url
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
Show All 31 Linesdef api_origin_search(request, url_pattern):
}) })
return result return result
@api_route(r'/origin/metadata-search/', @api_route(r'/origin/metadata-search/',
'api-1-origin-metadata-search') 'api-1-origin-metadata-search')
@api_doc('/origin/metadata-search/', noargs=True, need_params=True) @api_doc('/origin/metadata-search/', noargs=True, need_params=True)
@format_docstring() @format_docstring(return_origin_array=DOC_RETURN_ORIGIN_ARRAY)
def api_origin_metadata_search(request): def api_origin_metadata_search(request):
""" """
.. http:get:: /api/1/origin/metadata-search/ .. http:get:: /api/1/origin/metadata-search/
Search for software origins whose metadata (expressed as a Search for software origins whose metadata (expressed as a
JSON-LD/CodeMeta dictionary) match the provided criteria. JSON-LD/CodeMeta dictionary) match the provided criteria.
For now, only full-text search on this dictionary is supported. For now, only full-text search on this dictionary is supported.
:query str fulltext: a string that will be matched against origin :query str fulltext: a string that will be matched against origin
metadata; results are ranked and ordered starting with the best metadata; results are ranked and ordered starting with the best
ones. ones.
:query int limit: the maximum number of found origins to return :query int limit: the maximum number of found origins to return
(bounded to 100) (bounded to 100)
:>jsonarr number origin_id: the origin unique identifier {return_origin_array}
:>jsonarr dict metadata: metadata of the origin (as a
JSON-LD/CodeMeta dictionary)
:>jsonarr string from_revision: the revision used to extract these
metadata (the current HEAD or one of the former HEADs)
:>jsonarr dict tool: the tool used to extract these metadata
{common_headers} {common_headers}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
Show All 12 Linesdef api_origin_metadata_search(request):
results = api_lookup(service.search_origin_metadata, fulltext, limit) results = api_lookup(service.search_origin_metadata, fulltext, limit)
return { return {
'results': results, 'results': results,
} }
@api_route(r'/origin/(?P<origin_url>.*)/visits/', 'api-1-origin-visits')
@api_route(r'/origin/(?P<origin_id>[0-9]+)/visits/', 'api-1-origin-visits') @api_route(r'/origin/(?P<origin_id>[0-9]+)/visits/', 'api-1-origin-visits')
@api_doc('/origin/visits/') @api_doc('/origin/visits/')
@format_docstring() @format_docstring(
def api_origin_visits(request, origin_id): return_origin_visit_array=DOC_RETURN_ORIGIN_VISIT_ARRAY)
def api_origin_visits(request, origin_id=None, origin_url=None):
""" """
.. http:get:: /api/1/origin/(origin_url)/visits/
Get information about all visits of a software origin.
Visits are returned sorted in descending order according
to their date.
:param str origin_url: a software origin URL
:query int per_page: specify the number of visits to list, for
pagination purposes
:query int last_visit: visit to start listing from, for pagination
purposes
{common_headers}
{resheader_link}
{return_origin_visit_array}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options`
:statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive
**Example:**
.. parsed-literal::
:swh_web_api:`origin/https://github.com/hylang/hy/visits/`
.. http:get:: /api/1/origin/(origin_id)/visits/ .. http:get:: /api/1/origin/(origin_id)/visits/
Get information about all visits of a software origin. Get information about all visits of a software origin.
Visits are returned sorted in descending order according Visits are returned sorted in descending order according
to their date. to their date.
:param int origin_id: a software origin identifier :param int origin_id: a software origin identifier
:query int per_page: specify the number of visits to list, for :query int per_page: specify the number of visits to list, for
pagination purposes pagination purposes
:query int last_visit: visit to start listing from, for pagination :query int last_visit: visit to start listing from, for pagination
purposes purposes
{common_headers} {common_headers}
{resheader_link} {resheader_link}
:>jsonarr string date: ISO representation of the visit date (in UTC) {return_origin_visit_array}
:>jsonarr number id: the unique identifier of the origin
:>jsonarr string origin_visit_url: link to
:http:get:`/api/1/origin/(origin_id)/visit/(visit_id)/` in order to
get information about the visit
:>jsonarr string snapshot: the snapshot identifier of the visit
:>jsonarr string snapshot_url: link to
:http:get:`/api/1/snapshot/(snapshot_id)/` in order to get
information about the snapshot of the visit
:>jsonarr string status: status of the visit (either **full**,
**partial** or **ongoing**)
:>jsonarr number visit: the unique identifier of the visit
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin can not be found in the archive :statuscode 404: requested origin can not be found in the archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/1/visits/` :swh_web_api:`origin/1/visits/`
""" """
result = {} result = {}
origin_id = int(origin_id) if origin_url:
origin_query = {'url': origin_url}
notfound_msg = 'No origin {} found'.format(origin_url)
url_args_next = {'origin_url': origin_url}
else:
origin_query = {'id': int(origin_id)}
notfound_msg = 'No origin {} found'.format(origin_id)
url_args_next = {'origin_id': origin_id}
per_page = int(request.query_params.get('per_page', '10')) per_page = int(request.query_params.get('per_page', '10'))
last_visit = request.query_params.get('last_visit') last_visit = request.query_params.get('last_visit')
if last_visit: if last_visit:
last_visit = int(last_visit) last_visit = int(last_visit)
def _lookup_origin_visits( def _lookup_origin_visits(
origin_id, last_visit=last_visit, per_page=per_page): origin_query, last_visit=last_visit, per_page=per_page):
all_visits = get_origin_visits({'id': origin_id}) all_visits = get_origin_visits(origin_query)
all_visits.reverse() all_visits.reverse()
visits = [] visits = []
if not last_visit: if not last_visit:
visits = all_visits[:per_page] visits = all_visits[:per_page]
else: else:
for i, v in enumerate(all_visits): for i, v in enumerate(all_visits):
if v['visit'] == last_visit: if v['visit'] == last_visit:
visits = all_visits[i+1:i+1+per_page] visits = all_visits[i+1:i+1+per_page]
break break
for v in visits: for v in visits:
yield v yield v
results = api_lookup(_lookup_origin_visits, origin_id, results = api_lookup(_lookup_origin_visits, origin_query,
notfound_msg='No origin {} found'.format(origin_id), notfound_msg=notfound_msg,
enrich_fn=partial(_enrich_origin_visit, enrich_fn=partial(_enrich_origin_visit,
with_origin_url=False, with_origin_link=False,
with_origin_visit_url=True)) with_origin_visit_link=True))
if results: if results:
nb_results = len(results) nb_results = len(results)
if nb_results == per_page: if nb_results == per_page:
new_last_visit = results[-1]['visit'] new_last_visit = results[-1]['visit']
query_params = {} query_params = {}
query_params['last_visit'] = new_last_visit query_params['last_visit'] = new_last_visit
if request.query_params.get('per_page'): if request.query_params.get('per_page'):
query_params['per_page'] = per_page query_params['per_page'] = per_page
result['headers'] = { result['headers'] = {
'link-next': reverse('api-1-origin-visits', 'link-next': reverse('api-1-origin-visits',
url_args={'origin_id': origin_id}, url_args=url_args_next,
query_params=query_params) query_params=query_params)
} }
result.update({ result.update({
'results': results 'results': results
}) })
return result return result
@api_route(r'/origin/(?P<origin_url>.*)/visit/(?P<visit_id>[0-9]+)/',
'api-1-origin-visit')
@api_route(r'/origin/(?P<origin_id>[0-9]+)/visit/(?P<visit_id>[0-9]+)/', @api_route(r'/origin/(?P<origin_id>[0-9]+)/visit/(?P<visit_id>[0-9]+)/',
'api-1-origin-visit') 'api-1-origin-visit')
@api_doc('/origin/visit/') @api_doc('/origin/visit/')
@format_docstring() @format_docstring(return_origin_visit=DOC_RETURN_ORIGIN_VISIT)
def api_origin_visit(request, origin_id, visit_id): def api_origin_visit(request, visit_id, origin_url=None, origin_id=None):
""" """
.. http:get:: /api/1/origin/(origin_url)/visit/(visit_id)/
Get information about a specific visit of a software origin.
:param str origin_url: a software origin URL
:param int visit_id: a visit identifier
{common_headers}
{return_origin_visit}
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options`
:statuscode 200: no error
:statuscode 404: requested origin or visit can not be found in the
archive
**Example:**
.. parsed-literal::
:swh_web_api:`origin/https://github.com/hylang/hy/visit/1/`
.. http:get:: /api/1/origin/(origin_id)/visit/(visit_id)/ .. http:get:: /api/1/origin/(origin_id)/visit/(visit_id)/
Get information about a specific visit of a software origin. Get information about a specific visit of a software origin.
:param int origin_id: a software origin identifier :param int origin_id: a software origin identifier
:param int visit_id: a visit identifier :param int visit_id: a visit identifier
{common_headers} {common_headers}
:>json string date: ISO representation of the visit date (in UTC) {return_origin_visit}
:>json number origin: the origin unique identifier
:>json string origin_url: link to get information about the origin
:>jsonarr string snapshot: the snapshot identifier of the visit
:>jsonarr string snapshot_url: link to
:http:get:`/api/1/snapshot/(snapshot_id)/` in order to get
information about the snapshot of the visit
:>json string status: status of the visit (either **full**,
**partial** or **ongoing**)
:>json number visit: the unique identifier of the visit
**Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`, **Allowed HTTP Methods:** :http:method:`get`, :http:method:`head`,
:http:method:`options` :http:method:`options`
:statuscode 200: no error :statuscode 200: no error
:statuscode 404: requested origin or visit can not be found in the :statuscode 404: requested origin or visit can not be found in the
archive archive
**Example:** **Example:**
.. parsed-literal:: .. parsed-literal::
:swh_web_api:`origin/1500/visit/1/` :swh_web_api:`origin/1500/visit/1/`
""" """
if not origin_url:
origin_url = service.lookup_origin({'id': int(origin_id)})['url']
return api_lookup( return api_lookup(
service.lookup_origin_visit, int(origin_id), int(visit_id), service.lookup_origin_visit, origin_url, int(visit_id),
notfound_msg=('No visit {} for origin {} found' notfound_msg=('No visit {} for origin {} found'
.format(visit_id, origin_id)), .format(visit_id, origin_url)),
enrich_fn=partial(_enrich_origin_visit, enrich_fn=partial(_enrich_origin_visit,
with_origin_url=True, with_origin_link=True,
with_origin_visit_url=False)) with_origin_visit_link=False))
@api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)' @api_route(r'/origin/(?P<origin_type>[a-z]+)/url/(?P<origin_url>.+)'
'/intrinsic-metadata', 'api-origin-intrinsic-metadata') '/intrinsic-metadata', 'api-origin-intrinsic-metadata')
@api_doc('/origin/intrinsic-metadata/') @api_doc('/origin/intrinsic-metadata/')
@format_docstring() @format_docstring()
def api_origin_intrinsic_metadata(request, origin_type, origin_url): def api_origin_intrinsic_metadata(request, origin_type, origin_url):
""" """
Show All 20 Lines .. http:get:: /api/1/origin/(origin_type)/url/(origin_url)/intrinsic-metadata
:swh_web_api:`origin/git/url/https://github.com/python/cpython/intrinsic-metadata` :swh_web_api:`origin/git/url/https://github.com/python/cpython/intrinsic-metadata`
""" # noqa """ # noqa
ori_dict = { ori_dict = {
'type': origin_type, 'type': origin_type,
'url': origin_url 'url': origin_url
} }
error_msg = 'Origin with type %s and URL %s not found' % ( error_msg = 'Origin with URL %s not found' % ori_dict['url']
ori_dict['type'], ori_dict['url'])
return api_lookup( return api_lookup(
service.lookup_origin_intrinsic_metadata, ori_dict, service.lookup_origin_intrinsic_metadata, ori_dict,
notfound_msg=error_msg, notfound_msg=error_msg,
enrich_fn=_enrich_origin) enrich_fn=_enrich_origin)
View Options

swh/loader/mercurial/tests/data/the-sandbox.json

Loading...
View Options

swh/loader/mercurial/tests/data/transplant.json

Loading...
View Options

swh/loader/mercurial/tests/dvcs_loader_checker.py

Loading...
View Options

swh/loader/mercurial/tests/test_identify.py

Loading...
View Options

swh/loader/mercurial/tests/test_loader.py

Loading...
About · Developer information · Legal