diff --git a/docs/developers-info.rst b/docs/developers-info.rst index e89a6b126..dc2c2af3f 100644 --- a/docs/developers-info.rst +++ b/docs/developers-info.rst @@ -1,128 +1,131 @@ Developers Information ====================== Sample configuration -------------------- The configuration will be taken from the default configuration file: *~/.config/swh/webapp/webapp.yml*. The following introduces a default configuration file: .. sourcecode:: yaml storage: cls: remote args: url: http://localhost:5002 debug: false throttling: cache_uri: None scopes: swh_api: limiter_rate: 120/h exempted_networks: - 127.0.0.0/8 Run server ---------- Either use the django manage script directly (useful in development mode as it offers various commands): .. sourcecode:: shell $ python3 -m swh.web.manage runserver or use the following shortcut: .. sourcecode:: shell $ make run Modules description ------------------- Common to all web applications ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Configuration and settings """""""""""""""""""""""""" * :mod:`swh.web.config`: holds the SWH configuration for the web applications. * :mod:`swh.web.doc_config`: utility module used to extend the sphinx configuration when building the documentation. * :mod:`swh.web.manage`: Django management module for developpers. - * :mod:`swh.web.urls`: module that holds the whole URI scheme of all + * :mod:`swh.web.urls`: module that holds the whole URI scheme of all the web applications. * :mod:`swh.web.wsgi`: WSGI module to use when deploying the web applications in production. * :mod:`swh.web.settings.common`: Common Django settings * :mod:`swh.web.settings.development`: Django settings for development * :mod:`swh.web.settings.production`: Django settings for production * :mod:`swh.web.settings.tests`: Django settings for tests Common utilities """""""""""""""" - * :mod:`swh.web.common.converters`: conversion module used to transform SWH raw data - to serializable ones. It is used by :mod:`swh.web.common.service`: to convert data + * :mod:`swh.web.common.converters`: conversion module used to transform SWH raw data + to serializable ones. It is used by :mod:`swh.web.common.service`: to convert data before transmitting then to Django views. * :mod:`swh.web.common.exc`: module defining exceptions used in the web applications. * :mod:`swh.web.common.highlightjs`: utility module to ease the use of the highlightjs_ library in produced Django views. - * :mod:`swh.web.common.query`: Utilities to parse data from HTTP endpoints. It is used + * :mod:`swh.web.common.query`: Utilities to parse data from HTTP endpoints. It is used by :mod:`swh.web.common.service`. * :mod:`swh.web.common.service`: Orchestration layer used by views module - in charge of communication with :mod:`swh.storage` to retrieve information and + in charge of communication with :mod:`swh.storage` to retrieve information and perform conversion for the upper layer. * :mod:`swh.web.common.swh_templatetags`: Custom Django template tags library for swh. - * :mod:`swh.web.common.throttling`: Custom request rate limiter to use with the `Django REST Framework + * :mod:`swh.web.common.throttling`: Custom request rate limiter to use with the `Django REST Framework `_ - * :mod:`swh.web.common.urlsindex`: Utilities to help the registering of endpoints + * :mod:`swh.web.common.urlsindex`: Utilities to help the registering of endpoints for the web applications * :mod:`swh.web.common.utils`: Utility functions used in the web applications implementation - - + + SWH Web API application ^^^^^^^^^^^^^^^^^^^^^^^ * :mod:`swh.web.api.apidoc`: Utilities to document the web api for its html browsable rendering. * :mod:`swh.web.api.apiresponse`: Utility module to ease the generation of web api responses. * :mod:`swh.web.api.apiurls`: Utilities to facilitate the registration of SWH web api endpoints. * :mod:`swh.web.api.urls`: Module that defines the whole URI scheme for the api endpoints * :mod:`swh.web.api.utils`: Utility functions used in the SWH web api implementation. * :mod:`swh.web.api.views.content`: Implementation of API endpoints for getting information about SWH contents. * :mod:`swh.web.api.views.directory`: Implementation of API endpoints for getting information about SWH directories. * :mod:`swh.web.api.views.entity`: Implementation of API endpoints for getting information about SWH entities. * :mod:`swh.web.api.views.origin`: Implementation of API endpoints for getting information about SWH origins. * :mod:`swh.web.api.views.person`: Implementation of API endpoints for getting information about SWH persons. * :mod:`swh.web.api.views.release`: Implementation of API endpoints for getting information about SWH releases. * :mod:`swh.web.api.views.revision`: Implementation of API endpoints for getting information about SWH revisions. * :mod:`swh.web.api.views.stat`: Implementation of API endpoints for getting information about SWH archive statistics. * :mod:`swh.web.api.views.utils`: Utilities used in the web api endpoints implementation. SWH Web browse application ^^^^^^^^^^^^^^^^^^^^^^^^^^ * :mod:`swh.web.browse.browseurls`: Utilities to facilitate the registration of SWH web browse endpoints. * :mod:`swh.web.browse.urls`: Module that defines the whole URI scheme for the SWH web browse endpoints. * :mod:`swh.web.browse.utils`: Utilities functions used troughout the SWH web browse endpoints implementation. * :mod:`swh.web.browse.views.content`: Implementation of endpoints for browsing SWH contents. * :mod:`swh.web.browse.views.directory`: Implementation of endpoints for browsing SWH directories. + * :mod:`swh.web.browse.views.identifiers`: Implementation of endpoints for browsing SWH objects + through persistent identifiers. * :mod:`swh.web.browse.views.origin`: Implementation of endpoints for browsing SWH origins. * :mod:`swh.web.browse.views.person`: Implementation of endpoints for browsing SWH persons. + * :mod:`swh.web.browse.views.release`: Implementation of endpoints for browsing SWH releases. * :mod:`swh.web.browse.views.revision`: Implementation of endpoints for browsing SWH revisions. - + .. _highlightjs: https://highlightjs.org/ diff --git a/docs/uri-scheme-browse-release.rst b/docs/uri-scheme-browse-release.rst index 1938b52a8..608c1b507 100644 --- a/docs/uri-scheme-browse-release.rst +++ b/docs/uri-scheme-browse-release.rst @@ -1,23 +1,25 @@ Release ^^^^^^^ .. http:get:: /browse/release/(sha1_git)/ HTML view that displays metadata associated to a SWH release: * the author * the release date * the release name * the associated message * the type of target the release points to (revision, directory, content or relase) * the link to browse the release target :param string sha1_git: hexadecimal representation for the *sha1_git* identifier of a SWH release :statuscode 200: no error :statuscode 404: requested release can not be found in the SWH archive + **Examples:** + .. parsed-literal:: :swh_web_browse:`release/208f61cc7a5dbc9879ae6e5c2f95891e270f09ef/` :swh_web_browse:`release/f883596e997fe5bcbc5e89bee01b869721326109/` diff --git a/docs/uri-scheme-browse-revision.rst b/docs/uri-scheme-browse-revision.rst index 9990ccd80..8d6f435ea 100644 --- a/docs/uri-scheme-browse-revision.rst +++ b/docs/uri-scheme-browse-revision.rst @@ -1,60 +1,64 @@ Revision ^^^^^^^^ .. http:get:: /browse/revision/(sha1_git)/ HTML view that displays the metadata associated to a SWH revision. - It notably shows the revision date and message but also offers + It notably shows the revision date and message but also offers links to get more details on: * the author * the committer * the directory that revision points to * the history log reachable from that revision :param string sha1_git: hexadecimal representation for the *sha1_git* - identifier of a SWH revision + identifier of a SWH revision :statuscode 200: no error :statuscode 404: requested revision can not be found in the SWH archive + **Examples:** + .. parsed-literal:: :swh_web_browse:`revision/f1b94134a4b879bc55c3dacdb496690c8ebdc03f/` :swh_web_browse:`revision/d1aa2b3f607b35dc5dbf613b2334b6d243ec2bda/` .. http:get:: /browse/revision/(sha1_git)/log/ - HTML view that displays the list of revisions heading to + HTML view that displays the list of revisions heading to a given one. In other words, it shows a commit log. The following data are displayed for each log entry: * author of the revision * link to the revision metadata * message associated to the revision * date of the revision * link to browse the associated source tree N log entries are displayed per page (default is 20). In order to navigate in a large history, two buttons are present at the bottom of the view: * *Newer*: fetch and display if available the N more recent log entries than the ones currently displayed * *Older*: fetch and display if available the N older log entries than the ones currently displayed - :param string sha1_git: hexadecimal representation for the *sha1_git* - identifier of a SWH revision - :query string revs_breadcrumb: used internally to store + :param string sha1_git: hexadecimal representation for the *sha1_git* + identifier of a SWH revision + :query string revs_breadcrumb: used internally to store the navigation breadcrumbs (i.e. the list of descendant revisions - visited so far). It must be a string in the form + visited so far). It must be a string in the form "[//.../]" where rev_i corresponds to a revision sha1_git. - :query int per_page: the number of log entries to display per page + :query int per_page: the number of log entries to display per page (default is 20, max is 50) :statuscode 200: no error :statuscode 404: requested revision can not be found in the SWH archive + **Examples:** + .. parsed-literal:: :swh_web_browse:`revision/f1b94134a4b879bc55c3dacdb496690c8ebdc03f/log/` :swh_web_browse:`revision/d1aa2b3f607b35dc5dbf613b2334b6d243ec2bda/log/` \ No newline at end of file diff --git a/docs/uri-scheme-browse.rst b/docs/uri-scheme-browse.rst index 7eefa3cba..a950afe9b 100644 --- a/docs/uri-scheme-browse.rst +++ b/docs/uri-scheme-browse.rst @@ -1,96 +1,96 @@ URI scheme for SWH Web Browse application ========================================= This web application aims to provide HTML views to easily navigate in the SWH archive, thus it needs to be reached from a web browser. If you intend to query the SWH archive programmatically through any HTTP client, please refer to the :ref:`swh-web-api-urls` section instead. Context-independent browsing ---------------------------- Context-independent URLs provide information about SWH objects (e.g., revisions, directories, contents, person, ...), independently of the contexts where they have been found (e.g., specific repositories, branches, commits, ...). The following endpoints are the same of the API case (see below), and just render the corresponding information for user consumption. Where hyperlinks are created, they always point to other context-independent user URLs: * :http:get:`/browse/content/[(algo_hash):](hash)/`: Display a SWH content * :http:get:`/browse/content/[(algo_hash):](hash)/raw/`: Get / Download SWH content raw data * :http:get:`/browse/directory/(sha1_git)/[(path)/]`: Browse the content of a SWH directory * :http:get:`/browse/person/(person_id)/`: Information on a SWH person * :http:get:`/browse/revision/(sha1_git)/`: Browse a SWH revision * :http:get:`/browse/revision/(sha1_git)/log/`: Browse history log heading to a SWH revision Context-dependent browsing -------------------------- Context-dependent URLs provide information about SWH objects, limited to specific contexts where the objects have been found. For instance, instead of having to specify a (root) revision by *sha1_git*, users might want to specify a place and a time. In SWH a "place" is an origin, with an optional branch name; a "time" is a timestamp at which some place has been observed by SWH crawlers. Wherever a revision context is expected in a path (i.e., a **/browse/revision/(sha1_git)/** path fragment) we can put in its stead a path fragment of the form **/origin/(origin_type)/url/(origin_url)/[/visit/(timestamp)/][?branch=(branch)]**. Such a fragment is resolved, internally by the SWH archive, to a *sha1_git* as follows: - if *timestamp* is absent: look for the most recent crawl of origin *origin_id* - if *timestamp* is given: look for the closest crawl of origin *origin_id* from timestamp *timestamp* - if *branch* is given as a query parameter: look for the branch *branch* - if *branch* is absent: look for branch "HEAD" or "master" - return the *sha1_git* pointed by the chosen branch The already mentioned URLs for revision contexts can therefore be alternatively specified by users as: * :http:get:`/browse/origin/(origin_type)/url/(origin_url)/directory/[(path)/]` * :http:get:`/browse/origin/(origin_type)/url/(origin_url)/visit/(timestamp)/directory/[(path)/]` * :http:get:`/browse/origin/(origin_type)/url/(origin_url)/content/(path)/` * :http:get:`/browse/origin/(origin_type)/url/(origin_url)/visit/(timestamp)/content/(path)/` * :http:get:`/browse/origin/(origin_type)/url/(origin_url)/log/` * :http:get:`/browse/origin/(origin_type)/url/(origin_url)/visit/(timestamp)/log/` Typing: - *origin_id* is given as an integer identifier, pointing into the origin table. There will be separate mechanisms for finding origins by other means (e.g., URLs, metadata, etc). - *branch* name is given as per the corresponding VCS (e.g., Git) as a query parameter to the requestes URL. - *timestamp* is given in a format as liberal as possible, to uphold the principle of least surprise. At the very minimum it is possible to enter timestamps as: - Unix epoch timestamp (see for instance the output of `date +%s`) - ISO 8601 timestamps (see for instance the output of `date -I`, `date -Is`) - YYYY[MM[DD[HH[MM[SS]]]]] ad-hoc format - YYYY[-MM[-DD[ HH:[MM:[SS:]]]]] ad-hoc format SWH Browse Urls --------------- .. include:: uri-scheme-browse-content.rst .. include:: uri-scheme-browse-directory.rst +.. include:: uri-scheme-browse-identifiers.rst + .. include:: uri-scheme-browse-origin.rst .. include:: uri-scheme-browse-person.rst -.. include:: uri-scheme-browse-revision.rst - .. include:: uri-scheme-browse-release.rst -.. include:: uri-scheme-browse-identifiers.rst +.. include:: uri-scheme-browse-revision.rst