diff --git a/PKG-INFO b/PKG-INFO index 394f731f..f8495fe3 100644 --- a/PKG-INFO +++ b/PKG-INFO @@ -1,127 +1,127 @@ Metadata-Version: 2.1 Name: swh.web -Version: 0.0.163 +Version: 0.0.164 Summary: Software Heritage Web UI Home-page: https://forge.softwareheritage.org/diffusion/DWUI/ Author: Software Heritage developers Author-email: swh-devel@inria.fr License: UNKNOWN Project-URL: Bug Reports, https://forge.softwareheritage.org/maniphest Project-URL: Funding, https://www.softwareheritage.org/donate Project-URL: Source, https://forge.softwareheritage.org/source/swh-web Description: # swh-web This repository holds the development of Software Heritage web applications: * SWH Web API (https://archive.softwareheritage.org/api): enables to query the content of the SWH archive through HTTP requests and get responses in JSON or YAML. * SWH Web browse (https://archive.softwareheritage.org/browse): graphical interface that eases the navigation in the SWH archive. Documentation about how to use these components but also the details of their URI schemes can be found in the docs folder. The produced HTML documentation can be read and browsed at https://docs.softwareheritage.org/devel/swh-web/index.html. ## Technical details Those applications are powered by: * [Django Web Framework](https://www.djangoproject.com/) on the backend side with the following extensions enabled: * [django-rest-framework](http://www.django-rest-framework.org/) * [django-webpack-loader](https://github.com/owais/django-webpack-loader) * [django-js-reverse](http://django-js-reverse.readthedocs.io/en/latest/) * [webpack](https://webpack.js.org/) on the frontend side for better static assets management, including: * assets dependencies management and retrieval through [npm](https://www.npmjs.com/) * linting of custom javascript code (through [eslint](https://eslint.org/)) and stylesheets (through [stylelint](https://stylelint.io/)) * use of [es6](http://es6-features.org) syntax and advanced javascript feature like [async/await](https://javascript.info/async-await) or [fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) thanks to [babel](https://babeljs.io/) (es6 to es5 transpiler and polyfills provider) * assets minification (using [UglifyJS](https://github.com/mishoo/UglifyJS2) and [cssnano](http://cssnano.co/)) but also dead code elimination for production use ## How to build and run ### Requirements First you will need [Python 3](https://www.python.org) and a complete [swh development environment](https://forge.softwareheritage.org/source/swh-environment/) installed. To run the backend, you need to have the following Python 3 modules installed: * beautifulsoup4 * django >= 1.10.7 * djangorestframework >= 3.4.0 * django_webpack_loader * django_js_reverse * docutils * file_magic >= 0.3.0 * htmlmin * lxml * pygments * pypandoc * python-dateutil * pyyaml * requests To compile the frontend assets, you need to have [nodejs](https://nodejs.org/en/) >= 8.x (preferably version 8.x LTS) and [npm](https://www.npmjs.com/) (or [yarn](https://yarnpkg.com/en/)) installed. If you are on Debian stretch, you can easily install an up to date nodejs/npm from the stretch-backports repository or by following the instructions located at https://github.com/nodesource/distributions. Once you have installed nodejs, issue the following command in the root directory of swh-web in order to retrieve all the frontend dependencies: ``` $ npm install ``` or if you prefer to use yarn (faster than npm): ``` $ yarn install ``` Please note that the static assets bundles generated by webpack are not stored in the git repository. Follow the instructions below in order to generate them in order to be able to run the frontend part of the web applications. ### Make targets Below is the list of available make targets that can be executed from the root directory of swh-web in order to build and/or execute the web applications under various configurations: * **run-django-webpack-devserver**: Compile and serve not optimized (without mignification and dead code elimination) frontend static assets using [webpack-dev-server](https://github.com/webpack/webpack-dev-server) and run django server with development settings. This is the recommended target to use when developing swh-web as it enables automatic reloading of backend and frontend part of the applications when modifying source files (*.py, *.js, *.css, *.html). * **run-django-webpack-dev**: Compile not optimized (no minification, no dead code elimination) frontend static assets using webpack and run django server with development settings. This is the recommended target when one only wants to develop the backend side of the application. * **run-django-webpack-prod**: Compile optimized (with minification and dead code elimination) frontend static assets using webpack and run django server with production settings. This is useful to test the applications in production mode (with the difference that static assets are served by django). Production settings notably enable advanced django caching and you will need to have [memcached](https://memcached.org/) installed for that feature to work. * **run-django-server-dev**: Run the django server with development settings but without compiling frontend static assets through webpack. * **run-django-server-prod**: Run the django server with production settings but without compiling frontend static assets through webpack. * **run-gunicorn-server**: Run the web applications with production settings in a [gunicorn](http://gunicorn.org/) worker as they will be in real production environment. Once one of these targets executed, the web applications can be executed by pointing your browser to http://localhost:5004. ### Npm/Yarn targets Below is a list of available npm/yarn targets in order to only execute the frontend static assets compilation (no web server will be executed): * **build-dev**: compile not optimized (without mignification and dead code elimination) frontend static assets and store the results in the `swh/web/static` folder. * **build**: compile optimized (with mignification and dead code elimination) frontend static assets and store the results in the `swh/web/static` folder. **The build target must be executed prior performing the Debian packaging of swh-web** in order for the package to contain the optimized assets dedicated to production environment. To execute these targets, issue the following commmand: ``` $ npm run ``` or if you prefer using yarn: ``` $ yarn ``` Platform: UNKNOWN Classifier: Programming Language :: Python :: 3 Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+) Classifier: Operating System :: OS Independent Classifier: Development Status :: 5 - Production/Stable Classifier: Framework :: Django Description-Content-Type: text/markdown Provides-Extra: testing diff --git a/debian/changelog b/debian/changelog index 0018d5bb..9209d671 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,1216 +1,1216 @@ -swh-web (0.0.163-1~swh1~bpo9+1) stretch-swh; urgency=medium +swh-web (0.0.164-1~swh1) unstable-swh; urgency=medium - * Rebuild for stretch-backports. + * version 0.0.164 - -- Antoine Lambert Wed, 31 Oct 2018 17:17:05 +0100 + -- Antoine Lambert Wed, 31 Oct 2018 17:38:39 +0100 swh-web (0.0.163-1~swh1) unstable-swh; urgency=medium * version 0.0.163 -- Antoine Lambert Wed, 31 Oct 2018 17:17:05 +0100 swh-web (0.0.162-1~swh1) unstable-swh; urgency=medium * version 0.0.162 -- Antoine Lambert Thu, 18 Oct 2018 17:57:52 +0200 swh-web (0.0.161-1~swh1) unstable-swh; urgency=medium * version 0.0.161 -- Antoine Lambert Wed, 17 Oct 2018 15:30:50 +0200 swh-web (0.0.160-1~swh1) unstable-swh; urgency=medium * version 0.0.160 -- Antoine Lambert Fri, 12 Oct 2018 15:28:05 +0200 swh-web (0.0.159-1~swh1) unstable-swh; urgency=medium * version 0.0.159 -- Antoine Lambert Fri, 12 Oct 2018 10:18:46 +0200 swh-web (0.0.158-1~swh1) unstable-swh; urgency=medium * version 0.0.158 -- Antoine Lambert Thu, 11 Oct 2018 17:49:17 +0200 swh-web (0.0.157-1~swh1) unstable-swh; urgency=medium * version 0.0.157 -- Antoine Lambert Thu, 27 Sep 2018 17:21:28 +0200 swh-web (0.0.156-1~swh1) unstable-swh; urgency=medium * version 0.0.156 -- Antoine Lambert Thu, 20 Sep 2018 14:40:37 +0200 swh-web (0.0.155-1~swh1) unstable-swh; urgency=medium * version 0.0.155 -- Antoine Lambert Tue, 18 Sep 2018 10:44:38 +0200 swh-web (0.0.154-1~swh1) unstable-swh; urgency=medium * version 0.0.154 -- Antoine Lambert Fri, 14 Sep 2018 16:37:48 +0200 swh-web (0.0.153-1~swh1) unstable-swh; urgency=medium * version 0.0.153 -- Antoine Lambert Wed, 12 Sep 2018 16:44:06 +0200 swh-web (0.0.152-1~swh1) unstable-swh; urgency=medium * version 0.0.152 -- Antoine Lambert Wed, 12 Sep 2018 16:04:47 +0200 swh-web (0.0.151-1~swh1) unstable-swh; urgency=medium * version 0.0.151 -- Antoine Lambert Tue, 04 Sep 2018 17:28:46 +0200 swh-web (0.0.150-1~swh1) unstable-swh; urgency=medium * version 0.0.150 -- Antoine Lambert Tue, 04 Sep 2018 15:15:05 +0200 swh-web (0.0.149-1~swh1) unstable-swh; urgency=medium * version 0.0.149 -- Antoine Lambert Thu, 30 Aug 2018 16:23:05 +0200 swh-web (0.0.148-1~swh1) unstable-swh; urgency=medium * version 0.0.148 -- Antoine Lambert Thu, 30 Aug 2018 11:27:42 +0200 swh-web (0.0.147-1~swh1) unstable-swh; urgency=medium * version 0.0.147 -- Antoine Lambert Fri, 03 Aug 2018 14:41:04 +0200 swh-web (0.0.146-1~swh1) unstable-swh; urgency=medium * version 0.0.146 -- Antoine Lambert Fri, 27 Jul 2018 16:37:33 +0200 swh-web (0.0.145-1~swh1) unstable-swh; urgency=medium * version 0.0.145 -- Antoine Lambert Fri, 27 Jul 2018 16:10:36 +0200 swh-web (0.0.144-1~swh1) unstable-swh; urgency=medium * version 0.0.144 -- Antoine Lambert Fri, 20 Jul 2018 16:26:52 +0200 swh-web (0.0.143-1~swh1) unstable-swh; urgency=medium * version 0.0.143 -- Antoine Lambert Fri, 20 Jul 2018 16:19:56 +0200 swh-web (0.0.142-1~swh1) unstable-swh; urgency=medium * version 0.0.142 -- Antoine Lambert Fri, 20 Jul 2018 15:51:20 +0200 swh-web (0.0.141-1~swh1) unstable-swh; urgency=medium * version 0.0.141 -- Antoine Lambert Fri, 06 Jul 2018 14:11:39 +0200 swh-web (0.0.140-1~swh1) unstable-swh; urgency=medium * version 0.0.140 -- Antoine Lambert Fri, 29 Jun 2018 16:42:06 +0200 swh-web (0.0.139-1~swh1) unstable-swh; urgency=medium * version 0.0.139 -- Antoine Lambert Wed, 27 Jun 2018 16:47:17 +0200 swh-web (0.0.138-1~swh1) unstable-swh; urgency=medium * version 0.0.138 -- Antoine Lambert Wed, 13 Jun 2018 12:18:23 +0200 swh-web (0.0.137-1~swh1) unstable-swh; urgency=medium * version 0.0.137 -- Antoine Lambert Wed, 13 Jun 2018 11:52:05 +0200 swh-web (0.0.136-1~swh1) unstable-swh; urgency=medium * version 0.0.136 -- Antoine Lambert Tue, 05 Jun 2018 18:59:20 +0200 swh-web (0.0.135-1~swh1) unstable-swh; urgency=medium * version 0.0.135 -- Antoine Lambert Fri, 01 Jun 2018 17:47:58 +0200 swh-web (0.0.134-1~swh1) unstable-swh; urgency=medium * version 0.0.134 -- Antoine Lambert Thu, 31 May 2018 17:56:04 +0200 swh-web (0.0.133-1~swh1) unstable-swh; urgency=medium * version 0.0.133 -- Antoine Lambert Tue, 29 May 2018 18:13:59 +0200 swh-web (0.0.132-1~swh1) unstable-swh; urgency=medium * version 0.0.132 -- Antoine Lambert Tue, 29 May 2018 14:25:16 +0200 swh-web (0.0.131-1~swh1) unstable-swh; urgency=medium * version 0.0.131 -- Antoine Lambert Fri, 25 May 2018 17:31:58 +0200 swh-web (0.0.130-1~swh1) unstable-swh; urgency=medium * version 0.0.130 -- Antoine Lambert Fri, 25 May 2018 11:59:17 +0200 swh-web (0.0.129-1~swh1) unstable-swh; urgency=medium * version 0.0.129 -- Antoine Lambert Thu, 24 May 2018 18:28:48 +0200 swh-web (0.0.128-1~swh1) unstable-swh; urgency=medium * version 0.0.128 -- Antoine Lambert Wed, 16 May 2018 13:52:33 +0200 swh-web (0.0.127-1~swh1) unstable-swh; urgency=medium * version 0.0.127 -- Antoine Lambert Fri, 04 May 2018 19:14:58 +0200 swh-web (0.0.126-1~swh1) unstable-swh; urgency=medium * version 0.0.126 -- Antoine Lambert Fri, 04 May 2018 15:29:49 +0200 swh-web (0.0.125-1~swh1) unstable-swh; urgency=medium * version 0.0.125 -- Antoine Lambert Fri, 20 Apr 2018 15:45:05 +0200 swh-web (0.0.124-1~swh1) unstable-swh; urgency=medium * version 0.0.124 -- Antoine Lambert Fri, 20 Apr 2018 14:46:00 +0200 swh-web (0.0.123-1~swh1) unstable-swh; urgency=medium * version 0.0.123 -- Antoine Lambert Mon, 26 Mar 2018 11:34:32 +0200 swh-web (0.0.122-1~swh1) unstable-swh; urgency=medium * version 0.0.122 -- Antoine Lambert Wed, 14 Mar 2018 17:23:15 +0100 swh-web (0.0.121-1~swh1) unstable-swh; urgency=medium * version 0.0.121 -- Antoine Lambert Wed, 07 Mar 2018 18:02:29 +0100 swh-web (0.0.120-1~swh1) unstable-swh; urgency=medium * version 0.0.120 -- Antoine Lambert Wed, 07 Mar 2018 17:31:08 +0100 swh-web (0.0.119-1~swh1) unstable-swh; urgency=medium * version 0.0.119 -- Antoine Lambert Thu, 01 Mar 2018 18:11:40 +0100 swh-web (0.0.118-1~swh1) unstable-swh; urgency=medium * version 0.0.118 -- Antoine Lambert Thu, 22 Feb 2018 17:26:28 +0100 swh-web (0.0.117-1~swh1) unstable-swh; urgency=medium * version 0.0.117 -- Antoine Lambert Wed, 21 Feb 2018 14:56:27 +0100 swh-web (0.0.116-1~swh1) unstable-swh; urgency=medium * version 0.0.116 -- Antoine Lambert Mon, 19 Feb 2018 17:47:57 +0100 swh-web (0.0.115-1~swh1) unstable-swh; urgency=medium * version 0.0.115 -- Antoine Lambert Mon, 19 Feb 2018 12:00:47 +0100 swh-web (0.0.114-1~swh1) unstable-swh; urgency=medium * version 0.0.114 -- Antoine Lambert Fri, 16 Feb 2018 16:13:58 +0100 swh-web (0.0.113-1~swh1) unstable-swh; urgency=medium * version 0.0.113 -- Antoine Lambert Thu, 15 Feb 2018 15:52:57 +0100 swh-web (0.0.112-1~swh1) unstable-swh; urgency=medium * version 0.0.112 -- Antoine Lambert Thu, 08 Feb 2018 12:10:44 +0100 swh-web (0.0.111-1~swh1) unstable-swh; urgency=medium * Release swh.web v0.0.111 * Support snapshot information in origin_visit -- Nicolas Dandrimont Tue, 06 Feb 2018 14:54:29 +0100 swh-web (0.0.110-1~swh1) unstable-swh; urgency=medium * version 0.0.110 -- Antoine Lambert Fri, 02 Feb 2018 15:52:10 +0100 swh-web (0.0.109-1~swh1) unstable-swh; urgency=medium * version 0.0.109 -- Antoine Lambert Thu, 01 Feb 2018 18:04:10 +0100 swh-web (0.0.108-1~swh1) unstable-swh; urgency=medium * version 0.0.108 -- Antoine Lambert Tue, 23 Jan 2018 17:31:13 +0100 swh-web (0.0.107-1~swh1) unstable-swh; urgency=medium * version 0.0.107 -- Antoine Lambert Tue, 23 Jan 2018 12:13:58 +0100 swh-web (0.0.106-1~swh1) unstable-swh; urgency=medium * version 0.0.106 -- Antoine Lambert Thu, 18 Jan 2018 15:28:44 +0100 swh-web (0.0.105-1~swh1) unstable-swh; urgency=medium * version 0.0.105 -- Antoine Lambert Tue, 09 Jan 2018 17:32:29 +0100 swh-web (0.0.104-1~swh1) unstable-swh; urgency=medium * version 0.0.104 -- Antoine Lambert Tue, 09 Jan 2018 14:29:32 +0100 swh-web (0.0.103-1~swh1) unstable-swh; urgency=medium * version 0.0.103 -- Antoine Lambert Thu, 04 Jan 2018 16:48:56 +0100 swh-web (0.0.102-1~swh1) unstable-swh; urgency=medium * version 0.0.102 -- Antoine Lambert Thu, 14 Dec 2017 15:13:22 +0100 swh-web (0.0.101-1~swh1) unstable-swh; urgency=medium * version 0.0.101 -- Antoine Pietri Fri, 08 Dec 2017 16:38:05 +0100 swh-web (0.0.100-1~swh1) unstable-swh; urgency=medium * v0.0.100 * swh.web.common.service: Read indexer data through the indexer * storage -- Antoine R. Dumont (@ardumont) Thu, 07 Dec 2017 16:25:12 +0100 swh-web (0.0.99-1~swh1) unstable-swh; urgency=medium * version 0.0.99 -- Antoine Lambert Wed, 06 Dec 2017 17:07:37 +0100 swh-web (0.0.98-1~swh1) unstable-swh; urgency=medium * version 0.0.98 -- Antoine Lambert Wed, 06 Dec 2017 15:41:13 +0100 swh-web (0.0.97-1~swh1) unstable-swh; urgency=medium * version 0.0.97 -- Antoine Lambert Fri, 24 Nov 2017 16:24:07 +0100 swh-web (0.0.96-1~swh1) unstable-swh; urgency=medium * version 0.0.96 -- Antoine Lambert Fri, 24 Nov 2017 15:22:16 +0100 swh-web (0.0.95-1~swh1) unstable-swh; urgency=medium * version 0.0.95 -- Antoine Lambert Thu, 09 Nov 2017 18:14:31 +0100 swh-web (0.0.94-1~swh1) unstable-swh; urgency=medium * version 0.0.94 -- Antoine Lambert Mon, 06 Nov 2017 16:19:48 +0100 swh-web (0.0.93-1~swh1) unstable-swh; urgency=medium * version 0.0.93 -- Antoine Lambert Fri, 27 Oct 2017 16:28:22 +0200 swh-web (0.0.92-1~swh1) unstable-swh; urgency=medium * version 0.0.92 -- Antoine Lambert Fri, 27 Oct 2017 16:07:47 +0200 swh-web (0.0.91-1~swh1) unstable-swh; urgency=medium * v0.0.91 -- Antoine Lambert Fri, 13 Oct 2017 20:40:07 +0200 swh-web (0.0.90-1~swh1) unstable-swh; urgency=medium * version 0.0.90 -- Antoine Lambert Wed, 04 Oct 2017 13:53:28 +0200 swh-web (0.0.89-1~swh1) unstable-swh; urgency=medium * version 0.0.89 -- Antoine Lambert Wed, 04 Oct 2017 10:42:11 +0200 swh-web (0.0.88-1~swh1) unstable-swh; urgency=medium * v0.0.88 * Fix default webapp configuration file lookup * Fix templating errors * Fix wrong default configuration * Add missing endpoint information about error (origin visit endpoint) -- Antoine R. Dumont (@ardumont) Wed, 13 Sep 2017 15:02:24 +0200 swh-web (0.0.87-1~swh1) unstable-swh; urgency=medium * v0.0.87 * throttling: permit the use to define cache server * throttling: improve configuration intent * configuration: Clarify config keys intent and improve config * management * docs: change content example to ls.c from GNU corutils * packaging: Fix dependency requirements -- Antoine R. Dumont (@ardumont) Tue, 12 Sep 2017 14:11:10 +0200 swh-web (0.0.86-1~swh1) unstable-swh; urgency=medium * v0.0.86 -- Antoine Lambert Fri, 08 Sep 2017 14:07:19 +0200 swh-web (0.0.85-1~swh1) unstable-swh; urgency=medium * v0.0.85 -- Antoine Lambert Fri, 08 Sep 2017 10:55:50 +0200 swh-web (0.0.84-1~swh1) unstable-swh; urgency=medium * Release swh.web.ui v0.0.84 * Prepare stretch packaging -- Nicolas Dandrimont Fri, 30 Jun 2017 18:18:55 +0200 swh-web (0.0.83-1~swh1) unstable-swh; urgency=medium * Release swh.web.ui v0.0.83 * Allow exemption by network for rate limiting -- Nicolas Dandrimont Wed, 24 May 2017 18:01:53 +0200 swh-web (0.0.82-1~swh1) unstable-swh; urgency=medium * v0.0.83 * Add new blake2s256 data column on content -- Antoine R. Dumont (@ardumont) Tue, 04 Apr 2017 16:54:25 +0200 swh-web (0.0.81-1~swh1) unstable-swh; urgency=medium * v0.0.81 * Migrate functions from swh.core.hashutil to swh.model.hashutil -- Antoine R. Dumont (@ardumont) Wed, 15 Mar 2017 16:26:42 +0100 swh-web (0.0.80-1~swh1) unstable-swh; urgency=medium * v0.0.80 * /api/1/content/raw/: Make no textual content request forbidden -- Antoine R. Dumont (@ardumont) Wed, 15 Mar 2017 12:35:43 +0100 swh-web (0.0.79-1~swh1) unstable-swh; urgency=medium * v0.0.79 * /api/1/content/raw/: Improve error msg when content not available * /api/1/content/raw/: Open endpoint documentation in api endpoints * index -- Antoine R. Dumont (@ardumont) Wed, 15 Mar 2017 11:43:00 +0100 swh-web (0.0.78-1~swh1) unstable-swh; urgency=medium * v0.0.78 * /api/1/content/raw/: Open endpoint to download only text-ish * contents (other contents are deemed unavailable) * /api/1/content/raw/: Permit the user to provide a 'filename' * parameter to name the downloaded contents as they see fit. -- Antoine R. Dumont (@ardumont) Wed, 15 Mar 2017 10:48:21 +0100 swh-web (0.0.77-1~swh1) unstable-swh; urgency=medium * v0.0.77 * API doc: add warning about API instability * API: Unify remaining dates as iso8601 string * /api/1/revision/: Merge 'parents' key into a dict list * /api/1/release/: Enrich output with author_url if author mentioned * packaging: split internal and external requirements in separate files -- Antoine R. Dumont (@ardumont) Tue, 21 Feb 2017 11:37:19 +0100 swh-web (0.0.76-1~swh1) unstable-swh; urgency=medium * Release swh.web.ui v0.0.76 * Refactor APIDoc to be more sensible * Share rate limits between all the api_ queries -- Nicolas Dandrimont Thu, 02 Feb 2017 17:32:57 +0100 swh-web (0.0.75-1~swh1) unstable-swh; urgency=medium * v0.0.75 * Remove build dependency on libjs-cryptojs, libjs-jquery-flot*, * libjs-jquery-datatables * views/browse,api: move main apidoc views to views/api -- Antoine R. Dumont (@ardumont) Thu, 02 Feb 2017 15:03:20 +0100 swh-web (0.0.74-1~swh1) unstable-swh; urgency=medium * Release swh.web.ui v0.0.74 * Various interface cleanups for API documentation * Return Error types in API error return values -- Nicolas Dandrimont Thu, 02 Feb 2017 11:03:56 +0100 swh-web (0.0.73-1~swh1) unstable-swh; urgency=medium * Deploy swh.web.ui v0.0.73 * Add a bazillion of style fixes. -- Nicolas Dandrimont Wed, 01 Feb 2017 22:44:10 +0100 swh-web (0.0.72-1~swh1) unstable-swh; urgency=medium * v0.0.72 * apidoc rendering: Improvments * apidoc: add usual copyright/license/contact footer * apidoc: show status code if != 200 * apidoc: hide /content/known/ from the doc * apidoc: document upcoming v. available in endpoint index * apidoc: vertically distantiate jquery search box and preceding text -- Antoine R. Dumont (@ardumont) Wed, 01 Feb 2017 18:34:56 +0100 swh-web (0.0.71-1~swh1) unstable-swh; urgency=medium * v0.0.71 * add static/robots.txt, disabling crawling of /api/ * re-root content-specific endpoints under /api/1/content/ * fix not converted empty bytes string * /revision/origin/: Make the timestamp default to the most recent visit * api: simplify HTML layout by dropping redundant nav and about page * apidoc: document correctly endpoints /content/known/, * /revision/{origin,origin/log}/ and /stat/counters/ -- Antoine R. Dumont (@ardumont) Wed, 01 Feb 2017 16:23:56 +0100 swh-web (0.0.70-1~swh1) unstable-swh; urgency=medium * v0.0.70 * apidoc: Review documentation for * endpoints (person/release/revision/visit-related/upcoming methods) * apidoc: List only method docstring's first paragraph in endpoint index * apidoc: Render type annotation for optional parameter * apidoc: Improve rendering issues * api: Fix problem in origin visit by type and url lookup -- Antoine R. Dumont (@ardumont) Wed, 01 Feb 2017 11:28:32 +0100 swh-web (0.0.69-1~swh1) unstable-swh; urgency=medium * v0.0.69 * Improve documentation information and rendering -- Antoine R. Dumont (@ardumont) Tue, 31 Jan 2017 14:31:19 +0100 swh-web (0.0.68-1~swh1) unstable-swh; urgency=medium * v0.0.68 * Improve ui with last nitpicks * Remove endpoints not supposed to be displayed -- Antoine R. Dumont (@ardumont) Wed, 25 Jan 2017 13:29:49 +0100 swh-web (0.0.67-1~swh1) unstable-swh; urgency=medium * v0.0.67 * Improve rendering style - pass 4 -- Antoine R. Dumont (@ardumont) Tue, 24 Jan 2017 15:30:58 +0100 swh-web (0.0.66-1~swh1) unstable-swh; urgency=medium * v0.0.66 * Improve rendering style - pass 4 -- Antoine R. Dumont (@ardumont) Tue, 24 Jan 2017 15:24:05 +0100 swh-web (0.0.65-1~swh1) unstable-swh; urgency=medium * v0.0.65 * Unify rendering style with www.s.o - pass 3 -- Antoine R. Dumont (@ardumont) Mon, 23 Jan 2017 19:58:19 +0100 swh-web (0.0.64-1~swh1) unstable-swh; urgency=medium * v0.0.64 * Unify rendering style with www.s.o - pass 2 -- Antoine R. Dumont (@ardumont) Mon, 23 Jan 2017 19:28:31 +0100 swh-web (0.0.63-1~swh1) unstable-swh; urgency=medium * v0.0.63 * Unify rendering style with www.s.o - pass 1 -- Antoine R. Dumont (@ardumont) Mon, 23 Jan 2017 16:06:30 +0100 swh-web (0.0.62-1~swh1) unstable-swh; urgency=medium * Release swh-web-ui v0.0.62 * Add flask-limiter to dependencies and wire it in -- Nicolas Dandrimont Fri, 20 Jan 2017 16:29:48 +0100 swh-web (0.0.61-1~swh1) unstable-swh; urgency=medium * v0.0.61 * Fix revision's metadata field limitation -- Antoine R. Dumont (@ardumont) Fri, 20 Jan 2017 15:26:37 +0100 swh-web (0.0.60-1~swh1) unstable-swh; urgency=medium * v0.0.60 * Improve escaping data -- Antoine R. Dumont (@ardumont) Fri, 20 Jan 2017 12:21:22 +0100 swh-web (0.0.59-1~swh1) unstable-swh; urgency=medium * v0.0.59 * Unify pagination on /revision/log/ and /revision/origin/log/ endpoints -- Antoine R. Dumont (@ardumont) Thu, 19 Jan 2017 15:59:06 +0100 swh-web (0.0.58-1~swh1) unstable-swh; urgency=medium * v0.0.58 * Pagination on /api/1/origin/visits/ endpoint -- Antoine R. Dumont (@ardumont) Thu, 19 Jan 2017 14:48:57 +0100 swh-web (0.0.57-1~swh1) unstable-swh; urgency=medium * v0.0.57 * Improve documentation information on api endpoints -- Antoine R. Dumont (@ardumont) Thu, 19 Jan 2017 13:32:56 +0100 swh-web (0.0.56-1~swh1) unstable-swh; urgency=medium * v0.0.56 * Add abilities to display multiple examples on each doc endpoint. -- Antoine R. Dumont (@ardumont) Wed, 18 Jan 2017 14:43:58 +0100 swh-web (0.0.55-1~swh1) unstable-swh; urgency=medium * v0.0.55 * api /content/search/ to /content/known/ * Adapt return values to empty list/dict instead of null * Remove empty values when mono-values are null * Fix broken entity endpoint * Update upcoming endpoints * apidoc: Remove hard-coded example and provide links to follow -- Antoine R. Dumont (@ardumont) Wed, 18 Jan 2017 11:27:45 +0100 swh-web (0.0.54-1~swh1) unstable-swh; urgency=medium * v0.0.54 * Improve documentation description and browsability * Fix css style -- Antoine R. Dumont (@ardumont) Mon, 16 Jan 2017 17:18:21 +0100 swh-web (0.0.53-1~swh1) unstable-swh; urgency=medium * v0.0.53 * apidoc: Update upcoming and hidden endpoints information * apidoc: Enrich route information with tags * apidoc: /api/1/revision/origin/log/: Add pagination explanation * apidoc: /api/1/revision/log/: Add pagination explanation * api: Fix filtering fields to work in depth -- Antoine R. Dumont (@ardumont) Fri, 13 Jan 2017 17:33:01 +0100 swh-web (0.0.52-1~swh1) unstable-swh; urgency=medium * v0.0.52 * Fix doc generation regarding arg and exception * Fix broken examples * Add missing documentation on not found origin visit -- Antoine R. Dumont (@ardumont) Thu, 12 Jan 2017 17:38:59 +0100 swh-web (0.0.51-1~swh1) unstable-swh; urgency=medium * v0.0.51 * Update configuration file from ini to yml -- Antoine R. Dumont (@ardumont) Fri, 16 Dec 2016 13:27:08 +0100 swh-web (0.0.50-1~swh1) unstable-swh; urgency=medium * v0.0.50 * Fix issue regarding data structure change in ctags' reading api endpoint -- Antoine R. Dumont (@ardumont) Tue, 06 Dec 2016 16:08:01 +0100 swh-web (0.0.49-1~swh1) unstable-swh; urgency=medium * v0.0.49 * Rendering improvments -- Antoine R. Dumont (@ardumont) Thu, 01 Dec 2016 16:29:31 +0100 swh-web (0.0.48-1~swh1) unstable-swh; urgency=medium * v0.0.48 * Fix api doc example to actual existing data * Improve search symbol view experience -- Antoine R. Dumont (@ardumont) Thu, 01 Dec 2016 15:32:44 +0100 swh-web (0.0.47-1~swh1) unstable-swh; urgency=medium * v0.0.47 * Improve search content ui (add datatable) * Improve search symbol ui (add datatable without pagination, with * multi-field search) * Split those views to improve readability -- Antoine R. Dumont (@ardumont) Thu, 01 Dec 2016 11:57:16 +0100 swh-web (0.0.46-1~swh1) unstable-swh; urgency=medium * v0.0.46 * Improve search output view on symbols -- Antoine R. Dumont (@ardumont) Wed, 30 Nov 2016 17:45:40 +0100 swh-web (0.0.45-1~swh1) unstable-swh; urgency=medium * v0.0.45 * Migrate search symbol api endpoint to strict equality search * Improve search symbol view result (based on that api) to navigate * through result * Permit to slice result per page with per page flag (limited to 100) * Unify behavior in renderer regarding pagination computation -- Antoine R. Dumont (@ardumont) Wed, 30 Nov 2016 11:00:49 +0100 swh-web (0.0.44-1~swh1) unstable-swh; urgency=medium * v0.0.44 * Rename appropriately /api/1/symbol to /api/1/content/symbol/ * Improve documentation on /api/1/content/symbol/ api endpoint -- Antoine R. Dumont (@ardumont) Tue, 29 Nov 2016 15:00:14 +0100 swh-web (0.0.43-1~swh1) unstable-swh; urgency=medium * v0.0.43 * Improve edge case when looking for ctags symbols * Add a lookup ui to search through symbols -- Antoine R. Dumont (@ardumont) Mon, 28 Nov 2016 16:42:33 +0100 swh-web (0.0.42-1~swh1) unstable-swh; urgency=medium * v0.0.42 * List ctags line as link to content in /browse/content/ view -- Antoine R. Dumont (@ardumont) Fri, 25 Nov 2016 16:21:12 +0100 swh-web (0.0.41-1~swh1) unstable-swh; urgency=medium * v0.0.41 * Improve browse content view by: * adding new information (license, mimetype, language) * highlighting source code -- Antoine R. Dumont (@ardumont) Fri, 25 Nov 2016 14:52:34 +0100 swh-web (0.0.40-1~swh1) unstable-swh; urgency=medium * v0.0.40 * Add pagination to symbol search endpoint -- Antoine R. Dumont (@ardumont) Thu, 24 Nov 2016 14:23:45 +0100 swh-web (0.0.39-1~swh1) unstable-swh; urgency=medium * v0.0.39 * Open /api/1/symbol// * Fix api breaking on /api/1/content/search/ -- Antoine R. Dumont (@ardumont) Thu, 24 Nov 2016 10:28:42 +0100 swh-web (0.0.38-1~swh1) unstable-swh; urgency=medium * v0.0.38 * Minor refactoring * Remove one commit which breaks production -- Antoine R. Dumont (@ardumont) Tue, 22 Nov 2016 16:26:03 +0100 swh-web (0.0.37-1~swh1) unstable-swh; urgency=medium * v0.0.37 * api: Open new endpoints on license, language, filetype * api: Update content endpoint to add url on new endpoints -- Antoine R. Dumont (@ardumont) Tue, 22 Nov 2016 15:04:07 +0100 swh-web (0.0.36-1~swh1) unstable-swh; urgency=medium * v0.0.36 * Adapt to latest origin_visit format -- Antoine R. Dumont (@ardumont) Thu, 08 Sep 2016 15:24:33 +0200 swh-web (0.0.35-1~swh1) unstable-swh; urgency=medium * v0.0.35 * Open /api/1/provenance// api endpoint * Open /api/1/origin//visits/() api endpoint * View: Fix redirection url issue -- Antoine R. Dumont (@ardumont) Mon, 05 Sep 2016 14:28:33 +0200 swh-web (0.0.34-1~swh1) unstable-swh; urgency=medium * v0.0.34 * Improve global ui navigation * Fix apidoc rendering issue * Open /api/1/provenance/ about content provenant information -- Antoine R. Dumont (@ardumont) Fri, 02 Sep 2016 11:42:04 +0200 swh-web (0.0.33-1~swh1) unstable-swh; urgency=medium * Release swh.web.ui v0.0.33 * New declarative API documentation mechanisms -- Nicolas Dandrimont Wed, 24 Aug 2016 16:25:24 +0200 swh-web (0.0.32-1~swh1) unstable-swh; urgency=medium * v0.0.32 * Activate tests during debian packaging * Fix issues on debian packaging * Fix useless jquery loading url * Improve date time parsing -- Antoine R. Dumont (@ardumont) Wed, 20 Jul 2016 12:35:09 +0200 swh-web (0.0.31-1~swh1) unstable-swh; urgency=medium * v0.0.31 * Unify jquery-flot library names with .min -- Antoine R. Dumont (@ardumont) Mon, 18 Jul 2016 11:11:59 +0200 swh-web (0.0.30-1~swh1) unstable-swh; urgency=medium * v0.0.30 * View: Open calendar ui view on origin * API: open /api/1/stat/visits// -- Antoine R. Dumont (@ardumont) Wed, 13 Jul 2016 18:42:40 +0200 swh-web (0.0.29-1~swh1) unstable-swh; urgency=medium * Release swh.web.ui v0.0.29 * All around enhancements of the web ui * Package now tested when building -- Nicolas Dandrimont Tue, 14 Jun 2016 17:58:42 +0200 swh-web (0.0.28-1~swh1) unstable-swh; urgency=medium * v0.0.28 * Fix packaging issues -- Antoine R. Dumont (@ardumont) Mon, 09 May 2016 16:21:04 +0200 swh-web (0.0.27-1~swh1) unstable-swh; urgency=medium * v0.0.27 * Fix packaging issue -- Antoine R. Dumont (@ardumont) Tue, 03 May 2016 16:52:40 +0200 swh-web (0.0.24-1~swh1) unstable-swh; urgency=medium * Release swh.web.ui v0.0.24 * New swh.storage API for timestamps -- Nicolas Dandrimont Fri, 05 Feb 2016 12:07:33 +0100 swh-web (0.0.23-1~swh1) unstable-swh; urgency=medium * v0.0.23 * Bump dependency requirements to latest swh.storage * Returns person's identifier on api + Hide person's emails in views endpoint * Try to decode the content's raw data and fail gracefully * Unify /directory api to Display content's raw data when path resolves to a file * Expose unconditionally the link to download the content's raw data * Download link data redirects to the api ones -- Antoine R. Dumont (@ardumont) Fri, 29 Jan 2016 17:50:31 +0100 swh-web (0.0.22-1~swh1) unstable-swh; urgency=medium * v0.0.22 * Open /browse/revision/origin/[/branch/][/ts/] /history// view * Open /browse/revision/origin/[/branch/][/ts/] / view * Open /browse/revision//history//directory/[] view * Open /browse/revision/origin/[/branch/][/ts/] /history//directory/[] view * Open /browse/revision/origin/[/branch/][/ts/] /directory/[] view * Open /browse/revision//directory// view * Open /browse/revision//history// view * Open /browse/revision//log/ view * Open /browse/entity// view * Release can point to other objects than revision * Fix misbehavior when retrieving git log * Fix another edge case when listing a directory that does not exist * Fix edge case when listing is empty * Fix person_get call * Update documentation about possible error codes -- Antoine R. Dumont (@ardumont) Tue, 26 Jan 2016 15:14:35 +0100 swh-web (0.0.21-1~swh1) unstable-swh; urgency=medium * v0.0.21 * Deal nicely with communication downtime with storage * Update to latest swh.storage api -- Antoine R. Dumont (@ardumont) Wed, 20 Jan 2016 16:31:34 +0100 swh-web (0.0.20-1~swh1) unstable-swh; urgency=medium * v0.0.20 * Open /api/1/entity// -- Antoine R. Dumont (@ardumont) Fri, 15 Jan 2016 16:40:56 +0100 swh-web (0.0.19-1~swh1) unstable-swh; urgency=medium * v0.0.19 * Improve directory_get_by_path integration with storage * Refactor - Only lookup sha1_git_root if needed + factorize service behavior -- Antoine R. Dumont (@ardumont) Fri, 15 Jan 2016 12:47:39 +0100 swh-web (0.0.18-1~swh1) unstable-swh; urgency=medium * v0.0.18 * Open /api/1/revision/origin/[/branch/][/ts/]/ history//directory/[] * origin/master Open /api/1/revision/origin/[/branch/][/ts/]/ history// * Open /api/1/revision/origin/[/branch/][/ts/]/ directory/[] * Open /api/1/revision/origin//branch//ts// * /directory/ apis can now point to files too. * Bump dependency requirement on latest swh.storage * Deactivate api querying occurrences for now * Improve function documentation -- Antoine R. Dumont (@ardumont) Wed, 13 Jan 2016 12:54:54 +0100 swh-web (0.0.17-1~swh1) unstable-swh; urgency=medium * v0.0.17 * Open /api/1/revision//directory/' * Open /api/1/revision//history//directory/ / * Enrich directory listing with url to next subdir * Improve testing coverage * Open 'limit' get query parameter to revision_log and revision_history api -- Antoine R. Dumont (@ardumont) Fri, 08 Jan 2016 11:36:55 +0100 swh-web (0.0.16-1~swh1) unstable-swh; urgency=medium * v0.0.16 * service.lookup_revision_log: Add a limit to the number of commits * Fix docstring rendering -- Antoine R. Dumont (@ardumont) Wed, 06 Jan 2016 15:37:21 +0100 swh-web (0.0.15-1~swh1) unstable-swh; urgency=medium * v0.0.15 * Improve browsable api rendering style * Fix typo in jquery.min.js link * Fix docstring typos * packaging: * add python3-flask-api as package dependency -- Antoine R. Dumont (@ardumont) Wed, 06 Jan 2016 15:12:04 +0100 swh-web (0.0.14-1~swh1) unstable-swh; urgency=medium * v0.0.14 * Open /revision//history// * Add links to api * Improve browsable api rendering -> when api links exists, actual html links will be displayed * Fix production bugs (regarding browsable api) -- Antoine R. Dumont (@ardumont) Wed, 06 Jan 2016 11:42:18 +0100 swh-web (0.0.13-1~swh1) unstable-swh; urgency=medium * v0.0.13 * Open /browse/person/ view * Open /browse/origin/ view * Open /browse/release/ view * Open /browse/revision/ view * Deactivate temporarily /browse/content/ * Add default sha1 * Automatic doc endpoint on base path -- Antoine R. Dumont (@ardumont) Tue, 15 Dec 2015 17:01:27 +0100 swh-web (0.0.12-1~swh1) unstable-swh; urgency=medium * v0.0.12 * Update /api/1/release/ with latest internal standard * Update /api/1/revision/ with latest internal standard * Add global filtering on 'fields' parameter * Update /api/1/content/ with links to raw resource * Improve documentations * Open /api/1/revision//log/ * Open /browse/directory/ to list directory content * Open /browse/content// to show the content * Open /browse/content//raw to show the content * Open /api/1/person/ * Implementation detail * Add Flask API dependency * Split controller in api and views module * Unify internal apis' behavior -- Antoine R. Dumont (@ardumont) Mon, 07 Dec 2015 16:44:43 +0100 swh-web (0.0.11-1~swh1) unstable-swh; urgency=medium * v0.0.11 * Open /1/api/content// * Open /api/1/revision/ * Open /api/1/release/ * Open /api/1/uploadnsearch/ (POST) * Open /api/1/origin/ * Unify 404 and 400 responses on api * Increase code coverage -- Antoine R. Dumont (@ardumont) Thu, 19 Nov 2015 11:24:46 +0100 swh-web (0.0.10-1~swh1) unstable-swh; urgency=medium * v0.0.10 * set document.domain to parent domain softwareheritage.org * improve HTML templates to be (more) valid * cosmetic change in Content-Type JSON header -- Stefano Zacchiroli Mon, 02 Nov 2015 13:59:45 +0100 swh-web (0.0.9-1~swh1) unstable-swh; urgency=medium * v0.0.9 * Remove query entry in api response * Deal with bad request properly with api calls * Improve coverage * Improve dev starting up app * Fix duplicated print statement in dev app startup -- Antoine R. Dumont (@ardumont) Fri, 30 Oct 2015 17:24:15 +0100 swh-web (0.0.8-1~swh1) unstable-swh; urgency=medium * version 0.0.8 -- Stefano Zacchiroli Wed, 28 Oct 2015 20:59:40 +0100 swh-web (0.0.7-1~swh1) unstable-swh; urgency=medium * v0.0.7 * Add @jsonp abilities to /api/1/stat/counters endpoint -- Antoine R. Dumont (@ardumont) Mon, 19 Oct 2015 14:01:40 +0200 swh-web (0.0.4-1~swh1) unstable-swh; urgency=medium * Prepare swh.web.ui v0.0.4 deployment -- Nicolas Dandrimont Fri, 16 Oct 2015 15:38:44 +0200 swh-web (0.0.3-1~swh1) unstable-swh; urgency=medium * Prepare deployment of swh-web-ui v0.0.3 -- Nicolas Dandrimont Wed, 14 Oct 2015 11:09:33 +0200 swh-web (0.0.2-1~swh1) unstable-swh; urgency=medium * Prepare swh.web.ui v0.0.2 deployment -- Nicolas Dandrimont Tue, 13 Oct 2015 16:25:46 +0200 swh-web (0.0.1-1~swh1) unstable-swh; urgency=medium * Initial release * v0.0.1 * Hash lookup to check existence in swh's backend * Hash lookup to detail a content -- Antoine R. Dumont (@ardumont) Thu, 01 Oct 2015 10:01:29 +0200 diff --git a/swh.web.egg-info/PKG-INFO b/swh.web.egg-info/PKG-INFO index 394f731f..f8495fe3 100644 --- a/swh.web.egg-info/PKG-INFO +++ b/swh.web.egg-info/PKG-INFO @@ -1,127 +1,127 @@ Metadata-Version: 2.1 Name: swh.web -Version: 0.0.163 +Version: 0.0.164 Summary: Software Heritage Web UI Home-page: https://forge.softwareheritage.org/diffusion/DWUI/ Author: Software Heritage developers Author-email: swh-devel@inria.fr License: UNKNOWN Project-URL: Bug Reports, https://forge.softwareheritage.org/maniphest Project-URL: Funding, https://www.softwareheritage.org/donate Project-URL: Source, https://forge.softwareheritage.org/source/swh-web Description: # swh-web This repository holds the development of Software Heritage web applications: * SWH Web API (https://archive.softwareheritage.org/api): enables to query the content of the SWH archive through HTTP requests and get responses in JSON or YAML. * SWH Web browse (https://archive.softwareheritage.org/browse): graphical interface that eases the navigation in the SWH archive. Documentation about how to use these components but also the details of their URI schemes can be found in the docs folder. The produced HTML documentation can be read and browsed at https://docs.softwareheritage.org/devel/swh-web/index.html. ## Technical details Those applications are powered by: * [Django Web Framework](https://www.djangoproject.com/) on the backend side with the following extensions enabled: * [django-rest-framework](http://www.django-rest-framework.org/) * [django-webpack-loader](https://github.com/owais/django-webpack-loader) * [django-js-reverse](http://django-js-reverse.readthedocs.io/en/latest/) * [webpack](https://webpack.js.org/) on the frontend side for better static assets management, including: * assets dependencies management and retrieval through [npm](https://www.npmjs.com/) * linting of custom javascript code (through [eslint](https://eslint.org/)) and stylesheets (through [stylelint](https://stylelint.io/)) * use of [es6](http://es6-features.org) syntax and advanced javascript feature like [async/await](https://javascript.info/async-await) or [fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) thanks to [babel](https://babeljs.io/) (es6 to es5 transpiler and polyfills provider) * assets minification (using [UglifyJS](https://github.com/mishoo/UglifyJS2) and [cssnano](http://cssnano.co/)) but also dead code elimination for production use ## How to build and run ### Requirements First you will need [Python 3](https://www.python.org) and a complete [swh development environment](https://forge.softwareheritage.org/source/swh-environment/) installed. To run the backend, you need to have the following Python 3 modules installed: * beautifulsoup4 * django >= 1.10.7 * djangorestframework >= 3.4.0 * django_webpack_loader * django_js_reverse * docutils * file_magic >= 0.3.0 * htmlmin * lxml * pygments * pypandoc * python-dateutil * pyyaml * requests To compile the frontend assets, you need to have [nodejs](https://nodejs.org/en/) >= 8.x (preferably version 8.x LTS) and [npm](https://www.npmjs.com/) (or [yarn](https://yarnpkg.com/en/)) installed. If you are on Debian stretch, you can easily install an up to date nodejs/npm from the stretch-backports repository or by following the instructions located at https://github.com/nodesource/distributions. Once you have installed nodejs, issue the following command in the root directory of swh-web in order to retrieve all the frontend dependencies: ``` $ npm install ``` or if you prefer to use yarn (faster than npm): ``` $ yarn install ``` Please note that the static assets bundles generated by webpack are not stored in the git repository. Follow the instructions below in order to generate them in order to be able to run the frontend part of the web applications. ### Make targets Below is the list of available make targets that can be executed from the root directory of swh-web in order to build and/or execute the web applications under various configurations: * **run-django-webpack-devserver**: Compile and serve not optimized (without mignification and dead code elimination) frontend static assets using [webpack-dev-server](https://github.com/webpack/webpack-dev-server) and run django server with development settings. This is the recommended target to use when developing swh-web as it enables automatic reloading of backend and frontend part of the applications when modifying source files (*.py, *.js, *.css, *.html). * **run-django-webpack-dev**: Compile not optimized (no minification, no dead code elimination) frontend static assets using webpack and run django server with development settings. This is the recommended target when one only wants to develop the backend side of the application. * **run-django-webpack-prod**: Compile optimized (with minification and dead code elimination) frontend static assets using webpack and run django server with production settings. This is useful to test the applications in production mode (with the difference that static assets are served by django). Production settings notably enable advanced django caching and you will need to have [memcached](https://memcached.org/) installed for that feature to work. * **run-django-server-dev**: Run the django server with development settings but without compiling frontend static assets through webpack. * **run-django-server-prod**: Run the django server with production settings but without compiling frontend static assets through webpack. * **run-gunicorn-server**: Run the web applications with production settings in a [gunicorn](http://gunicorn.org/) worker as they will be in real production environment. Once one of these targets executed, the web applications can be executed by pointing your browser to http://localhost:5004. ### Npm/Yarn targets Below is a list of available npm/yarn targets in order to only execute the frontend static assets compilation (no web server will be executed): * **build-dev**: compile not optimized (without mignification and dead code elimination) frontend static assets and store the results in the `swh/web/static` folder. * **build**: compile optimized (with mignification and dead code elimination) frontend static assets and store the results in the `swh/web/static` folder. **The build target must be executed prior performing the Debian packaging of swh-web** in order for the package to contain the optimized assets dedicated to production environment. To execute these targets, issue the following commmand: ``` $ npm run ``` or if you prefer using yarn: ``` $ yarn ``` Platform: UNKNOWN Classifier: Programming Language :: Python :: 3 Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+) Classifier: Operating System :: OS Independent Classifier: Development Status :: 5 - Production/Stable Classifier: Framework :: Django Description-Content-Type: text/markdown Provides-Extra: testing diff --git a/swh/web/browse/views/directory.py b/swh/web/browse/views/directory.py index 124c5794..6d42a71b 100644 --- a/swh/web/browse/views/directory.py +++ b/swh/web/browse/views/directory.py @@ -1,153 +1,155 @@ # Copyright (C) 2017-2018 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU Affero General Public License version 3, or any later version # See top-level LICENSE file for more information from django.shortcuts import render, redirect from django.template.defaultfilters import filesizeformat from swh.web.common import service from swh.web.common.utils import ( reverse, gen_path_info ) from swh.web.common.exc import handle_view_exception, NotFoundExc from swh.web.browse.utils import ( get_directory_entries, get_snapshot_context, get_readme_to_display, get_swh_persistent_ids, gen_link ) from swh.web.browse.browseurls import browse_route @browse_route(r'directory/(?P[0-9a-f]+)/', r'directory/(?P[0-9a-f]+)/(?P.+)/', view_name='browse-directory') def directory_browse(request, sha1_git, path=None): """Django view for browsing the content of a SWH directory identified by its sha1_git value. The url that points to it is :http:get:`/browse/directory/(sha1_git)/[(path)/]` """ # noqa root_sha1_git = sha1_git try: if path: dir_info = service.lookup_directory_with_path(sha1_git, path) # some readme files can reference assets reachable from the # browsed directory, handle that special case in order to # correctly displayed them if dir_info and dir_info['type'] == 'file': file_raw_url = reverse( 'browse-content-raw', kwargs={'query_string': dir_info['checksums']['sha1']}) return redirect(file_raw_url) sha1_git = dir_info['target'] dirs, files = get_directory_entries(sha1_git) origin_type = request.GET.get('origin_type', None) origin_url = request.GET.get('origin_url', None) if not origin_url: origin_url = request.GET.get('origin', None) snapshot_context = None if origin_url: try: snapshot_context = get_snapshot_context(None, origin_type, origin_url) except Exception: raw_dir_url = reverse('browse-directory', kwargs={'sha1_git': sha1_git}) error_message = \ ('The Software Heritage archive has a directory ' 'with the hash you provided but the origin ' 'mentioned in your request appears broken: %s. ' 'Please check the URL and try again.\n\n' 'Nevertheless, you can still browse the directory ' 'without origin information: %s' % (gen_link(origin_url), gen_link(raw_dir_url))) raise NotFoundExc(error_message) if snapshot_context: snapshot_context['visit_info'] = None except Exception as exc: return handle_view_exception(request, exc) path_info = gen_path_info(path) breadcrumbs = [] breadcrumbs.append({'name': root_sha1_git[:7], 'url': reverse('browse-directory', kwargs={'sha1_git': root_sha1_git})}) for pi in path_info: breadcrumbs.append({'name': pi['name'], 'url': reverse('browse-directory', kwargs={'sha1_git': root_sha1_git, 'path': pi['path']})}) path = '' if path is None else (path + '/') for d in dirs: if d['type'] == 'rev': d['url'] = reverse('browse-revision', kwargs={'sha1_git': d['target']}) else: d['url'] = reverse('browse-directory', kwargs={'sha1_git': root_sha1_git, 'path': path + d['name']}) sum_file_sizes = 0 readmes = {} for f in files: + if f['length'] is None: + continue query_string = 'sha1_git:' + f['target'] f['url'] = reverse('browse-content', kwargs={'query_string': query_string}, query_params={'path': root_sha1_git + '/' + path + f['name']}) sum_file_sizes += f['length'] f['length'] = filesizeformat(f['length']) if f['name'].lower().startswith('readme'): readmes[f['name']] = f['checksums']['sha1'] readme_name, readme_url, readme_html = get_readme_to_display(readmes) sum_file_sizes = filesizeformat(sum_file_sizes) dir_metadata = {'id': sha1_git, 'number of regular files': len(files), 'number of subdirectories': len(dirs), 'sum of regular file sizes': sum_file_sizes} vault_cooking = { 'directory_context': True, 'directory_id': sha1_git, 'revision_context': False, 'revision_id': None } swh_ids = get_swh_persistent_ids([{'type': 'directory', 'id': sha1_git}]) heading = 'Directory - %s' % sha1_git if breadcrumbs: dir_path = '/'.join([bc['name'] for bc in breadcrumbs]) + '/' heading += ' - %s' % dir_path return render(request, 'browse/directory.html', {'heading': heading, 'swh_object_id': swh_ids[0]['swh_id'], 'swh_object_name': 'Directory', 'swh_object_metadata': dir_metadata, 'dirs': dirs, 'files': files, 'breadcrumbs': breadcrumbs, 'top_right_link': None, 'top_right_link_text': None, 'readme_name': readme_name, 'readme_url': readme_url, 'readme_html': readme_html, 'snapshot_context': snapshot_context, 'vault_cooking': vault_cooking, 'show_actions_menu': True, 'swh_ids': swh_ids}) diff --git a/swh/web/browse/views/revision.py b/swh/web/browse/views/revision.py index 704dd4e5..0729aea4 100644 --- a/swh/web/browse/views/revision.py +++ b/swh/web/browse/views/revision.py @@ -1,547 +1,549 @@ # Copyright (C) 2017-2018 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU Affero General Public License version 3, or any later version # See top-level LICENSE file for more information import hashlib import json import textwrap from django.http import HttpResponse from django.shortcuts import render, redirect from django.template.defaultfilters import filesizeformat from django.utils.safestring import mark_safe from swh.model.identifiers import persistent_identifier from swh.web.common import service from swh.web.common.utils import ( reverse, format_utc_iso_date, gen_path_info ) from swh.web.common.exc import NotFoundExc, handle_view_exception from swh.web.browse.browseurls import browse_route from swh.web.browse.utils import ( gen_link, gen_person_link, gen_revision_link, prepare_revision_log_for_display, get_snapshot_context, gen_snapshot_directory_link, get_revision_log_url, get_directory_entries, gen_directory_link, request_content, prepare_content_for_display, content_display_max_size, gen_snapshot_link, get_readme_to_display, get_swh_persistent_ids ) def _gen_content_url(revision, query_string, path, snapshot_context): if snapshot_context: url_args = snapshot_context['url_args'] url_args['path'] = path query_params = snapshot_context['query_params'] query_params['revision'] = revision['id'] content_url = reverse('browse-origin-content', kwargs=url_args, query_params=query_params) else: content_path = '%s/%s' % (revision['directory'], path) content_url = reverse('browse-content', kwargs={'query_string': query_string}, query_params={'path': content_path}) return content_url def _gen_diff_link(idx, diff_anchor, link_text): if idx < _max_displayed_file_diffs: return gen_link(diff_anchor, link_text) else: return link_text # TODO: put in conf _max_displayed_file_diffs = 1000 def _gen_revision_changes_list(revision, changes, snapshot_context): """ Returns a HTML string describing the file changes introduced in a revision. As this string will be displayed in the browse revision view, links to adequate file diffs are also generated. Args: revision (str): hexadecimal representation of a revision identifier changes (list): list of file changes in the revision snapshot_context (dict): optional origin context used to reverse the content urls Returns: A string to insert in a revision HTML view. """ changes_msg = [] for i, change in enumerate(changes): hasher = hashlib.sha1() from_query_string = '' to_query_string = '' diff_id = 'diff-' if change['from']: from_query_string = 'sha1_git:' + change['from']['target'] diff_id += change['from']['target'] + '-' + change['from_path'] diff_id += '-' if change['to']: to_query_string = 'sha1_git:' + change['to']['target'] diff_id += change['to']['target'] + change['to_path'] change['path'] = change['to_path'] or change['from_path'] url_args = {'from_query_string': from_query_string, 'to_query_string': to_query_string} query_params = {'path': change['path']} change['diff_url'] = reverse('diff-contents', kwargs=url_args, query_params=query_params) hasher.update(diff_id.encode('utf-8')) diff_id = hasher.hexdigest() change['id'] = diff_id panel_diff_link = '#panel_' + diff_id if change['type'] == 'modify': change['content_url'] = \ _gen_content_url(revision, to_query_string, change['to_path'], snapshot_context) changes_msg.append('modified: %s' % _gen_diff_link(i, panel_diff_link, change['to_path'])) elif change['type'] == 'insert': change['content_url'] = \ _gen_content_url(revision, to_query_string, change['to_path'], snapshot_context) changes_msg.append('new file: %s' % _gen_diff_link(i, panel_diff_link, change['to_path'])) elif change['type'] == 'delete': parent = service.lookup_revision(revision['parents'][0]) change['content_url'] = \ _gen_content_url(parent, from_query_string, change['from_path'], snapshot_context) changes_msg.append('deleted: %s' % _gen_diff_link(i, panel_diff_link, change['from_path'])) elif change['type'] == 'rename': change['content_url'] = \ _gen_content_url(revision, to_query_string, change['to_path'], snapshot_context) link_text = change['from_path'] + ' → ' + change['to_path'] changes_msg.append('renamed: %s' % _gen_diff_link(i, panel_diff_link, link_text)) if not changes: changes_msg.append('No changes') return mark_safe('\n'.join(changes_msg)) @browse_route(r'revision/(?P[0-9a-f]+)/diff/', view_name='diff-revision') def _revision_diff(request, sha1_git): """ Browse internal endpoint to compute revision diff """ try: revision = service.lookup_revision(sha1_git) snapshot_context = None origin_type = request.GET.get('origin_type', None) origin_url = request.GET.get('origin_url', None) if not origin_url: origin_url = request.GET.get('origin', None) timestamp = request.GET.get('timestamp', None) visit_id = request.GET.get('visit_id', None) if origin_url: snapshot_context = get_snapshot_context(None, origin_type, origin_url, timestamp, visit_id) except Exception as exc: return handle_view_exception(request, exc) changes = service.diff_revision(sha1_git) changes_msg = _gen_revision_changes_list(revision, changes, snapshot_context) diff_data = { 'total_nb_changes': len(changes), 'changes': changes[:_max_displayed_file_diffs], 'changes_msg': changes_msg } diff_data_json = json.dumps(diff_data, separators=(',', ': ')) return HttpResponse(diff_data_json, content_type='application/json') NB_LOG_ENTRIES = 20 @browse_route(r'revision/(?P[0-9a-f]+)/log/', view_name='browse-revision-log') def revision_log_browse(request, sha1_git): """ Django view that produces an HTML display of the history log for a SWH revision identified by its id. The url that points to it is :http:get:`/browse/revision/(sha1_git)/log/`. """ # noqa try: per_page = int(request.GET.get('per_page', NB_LOG_ENTRIES)) revision_log = service.lookup_revision_log(sha1_git, limit=per_page+1) revision_log = list(revision_log) except Exception as exc: return handle_view_exception(request, exc) revs_breadcrumb = request.GET.get('revs_breadcrumb', None) revision_log_display_data = prepare_revision_log_for_display( revision_log, per_page, revs_breadcrumb) prev_rev = revision_log_display_data['prev_rev'] prev_revs_breadcrumb = revision_log_display_data['prev_revs_breadcrumb'] prev_log_url = None if prev_rev: prev_log_url = \ reverse('browse-revision-log', kwargs={'sha1_git': prev_rev}, query_params={'revs_breadcrumb': prev_revs_breadcrumb, 'per_page': per_page}) next_rev = revision_log_display_data['next_rev'] next_revs_breadcrumb = revision_log_display_data['next_revs_breadcrumb'] next_log_url = None if next_rev: next_log_url = \ reverse('browse-revision-log', kwargs={'sha1_git': next_rev}, query_params={'revs_breadcrumb': next_revs_breadcrumb, 'per_page': per_page}) revision_log_data = revision_log_display_data['revision_log_data'] for log in revision_log_data: log['directory'] = gen_directory_link( log['directory'], link_text='Browse files', link_attrs={'class': 'btn btn-default btn-sm', 'role': 'button'}) swh_rev_id = persistent_identifier('revision', sha1_git) return render(request, 'browse/revision-log.html', {'heading': 'Revision history', 'swh_object_id': swh_rev_id, 'swh_object_name': 'Revision history', 'swh_object_metadata': None, 'revision_log': revision_log_data, 'next_log_url': next_log_url, 'prev_log_url': prev_log_url, 'breadcrumbs': None, 'top_right_link': None, 'top_right_link_text': None, 'snapshot_context': None, 'vault_cooking': None, 'show_actions_menu': True, 'swh_ids': None}) @browse_route(r'revision/(?P[0-9a-f]+)/', r'revision/(?P[0-9a-f]+)/(?P.+)/', view_name='browse-revision') def revision_browse(request, sha1_git, extra_path=None): """ Django view that produces an HTML display of a SWH revision identified by its id. The url that points to it is :http:get:`/browse/revision/(sha1_git)/`. """ try: revision = service.lookup_revision(sha1_git) # some readme files can reference assets reachable from the # browsed directory, handle that special case in order to # correctly displayed them if extra_path: dir_info = \ service.lookup_directory_with_path(revision['directory'], extra_path) if dir_info and dir_info['type'] == 'file': file_raw_url = reverse( 'browse-content-raw', kwargs={'query_string': dir_info['checksums']['sha1']}) return redirect(file_raw_url) origin_info = None snapshot_context = None origin_type = request.GET.get('origin_type', None) origin_url = request.GET.get('origin_url', None) if not origin_url: origin_url = request.GET.get('origin', None) timestamp = request.GET.get('timestamp', None) visit_id = request.GET.get('visit_id', None) snapshot_id = request.GET.get('snapshot_id', None) path = request.GET.get('path', None) dir_id = None dirs, files = None, None content_data = None if origin_url: try: snapshot_context = get_snapshot_context(None, origin_type, origin_url, timestamp, visit_id) except Exception: raw_rev_url = reverse('browse-revision', kwargs={'sha1_git': sha1_git}) error_message = \ ('The Software Heritage archive has a revision ' 'with the hash you provided but the origin ' 'mentioned in your request appears broken: %s. ' 'Please check the URL and try again.\n\n' 'Nevertheless, you can still browse the revision ' 'without origin information: %s' % (gen_link(origin_url), gen_link(raw_rev_url))) raise NotFoundExc(error_message) origin_info = snapshot_context['origin_info'] snapshot_id = snapshot_context['snapshot_id'] elif snapshot_id: snapshot_context = get_snapshot_context(snapshot_id) if path: file_info = \ service.lookup_directory_with_path(revision['directory'], path) if file_info['type'] == 'dir': dir_id = file_info['target'] else: query_string = 'sha1_git:' + file_info['target'] content_data = request_content(query_string, raise_if_unavailable=False) else: dir_id = revision['directory'] if dir_id: path = '' if path is None else (path + '/') dirs, files = get_directory_entries(dir_id) except Exception as exc: return handle_view_exception(request, exc) revision_data = {} author_name = 'None' revision_data['author'] = 'None' if revision['author']: author_name = revision['author']['name'] or \ revision['author']['fullname'] revision_data['author'] = \ gen_person_link(revision['author']['id'], author_name, snapshot_context) revision_data['committer'] = 'None' if revision['committer']: revision_data['committer'] = \ gen_person_link(revision['committer']['id'], revision['committer']['name'], snapshot_context) revision_data['committer date'] = format_utc_iso_date( revision['committer_date']) revision_data['date'] = format_utc_iso_date(revision['date']) if snapshot_context: revision_data['snapshot id'] = snapshot_id revision_data['directory'] = \ gen_snapshot_directory_link(snapshot_context, sha1_git, link_text='Browse', link_attrs={'class': 'btn btn-default btn-sm', # noqa 'role': 'button'}) else: revision_data['directory'] = \ gen_directory_link(revision['directory'], link_text='Browse', link_attrs={'class': 'btn btn-default btn-sm', 'role': 'button'}) revision_data['id'] = sha1_git revision_data['merge'] = revision['merge'] revision_data['metadata'] = json.dumps(revision['metadata'], sort_keys=True, indent=4, separators=(',', ': ')) if origin_info: revision_data['context-independent revision'] = \ gen_revision_link(sha1_git, link_text='Browse', link_attrs={'class': 'btn btn-default btn-sm', 'role': 'button'}) revision_data['origin id'] = origin_info['id'] revision_data['origin type'] = origin_info['type'] revision_data['origin url'] = gen_link(origin_info['url'], origin_info['url']) browse_snapshot_link = \ gen_snapshot_link(snapshot_id, link_text='Browse', link_attrs={'class': 'btn btn-default btn-sm', 'role': 'button'}) revision_data['snapshot'] = browse_snapshot_link parents = '' for p in revision['parents']: parent_link = gen_revision_link(p, snapshot_context=snapshot_context) parents += parent_link + '
' revision_data['parents'] = mark_safe(parents) revision_data['synthetic'] = revision['synthetic'] revision_data['type'] = revision['type'] message_lines = ['None'] if revision['message']: message_lines = revision['message'].split('\n') parents_links = '%s parent%s ' % \ (len(revision['parents']), '' if len(revision['parents']) == 1 else 's') parents_links += ' ' for p in revision['parents']: parent_link = gen_revision_link(p, shorten_id=True, snapshot_context=snapshot_context) parents_links += parent_link if p != revision['parents'][-1]: parents_links += ' + ' path_info = gen_path_info(path) query_params = {'snapshot_id': snapshot_id, 'origin_type': origin_type, 'origin': origin_url, 'timestamp': timestamp, 'visit_id': visit_id} breadcrumbs = [] breadcrumbs.append({'name': revision['directory'][:7], 'url': reverse('browse-revision', kwargs={'sha1_git': sha1_git}, query_params=query_params)}) for pi in path_info: query_params['path'] = pi['path'] breadcrumbs.append({'name': pi['name'], 'url': reverse('browse-revision', kwargs={'sha1_git': sha1_git}, query_params=query_params)}) vault_cooking = { 'directory_context': False, 'directory_id': None, 'revision_context': True, 'revision_id': sha1_git } swh_objects = [{'type': 'revision', 'id': sha1_git}] content = None content_size = None mimetype = None language = None readme_name = None readme_url = None readme_html = None readmes = {} error_code = 200 error_message = '' error_description = '' if content_data: breadcrumbs[-1]['url'] = None content_size = content_data['length'] mimetype = content_data['mimetype'] if content_data['raw_data']: content_display_data = prepare_content_for_display( content_data['raw_data'], content_data['mimetype'], path) content = content_display_data['content_data'] language = content_display_data['language'] query_params = {} if path: query_params['filename'] = path_info[-1]['name'] top_right_link = reverse('browse-content-raw', kwargs={'query_string': query_string}, query_params=query_params) top_right_link_text = mark_safe( 'Raw File') swh_objects.append({'type': 'content', 'id': file_info['target']}) error_code = content_data['error_code'] error_message = content_data['error_message'] error_description = content_data['error_description'] else: for d in dirs: if d['type'] == 'rev': d['url'] = reverse('browse-revision', kwargs={'sha1_git': d['target']}) else: query_params['path'] = path + d['name'] d['url'] = reverse('browse-revision', kwargs={'sha1_git': sha1_git}, query_params=query_params) for f in files: + if f['length'] is None: + continue query_params['path'] = path + f['name'] f['url'] = reverse('browse-revision', kwargs={'sha1_git': sha1_git}, query_params=query_params) f['length'] = filesizeformat(f['length']) if f['name'].lower().startswith('readme'): readmes[f['name']] = f['checksums']['sha1'] readme_name, readme_url, readme_html = get_readme_to_display(readmes) top_right_link = get_revision_log_url(sha1_git, snapshot_context) top_right_link_text = mark_safe( '' 'History') vault_cooking['directory_context'] = True vault_cooking['directory_id'] = dir_id swh_objects.append({'type': 'directory', 'id': dir_id}) diff_revision_url = reverse('diff-revision', kwargs={'sha1_git': sha1_git}, query_params={'origin_type': origin_type, 'origin': origin_url, 'timestamp': timestamp, 'visit_id': visit_id}) if snapshot_id: swh_objects.append({'type': 'snapshot', 'id': snapshot_id}) swh_ids = get_swh_persistent_ids(swh_objects, snapshot_context) heading = 'Revision - %s - %s' %\ (sha1_git[:7], textwrap.shorten(message_lines[0], width=70)) if snapshot_context: context_found = 'snapshot: %s' % snapshot_context['snapshot_id'] if origin_info: context_found = 'origin: %s' % origin_info['url'] heading += ' - %s' % context_found return render(request, 'browse/revision.html', {'heading': heading, 'swh_object_id': swh_ids[0]['swh_id'], 'swh_object_name': 'Revision', 'swh_object_metadata': revision_data, 'message_header': message_lines[0], 'message_body': '\n'.join(message_lines[1:]), 'parents_links': mark_safe(parents_links), 'snapshot_context': snapshot_context, 'dirs': dirs, 'files': files, 'content': content, 'content_size': content_size, 'max_content_size': content_display_max_size, 'mimetype': mimetype, 'language': language, 'readme_name': readme_name, 'readme_url': readme_url, 'readme_html': readme_html, 'breadcrumbs': breadcrumbs, 'top_right_link': top_right_link, 'top_right_link_text': top_right_link_text, 'vault_cooking': vault_cooking, 'diff_revision_url': diff_revision_url, 'show_actions_menu': True, 'swh_ids': swh_ids, 'error_code': error_code, 'error_message': error_message, 'error_description': error_description}, status=error_code) diff --git a/swh/web/browse/views/utils/snapshot_context.py b/swh/web/browse/views/utils/snapshot_context.py index e8ae5f5d..39531244 100644 --- a/swh/web/browse/views/utils/snapshot_context.py +++ b/swh/web/browse/views/utils/snapshot_context.py @@ -1,881 +1,883 @@ # Copyright (C) 2018 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU Affero General Public License version 3, or any later version # See top-level LICENSE file for more information # Utility module implementing Django views for browsing the SWH archive # in a snapshot context. # Its purpose is to factorize code for the views reachable from the # /origin/.* and /snapshot/.* endpoints. from django.shortcuts import render, redirect from django.utils.safestring import mark_safe from django.template.defaultfilters import filesizeformat from swh.web.browse.utils import ( get_snapshot_context, get_directory_entries, gen_directory_link, gen_revision_link, request_content, gen_content_link, prepare_content_for_display, content_display_max_size, prepare_revision_log_for_display, gen_snapshot_directory_link, gen_revision_log_link, gen_link, get_readme_to_display, get_swh_persistent_ids, process_snapshot_branches ) from swh.web.common import service from swh.web.common.exc import ( handle_view_exception, NotFoundExc ) from swh.web.common.utils import ( reverse, gen_path_info, format_utc_iso_date ) def _get_branch(branches, branch_name, snapshot_id): """ Utility function to get a specific branch from a branches list. Its purpose is to get the default HEAD branch as some SWH origin (e.g those with svn type) does not have it. In that latter case, check if there is a master branch instead and returns it. """ filtered_branches = \ [b for b in branches if b['name'].endswith(branch_name)] if len(filtered_branches) > 0: return filtered_branches[0] elif branch_name == 'HEAD': filtered_branches = \ [b for b in branches if b['name'].endswith('master')] if len(filtered_branches) > 0: return filtered_branches[0] elif len(branches) > 0: return branches[0] else: # case where a large branches list has been truncated snp_branch = service.lookup_snapshot(snapshot_id, branches_from=branch_name, branches_count=1, target_types=['revision']) snp_branch, _ = process_snapshot_branches(snp_branch['branches']) if snp_branch: branches.append(snp_branch[0]) return snp_branch[0] return None def _get_release(releases, release_name): """ Utility function to get a specific release from a releases list. Returns None if the release can not be found in the list. """ filtered_releases = \ [r for r in releases if r['name'] == release_name] if len(filtered_releases) > 0: return filtered_releases[0] else: return None def _branch_not_found(branch_type, branch, branches, snapshot_id=None, origin_info=None, timestamp=None, visit_id=None): """ Utility function to raise an exception when a specified branch/release can not be found. """ if branch_type == 'branch': branch_type = 'Branch' branch_type_plural = 'branches' else: branch_type = 'Release' branch_type_plural = 'releases' if snapshot_id and len(branches) == 0: msg = 'Snapshot with id %s has an empty list' \ ' of %s!' % (snapshot_id, branch_type_plural) elif snapshot_id: msg = '%s %s for snapshot with id %s' \ ' not found!' % (branch_type, branch, snapshot_id) elif visit_id and len(branches) == 0: msg = 'Origin with type %s and url %s' \ ' for visit with id %s has an empty list' \ ' of %s!' % (origin_info['type'], origin_info['url'], visit_id, branch_type_plural) elif visit_id: msg = '%s %s associated to visit with' \ ' id %s for origin with type %s and url %s' \ ' not found!' % (branch_type, branch, visit_id, origin_info['type'], origin_info['url']) elif len(branches) == 0: msg = 'Origin with type %s and url %s' \ ' for visit with timestamp %s has an empty list' \ ' of %s!' % (origin_info['type'], origin_info['url'], timestamp, branch_type_plural) else: msg = '%s %s associated to visit with' \ ' timestamp %s for origin with type %s' \ ' and url %s not found!' % (branch_type, branch, timestamp, origin_info['type'], origin_info['url']) raise NotFoundExc(msg) def _process_snapshot_request(request, snapshot_id=None, origin_type=None, origin_url=None, timestamp=None, path=None, browse_context='directory'): """ Utility function to perform common input request processing for snapshot context views. """ visit_id = request.GET.get('visit_id', None) snapshot_context = get_snapshot_context(snapshot_id, origin_type, origin_url, timestamp, visit_id) swh_type = snapshot_context['swh_type'] origin_info = snapshot_context['origin_info'] branches = snapshot_context['branches'] releases = snapshot_context['releases'] url_args = snapshot_context['url_args'] query_params = snapshot_context['query_params'] if snapshot_context['visit_info']: timestamp = format_utc_iso_date(snapshot_context['visit_info']['date'], '%Y-%m-%dT%H:%M:%SZ') browse_view_name = 'browse-' + swh_type + '-' + browse_context root_sha1_git = None revision_id = request.GET.get('revision', None) release_name = request.GET.get('release', None) release_id = None branch_name = None if revision_id: revision = service.lookup_revision(revision_id) root_sha1_git = revision['directory'] branches.append({'name': revision_id, 'revision': revision_id, 'directory': root_sha1_git, 'url': None}) branch_name = revision_id query_params['revision'] = revision_id elif release_name: release = _get_release(releases, release_name) if release: root_sha1_git = release['directory'] revision_id = release['target'] release_id = release['id'] query_params['release'] = release_name else: _branch_not_found("release", release_name, releases, snapshot_id, origin_info, timestamp, visit_id) else: branch_name = request.GET.get('branch', None) if branch_name: query_params['branch'] = branch_name branch = _get_branch(branches, branch_name or 'HEAD', snapshot_context['snapshot_id']) if branch: branch_name = branch['name'] root_sha1_git = branch['directory'] revision_id = branch['revision'] else: _branch_not_found("branch", branch_name, branches, snapshot_id, origin_info, timestamp, visit_id) for b in branches: branch_url_args = dict(url_args) branch_query_params = dict(query_params) branch_query_params['branch'] = b['name'] if path: b['path'] = path branch_url_args['path'] = path b['url'] = reverse(browse_view_name, kwargs=branch_url_args, query_params=branch_query_params) for r in releases: release_url_args = dict(url_args) release_query_params = dict(query_params) release_query_params['release'] = r['name'] if path: r['path'] = path release_url_args['path'] = path r['url'] = reverse(browse_view_name, kwargs=release_url_args, query_params=release_query_params) snapshot_context['query_params'] = query_params snapshot_context['root_sha1_git'] = root_sha1_git snapshot_context['revision_id'] = revision_id snapshot_context['branch'] = branch_name snapshot_context['release'] = release_name snapshot_context['release_id'] = release_id return snapshot_context def browse_snapshot_directory(request, snapshot_id=None, origin_type=None, origin_url=None, timestamp=None, path=None): """ Django view implementation for browsing a directory in a snapshot context. """ try: snapshot_context = _process_snapshot_request(request, snapshot_id, origin_type, origin_url, timestamp, path, browse_context='directory') # noqa root_sha1_git = snapshot_context['root_sha1_git'] sha1_git = root_sha1_git if path: dir_info = service.lookup_directory_with_path(root_sha1_git, path) # some readme files can reference assets reachable from the # browsed directory, handle that special case in order to # correctly displayed them if dir_info and dir_info['type'] == 'file': file_raw_url = reverse( 'browse-content-raw', kwargs={'query_string': dir_info['checksums']['sha1']}) return redirect(file_raw_url) sha1_git = dir_info['target'] dirs, files = get_directory_entries(sha1_git) except Exception as exc: return handle_view_exception(request, exc) swh_type = snapshot_context['swh_type'] origin_info = snapshot_context['origin_info'] visit_info = snapshot_context['visit_info'] url_args = snapshot_context['url_args'] query_params = snapshot_context['query_params'] revision_id = snapshot_context['revision_id'] snapshot_id = snapshot_context['snapshot_id'] path_info = gen_path_info(path) browse_view_name = 'browse-' + swh_type + '-directory' breadcrumbs = [] breadcrumbs.append({'name': root_sha1_git[:7], 'url': reverse(browse_view_name, kwargs=url_args, query_params=query_params)}) for pi in path_info: bc_url_args = dict(url_args) bc_url_args['path'] = pi['path'] breadcrumbs.append({'name': pi['name'], 'url': reverse(browse_view_name, kwargs=bc_url_args, query_params=query_params)}) path = '' if path is None else (path + '/') for d in dirs: if d['type'] == 'rev': d['url'] = reverse('browse-revision', kwargs={'sha1_git': d['target']}) else: bc_url_args = dict(url_args) bc_url_args['path'] = path + d['name'] d['url'] = reverse(browse_view_name, kwargs=bc_url_args, query_params=query_params) sum_file_sizes = 0 readmes = {} browse_view_name = 'browse-' + swh_type + '-content' for f in files: + if f['length'] is None: + continue bc_url_args = dict(url_args) bc_url_args['path'] = path + f['name'] f['url'] = reverse(browse_view_name, kwargs=bc_url_args, query_params=query_params) sum_file_sizes += f['length'] f['length'] = filesizeformat(f['length']) if f['name'].lower().startswith('readme'): readmes[f['name']] = f['checksums']['sha1'] readme_name, readme_url, readme_html = get_readme_to_display(readmes) browse_view_name = 'browse-' + swh_type + '-log' history_url = reverse(browse_view_name, kwargs=url_args, query_params=query_params) sum_file_sizes = filesizeformat(sum_file_sizes) browse_dir_link = \ gen_directory_link(sha1_git, link_text='Browse', link_attrs={'class': 'btn btn-default btn-sm', 'role': 'button'}) browse_rev_link = \ gen_revision_link(revision_id, snapshot_context=snapshot_context, link_text='Browse', link_attrs={'class': 'btn btn-default btn-sm', 'role': 'button'}) dir_metadata = {'id': sha1_git, 'context-independent directory': browse_dir_link, 'number of regular files': len(files), 'number of subdirectories': len(dirs), 'sum of regular file sizes': sum_file_sizes, 'path': '/' + path, 'revision id': revision_id, 'revision': browse_rev_link, 'snapshot id': snapshot_id} if origin_info: dir_metadata['origin id'] = origin_info['id'] dir_metadata['origin type'] = origin_info['type'] dir_metadata['origin url'] = origin_info['url'] dir_metadata['origin visit date'] = format_utc_iso_date(visit_info['date']) # noqa dir_metadata['origin visit id'] = visit_info['visit'] snapshot_context_url = reverse('browse-snapshot-directory', kwargs={'snapshot_id': snapshot_id}, query_params=request.GET) browse_snapshot_link = \ gen_link(snapshot_context_url, link_text='Browse', link_attrs={'class': 'btn btn-default btn-sm', 'role': 'button'}) dir_metadata['snapshot context'] = browse_snapshot_link vault_cooking = { 'directory_context': True, 'directory_id': sha1_git, 'revision_context': True, 'revision_id': revision_id } swh_objects = [{'type': 'directory', 'id': sha1_git}, {'type': 'revision', 'id': revision_id}, {'type': 'snapshot', 'id': snapshot_id}] release_id = snapshot_context['release_id'] if release_id: swh_objects.append({'type': 'release', 'id': release_id}) swh_ids = get_swh_persistent_ids(swh_objects, snapshot_context) dir_path = '/'.join([bc['name'] for bc in breadcrumbs]) + '/' context_found = 'snapshot: %s' % snapshot_context['snapshot_id'] if origin_info: context_found = 'origin: %s' % origin_info['url'] heading = 'Directory - %s - %s - %s' %\ (dir_path, snapshot_context['branch'], context_found) return render(request, 'browse/directory.html', {'heading': heading, 'swh_object_name': 'Directory', 'swh_object_metadata': dir_metadata, 'dirs': dirs, 'files': files, 'breadcrumbs': breadcrumbs, 'top_right_link': history_url, 'top_right_link_text': mark_safe( '' 'History' ), 'readme_name': readme_name, 'readme_url': readme_url, 'readme_html': readme_html, 'snapshot_context': snapshot_context, 'vault_cooking': vault_cooking, 'show_actions_menu': True, 'swh_ids': swh_ids}) def browse_snapshot_content(request, snapshot_id=None, origin_type=None, origin_url=None, timestamp=None, path=None): """ Django view implementation for browsing a content in a snapshot context. """ try: snapshot_context = _process_snapshot_request(request, snapshot_id, origin_type, origin_url, timestamp, path, browse_context='content') root_sha1_git = snapshot_context['root_sha1_git'] content_info = service.lookup_directory_with_path(root_sha1_git, path) sha1_git = content_info['target'] query_string = 'sha1_git:' + sha1_git content_data = request_content(query_string, raise_if_unavailable=False) except Exception as exc: return handle_view_exception(request, exc) swh_type = snapshot_context['swh_type'] url_args = snapshot_context['url_args'] query_params = snapshot_context['query_params'] revision_id = snapshot_context['revision_id'] origin_info = snapshot_context['origin_info'] visit_info = snapshot_context['visit_info'] snapshot_id = snapshot_context['snapshot_id'] content = None language = None mimetype = None if content_data['raw_data'] is not None: content_display_data = prepare_content_for_display( content_data['raw_data'], content_data['mimetype'], path) content = content_display_data['content_data'] language = content_display_data['language'] mimetype = content_display_data['mimetype'] filename = None path_info = None browse_view_name = 'browse-' + swh_type + '-directory' breadcrumbs = [] split_path = path.split('/') filename = split_path[-1] path_info = gen_path_info(path[:-len(filename)]) breadcrumbs.append({'name': root_sha1_git[:7], 'url': reverse(browse_view_name, kwargs=url_args, query_params=query_params)}) for pi in path_info: bc_url_args = dict(url_args) bc_url_args['path'] = pi['path'] breadcrumbs.append({'name': pi['name'], 'url': reverse(browse_view_name, kwargs=bc_url_args, query_params=query_params)}) breadcrumbs.append({'name': filename, 'url': None}) browse_content_link = \ gen_content_link(sha1_git, link_text='Browse', link_attrs={'class': 'btn btn-default btn-sm', 'role': 'button'}) content_raw_url = reverse('browse-content-raw', kwargs={'query_string': query_string}, query_params={'filename': filename}) browse_rev_link = \ gen_revision_link(revision_id, snapshot_context=snapshot_context, link_text='Browse', link_attrs={'class': 'btn btn-default btn-sm', 'role': 'button'}) content_metadata = { 'context-independent content': browse_content_link, 'sha1 checksum': content_data['checksums']['sha1'], 'sha1_git checksum': content_data['checksums']['sha1_git'], 'sha256 checksum': content_data['checksums']['sha256'], 'blake2s256 checksum': content_data['checksums']['blake2s256'], 'mime type': content_data['mimetype'], 'encoding': content_data['encoding'], 'size': filesizeformat(content_data['length']), 'language': content_data['language'], 'licenses': content_data['licenses'], 'path': '/' + path[:-len(filename)], 'filename': filename, 'revision id': revision_id, 'revision': browse_rev_link, 'snapshot id': snapshot_id } if origin_info: content_metadata['origin id'] = origin_info['id'] content_metadata['origin type'] = origin_info['type'] content_metadata['origin url'] = origin_info['url'] content_metadata['origin visit date'] = format_utc_iso_date(visit_info['date']) # noqa content_metadata['origin visit id'] = visit_info['visit'] browse_snapshot_url = reverse('browse-snapshot-content', kwargs={'snapshot_id': snapshot_id, 'path': path}, query_params=request.GET) browse_snapshot_link = \ gen_link(browse_snapshot_url, link_text='Browse', link_attrs={'class': 'btn btn-default btn-sm', 'role': 'button'}) content_metadata['snapshot context'] = browse_snapshot_link cnt_sha1_git = content_data['checksums']['sha1_git'] swh_objects = [{'type': 'content', 'id': cnt_sha1_git}, {'type': 'revision', 'id': revision_id}, {'type': 'snapshot', 'id': snapshot_id}] release_id = snapshot_context['release_id'] if release_id: swh_objects.append({'type': 'release', 'id': release_id}) swh_ids = get_swh_persistent_ids(swh_objects, snapshot_context) content_path = '/'.join([bc['name'] for bc in breadcrumbs]) context_found = 'snapshot: %s' % snapshot_context['snapshot_id'] if origin_info: context_found = 'origin: %s' % origin_info['url'] heading = 'Content - %s - %s - %s' %\ (content_path, snapshot_context['branch'], context_found) return render(request, 'browse/content.html', {'heading': heading, 'swh_object_name': 'Content', 'swh_object_metadata': content_metadata, 'content': content, 'content_size': content_data['length'], 'max_content_size': content_display_max_size, 'mimetype': mimetype, 'language': language, 'breadcrumbs': breadcrumbs, 'top_right_link': content_raw_url, 'top_right_link_text': mark_safe( 'Raw File'), 'snapshot_context': snapshot_context, 'vault_cooking': None, 'show_actions_menu': True, 'swh_ids': swh_ids, 'error_code': content_data['error_code'], 'error_message': content_data['error_message'], 'error_description': content_data['error_description']}, status=content_data['error_code']) PER_PAGE = 100 def browse_snapshot_log(request, snapshot_id=None, origin_type=None, origin_url=None, timestamp=None): """ Django view implementation for browsing a revision history in a snapshot context. """ try: snapshot_context = _process_snapshot_request(request, snapshot_id, origin_type, origin_url, timestamp, browse_context='log') # noqa revision_id = snapshot_context['revision_id'] current_rev = revision_id per_page = int(request.GET.get('per_page', PER_PAGE)) revs_breadcrumb = request.GET.get('revs_breadcrumb', None) if revs_breadcrumb: current_rev = revs_breadcrumb.split('/')[-1] revision_log = service.lookup_revision_log(current_rev, limit=per_page+1) revision_log = list(revision_log) except Exception as exc: return handle_view_exception(request, exc) swh_type = snapshot_context['swh_type'] origin_info = snapshot_context['origin_info'] visit_info = snapshot_context['visit_info'] url_args = snapshot_context['url_args'] query_params = snapshot_context['query_params'] snapshot_id = snapshot_context['snapshot_id'] query_params['per_page'] = per_page revision_log_display_data = prepare_revision_log_for_display( revision_log, per_page, revs_breadcrumb, snapshot_context) browse_view_name = 'browse-' + swh_type + '-log' prev_rev = revision_log_display_data['prev_rev'] prev_revs_breadcrumb = revision_log_display_data['prev_revs_breadcrumb'] prev_log_url = None query_params['revs_breadcrumb'] = prev_revs_breadcrumb if prev_rev: prev_log_url = \ reverse(browse_view_name, kwargs=url_args, query_params=query_params) next_rev = revision_log_display_data['next_rev'] next_revs_breadcrumb = revision_log_display_data['next_revs_breadcrumb'] next_log_url = None query_params['revs_breadcrumb'] = next_revs_breadcrumb if next_rev: next_log_url = \ reverse(browse_view_name, kwargs=url_args, query_params=query_params) revision_log_data = revision_log_display_data['revision_log_data'] for i, log in enumerate(revision_log_data): params = { 'revision': revision_log[i]['id'], } if 'visit_id' in query_params: params['visit_id'] = query_params['visit_id'] log['directory'] = gen_snapshot_directory_link( snapshot_context, revision_log[i]['id'], link_text='Browse files', link_attrs={'class': 'btn btn-default btn-sm', 'role': 'button'}) browse_log_link = \ gen_revision_log_link(revision_id, link_text='Browse', link_attrs={'class': 'btn btn-default btn-sm', 'role': 'button'}) revision_metadata = { 'context-independent revision history': browse_log_link, 'snapshot id': snapshot_id } if origin_info: revision_metadata['origin id'] = origin_info['id'] revision_metadata['origin type'] = origin_info['type'] revision_metadata['origin url'] = origin_info['url'] revision_metadata['origin visit date'] = format_utc_iso_date(visit_info['date']) # noqa revision_metadata['origin visit id'] = visit_info['visit'] browse_snapshot_url = reverse('browse-snapshot-log', kwargs={'snapshot_id': snapshot_id}, query_params=request.GET) browse_snapshot_link = \ gen_link(browse_snapshot_url, link_text='Browse', link_attrs={'class': 'btn btn-default btn-sm', 'role': 'button'}) revision_metadata['snapshot context'] = browse_snapshot_link swh_objects = [{'type': 'revision', 'id': revision_id}, {'type': 'snapshot', 'id': snapshot_id}] release_id = snapshot_context['release_id'] if release_id: swh_objects.append({'type': 'release', 'id': release_id}) swh_ids = get_swh_persistent_ids(swh_objects, snapshot_context) context_found = 'snapshot: %s' % snapshot_context['snapshot_id'] if origin_info: context_found = 'origin: %s' % origin_info['url'] heading = 'Revision history - %s - %s' %\ (snapshot_context['branch'], context_found) return render(request, 'browse/revision-log.html', {'heading': heading, 'swh_object_name': 'Revision history', 'swh_object_metadata': revision_metadata, 'revision_log': revision_log_data, 'next_log_url': next_log_url, 'prev_log_url': prev_log_url, 'breadcrumbs': None, 'top_right_link': None, 'top_right_link_text': None, 'snapshot_context': snapshot_context, 'vault_cooking': None, 'show_actions_menu': True, 'swh_ids': swh_ids}) def browse_snapshot_branches(request, snapshot_id=None, origin_type=None, origin_url=None, timestamp=None): """ Django view implementation for browsing a list of branches in a snapshot context. """ try: snapshot_context = _process_snapshot_request(request, snapshot_id, origin_type, origin_url, timestamp) branches_bc = request.GET.get('branches_breadcrumbs', '') branches_bc = \ branches_bc.split(',') if branches_bc else [] branches_from = branches_bc[-1] if branches_bc else '' swh_type = snapshot_context['swh_type'] origin_info = snapshot_context['origin_info'] url_args = snapshot_context['url_args'] query_params = snapshot_context['query_params'] browse_view_name = 'browse-' + swh_type + '-directory' displayed_branches = \ service.lookup_snapshot(snapshot_context['snapshot_id'], branches_from, PER_PAGE+1, target_types=['revision'])['branches'] except Exception as exc: return handle_view_exception(request, exc) displayed_branches, _ = process_snapshot_branches(displayed_branches) for branch in displayed_branches: if snapshot_id: revision_url = reverse('browse-revision', kwargs={'sha1_git': branch['revision']}, query_params={'snapshot_id': snapshot_id}) else: revision_url = reverse('browse-revision', kwargs={'sha1_git': branch['revision']}, query_params={'origin_type': origin_type, 'origin': origin_info['url']}) query_params['branch'] = branch['name'] directory_url = reverse(browse_view_name, kwargs=url_args, query_params=query_params) del query_params['branch'] branch['revision_url'] = revision_url branch['directory_url'] = directory_url browse_view_name = 'browse-' + swh_type + '-branches' prev_branches_url = None next_branches_url = None if branches_bc: query_params_prev = dict(query_params) query_params_prev['branches_breadcrumbs'] = \ ','.join(branches_bc[:-1]) prev_branches_url = reverse(browse_view_name, kwargs=url_args, query_params=query_params_prev) elif branches_from: prev_branches_url = reverse(browse_view_name, kwargs=url_args, query_params=query_params) if len(displayed_branches) > PER_PAGE: query_params_next = dict(query_params) next_branch = displayed_branches[-1]['name'] del displayed_branches[-1] branches_bc.append(next_branch) query_params_next['branches_breadcrumbs'] = \ ','.join(branches_bc) next_branches_url = reverse(browse_view_name, kwargs=url_args, query_params=query_params_next) heading = 'Branches - ' if origin_info: heading += 'origin: %s' % origin_info['url'] else: heading += 'snapshot: %s' % snapshot_id return render(request, 'browse/branches.html', {'heading': heading, 'swh_object_name': 'Branches', 'swh_object_metadata': {}, 'top_right_link': None, 'top_right_link_text': None, 'displayed_branches': displayed_branches, 'prev_branches_url': prev_branches_url, 'next_branches_url': next_branches_url, 'snapshot_context': snapshot_context}) def browse_snapshot_releases(request, snapshot_id=None, origin_type=None, origin_url=None, timestamp=None): """ Django view implementation for browsing a list of releases in a snapshot context. """ try: snapshot_context = _process_snapshot_request(request, snapshot_id, origin_type, origin_url, timestamp) rel_bc = request.GET.get('releases_breadcrumbs', '') rel_bc = \ rel_bc.split(',') if rel_bc else [] rel_from = rel_bc[-1] if rel_bc else '' swh_type = snapshot_context['swh_type'] origin_info = snapshot_context['origin_info'] url_args = snapshot_context['url_args'] query_params = snapshot_context['query_params'] displayed_releases = \ service.lookup_snapshot(snapshot_context['snapshot_id'], rel_from, PER_PAGE+1, target_types=['release'])['branches'] except Exception as exc: return handle_view_exception(request, exc) _, displayed_releases = process_snapshot_branches(displayed_releases) for release in displayed_releases: if snapshot_id: release_url = reverse('browse-release', kwargs={'sha1_git': release['id']}, query_params={'snapshot_id': snapshot_id}) else: release_url = reverse('browse-release', kwargs={'sha1_git': release['id']}, query_params={'origin_type': origin_type, 'origin': origin_info['url']}) query_params['release'] = release['name'] del query_params['release'] release['release_url'] = release_url browse_view_name = 'browse-' + swh_type + '-releases' prev_releases_url = None next_releases_url = None if rel_bc: query_params_prev = dict(query_params) query_params_prev['releases_breadcrumbs'] = \ ','.join(rel_bc[:-1]) prev_releases_url = reverse(browse_view_name, kwargs=url_args, query_params=query_params_prev) elif rel_from: prev_releases_url = reverse(browse_view_name, kwargs=url_args, query_params=query_params) if len(displayed_releases) > PER_PAGE: query_params_next = dict(query_params) next_rel = displayed_releases[-1]['branch_name'] del displayed_releases[-1] rel_bc.append(next_rel) query_params_next['releases_breadcrumbs'] = \ ','.join(rel_bc) next_releases_url = reverse(browse_view_name, kwargs=url_args, query_params=query_params_next) heading = 'Releases - ' if origin_info: heading += 'origin: %s' % origin_info['url'] else: heading += 'snapshot: %s' % snapshot_id return render(request, 'browse/releases.html', {'heading': heading, 'top_panel_visible': False, 'top_panel_collapsible': False, 'swh_object_name': 'Releases', 'swh_object_metadata': {}, 'top_right_link': None, 'top_right_link_text': None, 'displayed_releases': displayed_releases, 'prev_releases_url': prev_releases_url, 'next_releases_url': next_releases_url, 'snapshot_context': snapshot_context, 'vault_cooking': None, 'show_actions_menu': False}) diff --git a/version.txt b/version.txt index 8687cbb1..256fe985 100644 --- a/version.txt +++ b/version.txt @@ -1 +1 @@ -v0.0.163-0-g29a4059 \ No newline at end of file +v0.0.164-0-ge50bd19 \ No newline at end of file