Changeset View
Changeset View
Standalone View
Standalone View
README.md
Show All 21 Lines | Those applications are powered by: | ||||
* [webpack](https://webpack.js.org/) on the frontend side for better static assets management, including: | * [webpack](https://webpack.js.org/) on the frontend side for better static assets management, including: | ||||
* assets dependencies management and retrieval through [yarn](https://yarnpkg.com/en/) | * assets dependencies management and retrieval through [yarn](https://yarnpkg.com/en/) | ||||
* linting of custom javascript code (through [eslint](https://eslint.org/)) and stylesheets (through [stylelint](https://stylelint.io/)) | * linting of custom javascript code (through [eslint](https://eslint.org/)) and stylesheets (through [stylelint](https://stylelint.io/)) | ||||
* use of [es6](http://es6-features.org) syntax and advanced javascript feature like [async/await](https://javascript.info/async-await) or [fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) thanks to [babel](https://babeljs.io/) (es6 to es5 transpiler and polyfills provider) | * use of [es6](http://es6-features.org) syntax and advanced javascript feature like [async/await](https://javascript.info/async-await) or [fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) thanks to [babel](https://babeljs.io/) (es6 to es5 transpiler and polyfills provider) | ||||
* assets minification (using [terser](https://github.com/terser-js/terser) and [cssnano](http://cssnano.co/)) but also dead code elimination for production use | * assets minification (using [terser](https://github.com/terser-js/terser) and [cssnano](http://cssnano.co/)) but also dead code elimination for production use | ||||
## How to build and run | ## How to build, run and test | ||||
### Requirements | ### Backend requirements | ||||
First you will need [Python 3](https://www.python.org) and a complete [swh development environment](https://forge.softwareheritage.org/source/swh-environment/) installed. | First you will need [Python 3](https://www.python.org) and a complete [swh development environment](https://forge.softwareheritage.org/source/swh-environment/) installed. | ||||
To run the backend, you need to have the following Python 3 modules installed: | To run the backend, you need to have the following [Python 3 modules](requirements.txt) installed. | ||||
* beautifulsoup4 | |||||
* django >= 1.11.0 | To run the backend tests, the following [Python 3 modules](requirements-test.txt) are also required to be installed. | ||||
* djangorestframework >= 3.4.0 | |||||
* django_webpack_loader | One easy way to install them is to use the `pip` tool: | ||||
* django_js_reverse | ``` | ||||
* docutils | $ pip install -r requirements.txt -r requirements-test.txt | ||||
* file_magic >= 0.3.0 | ``` | ||||
* htmlmin | |||||
* lxml | ### Frontend requirements | ||||
* pygments | |||||
* pypandoc | |||||
* python-dateutil | |||||
* pyyaml | |||||
* requests | |||||
To compile the frontend assets, you need to have [nodejs](https://nodejs.org/en/) >= 8.x and [yarn](https://yarnpkg.com/en/) installed. If you are on Debian stretch, you can easily install an up to date nodejs from the [stretch-backports](https://backports.debian.org/Instructions/) repository. | To compile the frontend assets, you need to have [nodejs](https://nodejs.org/en/) >= 8.x and [yarn](https://yarnpkg.com/en/) installed. If you are on Debian stretch, you can easily install an up to date nodejs from the [stretch-backports](https://backports.debian.org/Instructions/) repository. | ||||
vlorentz: This should be a pip command using the requirement files, or it will get out of date | |||||
Done Inline ActionsAck, I will replace those lists by a link to the requirement files anlambert: Ack, I will replace those lists by a link to the requirement files | |||||
Not Done Inline ActionsIt could help to mention they can be installed with pip -r requirements.txt vlorentz: It could help to mention they can be installed with `pip -r requirements.txt` | |||||
Packages for yarn can be installed by following [these instructions](https://yarnpkg.com/en/docs/install#debian-stable). | Packages for yarn can be installed by following [these instructions](https://yarnpkg.com/en/docs/install#debian-stable). | ||||
Alternatively, you can install yarn with `npm install yarn`, and add `YARN=node_modules/yarn/bin/yarn` as argument whenever you run `make`. | Alternatively, you can install yarn with `npm install yarn`, and add `YARN=node_modules/yarn/bin/yarn` as argument whenever you run `make`. | ||||
Please note that the static assets bundles generated by webpack are not stored in the git repository. Follow the instructions below in order to generate them in order to be able to run the frontend part of the web applications. | Please note that the static assets bundles generated by webpack are not stored in the git repository. Follow the instructions below in order to generate them in order to be able to run the frontend part of the web applications. | ||||
### Make targets | ### Make targets to execute the applications | ||||
Below is the list of available make targets that can be executed from the root directory of swh-web in order to build and/or execute the web applications under various configurations: | Below is the list of available make targets that can be executed from the root directory of swh-web in order to build and/or execute the web applications under various configurations: | ||||
* **run-django-webpack-devserver**: Compile and serve not optimized (without mignification and dead code elimination) frontend static assets using [webpack-dev-server](https://github.com/webpack/webpack-dev-server) and run django server with development settings. This is the recommended target to use when developing swh-web as it enables automatic reloading of backend and frontend part of the applications when modifying source files (*.py, *.js, *.css, *.html). | * **run-django-webpack-devserver**: Compile and serve not optimized (without mignification and dead code elimination) frontend static assets using [webpack-dev-server](https://github.com/webpack/webpack-dev-server) and run django server with development settings. This is the recommended target to use when developing swh-web as it enables automatic reloading of backend and frontend part of the applications when modifying source files (*.py, *.js, *.css, *.html). | ||||
* **run-django-webpack-dev**: Compile not optimized (no minification, no dead code elimination) frontend static assets using webpack and run django server with development settings. This is the recommended target when one only wants to develop the backend side of the application. | * **run-django-webpack-dev**: Compile not optimized (no minification, no dead code elimination) frontend static assets using webpack and run django server with development settings. This is the recommended target when one only wants to develop the backend side of the application. | ||||
* **run-django-webpack-prod**: Compile optimized (with minification and dead code elimination) frontend static assets using webpack and run django server with production settings. This is useful to test the applications in production mode (with the difference that static assets are served by django). Production settings notably enable advanced django caching and you will need to have [memcached](https://memcached.org/) installed for that feature to work. | * **run-django-webpack-prod**: Compile optimized (with minification and dead code elimination) frontend static assets using webpack and run django server with production settings. This is useful to test the applications in production mode (with the difference that static assets are served by django). Production settings notably enable advanced django caching and you will need to have [memcached](https://memcached.org/) installed for that feature to work. | ||||
* **run-django-server-dev**: Run the django server with development settings but without compiling frontend static assets through webpack. | * **run-django-server-dev**: Run the django server with development settings but without compiling frontend static assets through webpack. | ||||
* **run-django-server-prod**: Run the django server with production settings but without compiling frontend static assets through webpack. | * **run-django-server-prod**: Run the django server with production settings but without compiling frontend static assets through webpack. | ||||
* **run-gunicorn-server**: Run the web applications with production settings in a [gunicorn](http://gunicorn.org/) worker as they will be in real production environment. | * **run-gunicorn-server**: Run the web applications with production settings in a [gunicorn](http://gunicorn.org/) worker as they will be in real production environment. | ||||
Once one of these targets executed, the web applications can be executed by pointing your browser to http://localhost:5004. | Once one of these targets executed, the web applications can be executed by pointing your browser to http://localhost:5004. | ||||
### Make targets to test the applications | |||||
Some make targets are also available to easily execute the backend and frontend tests of the Software Heritage web applications. | |||||
The backend tests are powered by the [pytest](https://docs.pytest.org/en/latest/) and [hypothesis](https://hypothesis.readthedocs.io/en/latest/) frameworks while the frontend ones rely on the use of the [cypress](https://www.cypress.io/) tool. | |||||
Below is the exhaustive list of those targets: | |||||
* **test**: execute the backend tests using a fast hypothesis profile (only one input example will be provided for each test) | |||||
* **test-full**: execute the backend tests using a slower hypothesis profile (one hundred of input examples will be provided for each test which helps spotting possible bugs) | |||||
* **test-frontend**: execute the frontend tests using cypress in headless mode but with some slow test suites disabled | |||||
* **test-frontend-full**: execute the frontend tests using cypress in headless mode with all test suites enabled | |||||
* **test-frontend-ui**: execute the frontend tests using the cypress GUI but with some slow test suites disabled | |||||
* **test-frontend-full-ui**: execute the frontend tests using the cypress GUI with all test suites enabled | |||||
### Yarn targets | ### Yarn targets | ||||
Below is a list of available yarn targets in order to only execute the frontend static assets compilation (no web server will be executed): | Below is a list of available yarn targets in order to only execute the frontend static assets compilation (no web server will be executed): | ||||
* **build-dev**: compile not optimized (without mignification and dead code elimination) frontend static assets and store the results in the `swh/web/static` folder. | * **build-dev**: compile not optimized (without mignification and dead code elimination) frontend static assets and store the results in the `swh/web/static` folder. | ||||
* **build**: compile optimized (with mignification and dead code elimination) frontend static assets and store the results in the `swh/web/static` folder. | * **build**: compile optimized (with mignification and dead code elimination) frontend static assets and store the results in the `swh/web/static` folder. | ||||
**The build target must be executed prior performing the Debian packaging of swh-web** in order for the package to contain the optimized assets dedicated to production environment. | **The build target must be executed prior performing the Debian packaging of swh-web** in order for the package to contain the optimized assets dedicated to production environment. | ||||
To execute these targets, issue the following command: | To execute these targets, issue the following command: | ||||
``` | ``` | ||||
$ yarn <target_name> | $ yarn <target_name> | ||||
``` | ``` |
This should be a pip command using the requirement files, or it will get out of date