Paths

Table of Contentst

Add end to end tests for content rendering
ClosedPublic
Actions

Authored by anlambert on Jul 1 2019, 6:01 PM.

Details

Reviewers

vlorentz
swh-public-ci

Group Reviewers

Reviewers

Commits

rDWAPPS8421487367a4: cypress: Add end to end tests for content rendering
rDWAPPS12427620b5ad: common/highlightjs: Improve code highlighting of numerous source files

Summary

That diff aims to test HTML rendering of the contents stored inside the archive.
The purpose is to ensure no regressions will appear when some JavaScript dependencies
or our custom frontend code evolve.

A first batch of tests will check that almost all the programming languages supported by the
highlightjs library are correctly highlighted.

The other tests will check that some special contents (images, pdf files, ...) will be
correctly rendered.

In order to provide relevant input data to these new tests, some new endpoints are added
to swh-web for that purpose (only available when running the end to end tests).

Related T1845

Diff Detail

Repository

rDWAPPS Web applications

Branch

content-rendering-e2e-tests

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 6804
Build 9523: Cypress tests for swh-web diffs	Jenkins
Build 9522: tox-on-jenkins	Jenkins
Build 9521: arc lint + arc unit

Event Timeline

anlambert created this revision.Jul 1 2019, 6:01 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJul 1 2019, 6:01 PM

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/567/ for more details.

Harbormaster completed remote builds in B6615: Diff 5607.Jul 1 2019, 6:09 PM

Please add comments on the code added to swh/web/tests/data.py, I have no idea what it does.

And you are bundling lots of non-trivial code, some with no authoring attribution or license. I don't think we need all these tests. Testing a few languages and formats (and trusting highlightjs to do its job on the rest) should be enough.

swh/web/common/highlightjs.py
44	Where is that list coming from?

This revision now requires changes to proceed.Jul 1 2019, 6:14 PM

Please add comments on the code added to swh/web/tests/data.py, I have no idea what it does.

So the WIP tag ...

And you are bundling lots of non-trivial code, some with no authoring attribution or license. I don't think we need all these tests.

From my point of view, those tests need to be exhaustive as code highlighting is one of the most important feature of
swh-web/browse (and we are a source code archive right ? we should be able to highlight correctly a lot of contents)
I can try to find better source files with proper licenses in it but it will be complicated to find some for some format.

Testing a few languages and formats (and trusting highlightjs to do its job on the rest) should be enough.

I do not rely on the automatic language detection of highlightjs as it does not work so great and it implies
reading the source file content client-side (not really optimal in terms of performance). I use heuristics based
on file extensions, filenames or content mime types. This is not the perfect solution but it works pretty great.
Of course, this could be improved in the future.

swh/web/common/highlightjs.py
44	Generated from the highlightjs language aliases and completed by hand.

vlorentz added inline comments.Jul 1 2019, 6:49 PM

swh/web/common/highlightjs.py
44	Would it be possible to send highlightjs a PR to add the aliases; and bundle their data file directly?

anlambert planned changes to this revision.Jul 4 2019, 5:21 PM

Update:

rebase to master
add remaining programming languages to test highlighting
add test for Jupyter notebook rendering

anlambert retitled this revision from [WIP] Add end to end tests for content rendering to Add end to end tests for content rendering.Jul 8 2019, 5:38 PM

anlambert edited the summary of this revision. (Show Details)

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/588/ for more details.

Harbormaster completed remote builds in B6728: Diff 5705.Jul 8 2019, 5:41 PM

For your information, I have also tested how reliable the automatic language detection feature of the highlight.js library is.
Below are the results (tests without timing have failed):

Code highlighting tests
    1) should highlight source files with extension R
    2) should highlight source files with extension abnf
    ✓ should highlight source files with extension adb (1478ms)
    ✓ should highlight source files with extension adoc (1171ms)
    ✓ should highlight source files with extension ads (1783ms)
    3) should highlight source files with extension ahk
    4) should highlight source files with extension aj
    ✓ should highlight source files with extension applescript (694ms)
    5) should highlight source files with extension as
    ✓ should highlight source files with extension au3 (1522ms)
    6) should highlight source files with extension awk
    7) should highlight source files with extension bas
    ✓ should highlight source files with extension bat (836ms)
    ✓ should highlight source files with extension bf (783ms)
    8) should highlight source files with extension bnf
    ✓ should highlight source files with extension bsl (1231ms)
    ✓ should highlight source files with extension cal (1096ms)
    ✓ should highlight source files with extension capnp (1431ms)
    ✓ should highlight source files with extension cc (1183ms)
    9) should highlight source files with extension ceylon
    ✓ should highlight source files with extension clj (773ms)
    ✓ should highlight source files with extension cls (840ms)
    ✓ should highlight source files with extension cmake (941ms)
    ✓ should highlight source files with extension coffee (946ms)
    10) should highlight source files with extension cpp
    11) should highlight source files with extension cr
    ✓ should highlight source files with extension cs (872ms)
    ✓ should highlight source files with extension css (878ms)
    ✓ should highlight source files with extension d (1132ms)
    12) should highlight source files with extension dart
    ✓ should highlight source files with extension dcl (1453ms)
    13) should highlight source files with extension dfm
    14) should highlight source files with extension diff
    ✓ should highlight source files with extension do (1090ms)
    ✓ should highlight source files with extension dts (928ms)
    ✓ should highlight source files with extension dust (1082ms)
    15) should highlight source files with extension ebnf
    16) should highlight source files with extension elm
    ✓ should highlight source files with extension ep (1130ms)
    ✓ should highlight source files with extension erb (604ms)
    ✓ should highlight source files with extension erl (959ms)
    ✓ should highlight source files with extension ex (1055ms)
    ✓ should highlight source files with extension exs (868ms)
    ✓ should highlight source files with extension f90 (1025ms)
    17) should highlight source files with extension feature
    18) should highlight source files with extension flix
    ✓ should highlight source files with extension fs (1528ms)
    ✓ should highlight source files with extension gcode (799ms)
    ✓ should highlight source files with extension glsl (872ms)
    ✓ should highlight source files with extension gml (2535ms)
    19) should highlight source files with extension gms
    ✓ should highlight source files with extension go (1012ms)
    20) should highlight source files with extension golo
    21) should highlight source files with extension gradle
    22) should highlight source files with extension groovy
    23) should highlight source files with extension gss
    24) should highlight source files with extension haml
    25) should highlight source files with extension hbs
    26) should highlight source files with extension hs
    ✓ should highlight source files with extension hsp (1111ms)
    27) should highlight source files with extension hx
    ✓ should highlight source files with extension hy (2825ms)
    ✓ should highlight source files with extension icl (1104ms)
    28) should highlight source files with extension ini
    29) should highlight source files with extension ino
    30) should highlight source files with extension java
    31) should highlight source files with extension jl
    32) should highlight source files with extension js
    ✓ should highlight source files with extension json (1033ms)
    33) should highlight source files with extension kt
    34) should highlight source files with extension lasso
    35) should highlight source files with extension lc
    ✓ should highlight source files with extension ldif (802ms)
    ✓ should highlight source files with extension leaf (1086ms)
    ✓ should highlight source files with extension less (1026ms)
    36) should highlight source files with extension lisp
    ✓ should highlight source files with extension ll (1041ms)
    ✓ should highlight source files with extension ls (938ms)
    ✓ should highlight source files with extension lsl (1132ms)
    ✓ should highlight source files with extension lua (1030ms)
    37) should highlight source files with extension m
    ✓ should highlight source files with extension md (1461ms)
    ✓ should highlight source files with extension mel (1027ms)
    38) should highlight source files with extension mk
    39) should highlight source files with extension ml
    40) should highlight source files with extension moon
    41) should highlight source files with extension nb
    42) should highlight source files with extension nim
    ✓ should highlight source files with extension nix (779ms)
    ✓ should highlight source files with extension nsi (1502ms)
    43) should highlight source files with extension p
    ✓ should highlight source files with extension pas (1044ms)
    44) should highlight source files with extension pbi
    ✓ should highlight source files with extension pde (1964ms)
    45) should highlight source files with extension php
    46) should highlight source files with extension pl
    47) should highlight source files with extension pm
    48) should highlight source files with extension pony
    49) should highlight source files with extension pp
    50) should highlight source files with extension properties
    ✓ should highlight source files with extension proto (1654ms)
    51) should highlight source files with extension ps1
    ✓ should highlight source files with extension py (1057ms)
    52) should highlight source files with extension q
    53) should highlight source files with extension qml
    ✓ should highlight source files with extension rb (1220ms)
    ✓ should highlight source files with extension re (1053ms)
    54) should highlight source files with extension rib
    55) should highlight source files with extension rs
    ✓ should highlight source files with extension rsc (1811ms)
    56) should highlight source files with extension s
    ✓ should highlight source files with extension sas (1004ms)
    57) should highlight source files with extension scad
    ✓ should highlight source files with extension scala (1621ms)
    58) should highlight source files with extension sci
    ✓ should highlight source files with extension scm (972ms)
    ✓ should highlight source files with extension scss (1143ms)
    ✓ should highlight source files with extension sh (1335ms)
    ✓ should highlight source files with extension sl (1318ms)
    ✓ should highlight source files with extension smali (1371ms)
    ✓ should highlight source files with extension sml (807ms)
    ✓ should highlight source files with extension sqf (1264ms)
    59) should highlight source files with extension st
    ✓ should highlight source files with extension stan (898ms)
    60) should highlight source files with extension styl
    ✓ should highlight source files with extension subunit (1019ms)
    ✓ should highlight source files with extension swift (977ms)
    ✓ should highlight source files with extension tap (1231ms)
    61) should highlight source files with extension tcl
    ✓ should highlight source files with extension tex (1145ms)
    ✓ should highlight source files with extension thrift (854ms)
    62) should highlight source files with extension ts
    ✓ should highlight source files with extension v (2515ms)
    63) should highlight source files with extension vala
    ✓ should highlight source files with extension vb (1599ms)
    64) should highlight source files with extension vbs
    ✓ should highlight source files with extension vhd (5093ms)
    65) should highlight source files with extension vim
    66) should highlight source files with extension wl
    67) should highlight source files with extension xml
    ✓ should highlight source files with extension xqy (2337ms)
    68) should highlight source files with extension yml
    ✓ should highlight source files with extension zep (1373ms)
    ✓ should highlight source files with filenames .htaccess (1220ms)
    69) should highlight source files with filenames CMakeLists.txt
    70) should highlight source files with filenames Dockerfile
    71) should highlight source files with filenames Makefile
    ✓ should highlight source files with filenames access.log (1211ms)
    ✓ should highlight source files with filenames httpd.conf (1338ms)
    72) should highlight source files with filenames nginx.conf
    ✓ should highlight source files with filenames nginx.log (940ms)
    ✓ should highlight source files with filenames pf.conf (993ms)
    73) should highlight source files with filenames resolv.conf

So 73/152 tests are failing when the heuristics I have implemented to associate a programming language given a filename are not used.
This comforts me in the fact that using heuristics was the most reasonable approach.

anlambert mentioned this in T1909: Enable to manually select the programming language to highlight.Jul 10 2019, 4:52 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/1/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/1/console

This revision now requires changes to proceed.Jul 11 2019, 1:40 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/21/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/21/console

Rebase

Harbormaster failed remote builds in B6804: Diff 5784!Jul 11 2019, 2:26 PM

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/605/ for more details.

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/22/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/22/console

kalpitk added a subscriber: kalpitk.Jul 23 2019, 2:39 PM

kalpitk added inline comments.

cypress/integration/content-rendering.spec.js
8	Should we move this to cypress/support/index.js so that it can be accessed by D1756 also?

anlambert added inline comments.Jul 24 2019, 11:55 AM

cypress/integration/content-rendering.spec.js
8	I will rather move it in the `cypress/integration/utils/index.js` file. As this function is not needed in every tests, I think it is better to import it only when required. I will update that diff today with that change (and also rebase it on master).

Update:

rebase
fix failing cypress tests
move checkLanguageHighlighting function in the utils module

Next step: Write simple Hello World files or reuse code example from highlight.js demo for each language to test highlighting instead of using random source files downloaded from the web.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/627/ for more details.

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/223/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/223/console

Harbormaster failed remote builds in B7004: Diff 5964!Jul 24 2019, 2:06 PM

Update:

really fix cypress tests that were failing
Use different test source files (taken from the highlight.js demo website) to avoid license issues

This Diff should now be in a landable state.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/628/ for more details.

Build was aborted

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/224/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/224/console

Harbormaster failed remote builds in B7007: Diff 5965!Jul 24 2019, 4:54 PM

Update: ensure the cypress/fixtures folder exists as tests will fail otherwise

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/629/ for more details.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/225/ for more details.

Harbormaster completed remote builds in B7008: Diff 5966.Jul 24 2019, 5:06 PM

kalpitk added a child revision: D1756: cypress: Test Language-select.Jul 25 2019, 9:03 AM

Update: Ensure there is no remaining test source files with explicit licensing in it

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/632/ for more details.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/228/ for more details.

Harbormaster completed remote builds in B7041: Diff 6000.Jul 26 2019, 2:31 PM

Update: Every test source files are now taken from the highlight.js demo to ease their attribution

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/634/ for more details.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/230/ for more details.

Harbormaster completed remote builds in B7060: Diff 6022.Jul 29 2019, 1:31 PM

ardumont mentioned this in D1794: terraform: Prepare the workstation tools.Aug 1 2019, 9:45 AM

ardumont mentioned this in D1796: init-template: Explain how to bootstrap a debian template image.Aug 1 2019, 10:07 AM

vlorentz accepted this revision.Aug 5 2019, 10:23 AM

Rebase

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/649/ for more details.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/245/ for more details.

Harbormaster completed remote builds in B7144: Diff 6111.Aug 5 2019, 11:12 AM

This revision was not accepted when it landed; it landed in state Needs Review.Aug 5 2019, 11:14 AM

Closed by commit rDWAPPS12427620b5ad: common/highlightjs: Improve code highlighting of numerous source files (authored by anlambert). · Explain Why

This revision was automatically updated to reflect the committed changes.

Large Diff

This large diff affects 171 files. Files without inline comments have been collapsed. Expand All Files

Revision Contents
Changeset List

Path

Size

.gitignore

1 line

Makefile.local

2 lines

cypress.json

3 lines

cypress/

fixtures/

example.json

5 lines

integration/

content-rendering.spec.js

111 lines

plugins/

index.js

34 lines

swh/

web/

common/

highlightjs.py

57 lines

config.py

3 lines

settings/

tests.py

1 line

tests/

data.py

151 lines

resources/

contents/

code/

extensions/

3556336_prac4q1.s

27 lines

AccumFrag.glsl

11 lines

AnnotationCacheAspect.aj

116 lines

ApplicationJobRoutesTest.kt

50 lines

424 lines

52 lines

199 lines

43 lines

6 lines

24 lines

86 lines

50 lines

104 lines

52 lines

48 lines

196 lines

29 lines

25 lines

94 lines

Get Receipts from Notes.applescript

39 lines

Grammar.bnf

122 lines

JQueryProvider.groovy

188 lines

116 lines

38 lines

94 lines

43 lines

ParenMatcherHighlighter.cpp

101 lines

PolarisResponse.Class.ps1

64 lines

136 lines

90 lines

72 lines

15 lines

8 lines

23 lines

34 lines

32 lines

80 lines

71 lines

70 lines

94 lines

118 lines

14 lines

68 lines

42 lines

116 lines

18 lines

84 lines

44 lines

13 lines

122 lines

143 lines

101 lines

46 lines

159 lines

150 lines

120 lines

20 lines

11 lines

17 lines

8 lines

32 lines

72 lines

8 lines

77 lines

28 lines

43 lines

21 lines

207 lines

102 lines

60 lines

fnc_switchAttachment.sqf

91 lines

67 lines

79 lines

84 lines

37 lines

10 lines

193 lines

64 lines

118 lines

20 lines

12 lines

higher-order-functions.flix

14 lines

1 line

52 lines

48 lines

104 lines

48 lines

98 lines

66 lines

115 lines

81 lines

75 lines

91 lines

23 lines

24 lines

65 lines

67 lines

3 lines

78 lines

26 lines

78 lines

51 lines

111 lines

90 lines

perspShape_Final.0001.rib

186 lines

33 lines

45 lines

53 lines

24 lines

238 lines

19 lines

regression_model_example.stan

33 lines

20 lines

45 lines

134 lines

108 lines

22 lines

64 lines

58 lines

87 lines

51 lines

15 lines

20 lines

51 lines

96 lines

83 lines

subunit_v1-min.subunit

9 lines

13 lines

95 lines

23 lines

18 lines

79 lines

59 lines

112 lines

168 lines

64 lines

37 lines

42 lines

91 lines

49 lines

35 lines

44 lines

30 lines

filenames/

33 lines

68 lines

51 lines

14 lines

34 lines

150 lines

12 lines

64 lines

18 lines

other/

extensions/

197 lines

	tests/	resources/	contents/	code/	extensions/

urls.py

30 lines

urls.py

30 lines

	swh/	web/	tests/	resources/	contents/	code/	filenames/

	Makefile
	Makefile.local

64 lines

Add end to end tests for content renderingClosedPublicActions

Details

Diff Detail

Event Timeline

Large Diff

Revision ContentsChangeset List

Diff 5784

.gitignore

Makefile.local

cypress.json

cypress/fixtures/example.json

cypress/integration/content-rendering.spec.js

cypress/plugins/index.js

swh/web/common/highlightjs.py

swh/web/config.py

swh/web/settings/tests.py

swh/web/tests/data.py

swh/web/tests/resources/contents/code/extensions/3556336_prac4q1.s

swh/web/tests/resources/contents/code/extensions/AccumFrag.glsl

swh/web/tests/resources/contents/code/extensions/AnnotationCacheAspect.aj

swh/web/tests/resources/contents/code/extensions/ApplicationJobRoutesTest.kt

swh/web/tests/resources/contents/code/extensions/Aux.v

swh/web/tests/resources/contents/code/extensions/BinarySearch.swift

swh/web/tests/resources/contents/code/extensions/BuildPDFDocs.nb

swh/web/tests/resources/contents/code/extensions/CarBuilder.php

swh/web/tests/resources/contents/code/extensions/ConvertReal.dcl

swh/web/tests/resources/contents/code/extensions/ConvertReal.icl

swh/web/tests/resources/contents/code/extensions/Converter.java

swh/web/tests/resources/contents/code/extensions/Crayons.leaf

swh/web/tests/resources/contents/code/extensions/DBRelease.zep

swh/web/tests/resources/contents/code/extensions/DadosSubject.cls

swh/web/tests/resources/contents/code/extensions/Demo.au3

swh/web/tests/resources/contents/code/extensions/DigitalSignature.pas

swh/web/tests/resources/contents/code/extensions/FirstModule.vb

swh/web/tests/resources/contents/code/extensions/FormsMaps.cs

swh/web/tests/resources/contents/code/extensions/FreeUsage.scala

swh/web/tests/resources/contents/code/extensions/Get Receipts from Notes.applescript

swh/web/tests/resources/contents/code/extensions/Grammar.bnf

swh/web/tests/resources/contents/code/extensions/JQueryProvider.groovy

swh/web/tests/resources/contents/code/extensions/JsRT.ahk

swh/web/tests/resources/contents/code/extensions/Lazy.sml

swh/web/tests/resources/contents/code/extensions/Node.p

swh/web/tests/resources/contents/code/extensions/Number.extension.st

swh/web/tests/resources/contents/code/extensions/ParenMatcherHighlighter.cpp

swh/web/tests/resources/contents/code/extensions/PolarisResponse.Class.ps1

swh/web/tests/resources/contents/code/extensions/Preload.wl

swh/web/tests/resources/contents/code/extensions/README.md

swh/web/tests/resources/contents/code/extensions/RangeQuery.hs

swh/web/tests/resources/contents/code/extensions/RunScript.hx

swh/web/tests/resources/contents/code/extensions/Runtime.fs

swh/web/tests/resources/contents/code/extensions/Semver.re

swh/web/tests/resources/contents/code/extensions/SunflowerPattern.pde

swh/web/tests/resources/contents/code/extensions/Table 5-2-1.do

swh/web/tests/resources/contents/code/extensions/TimeRTCSet.ino

swh/web/tests/resources/contents/code/extensions/TmxMap.as

swh/web/tests/resources/contents/code/extensions/TouchFile.pbi

swh/web/tests/resources/contents/code/extensions/TrieNode.js

swh/web/tests/resources/contents/code/extensions/Tuple.elm

swh/web/tests/resources/contents/code/extensions/UseAnd.feature

swh/web/tests/resources/contents/code/extensions/Utils.pm

swh/web/tests/resources/contents/code/extensions/_functions.styl

swh/web/tests/resources/contents/code/extensions/abilis_tb100_dvk.dts

swh/web/tests/resources/contents/code/extensions/abs.ll

swh/web/tests/resources/contents/code/extensions/acme_screw.scad

swh/web/tests/resources/contents/code/extensions/admin.tcl

swh/web/tests/resources/contents/code/extensions/ale-fix-suggest.vim

swh/web/tests/resources/contents/code/extensions/apptype.d

swh/web/tests/resources/contents/code/extensions/assertions.xqy

swh/web/tests/resources/contents/code/extensions/badge.less

swh/web/tests/resources/contents/code/extensions/boss_flash.erl

swh/web/tests/resources/contents/code/extensions/build.gradle

swh/web/tests/resources/contents/code/extensions/cache.ts

swh/web/tests/resources/contents/code/extensions/calculator.capnp

swh/web/tests/resources/contents/code/extensions/color.sh

swh/web/tests/resources/contents/code/extensions/combined.tap

swh/web/tests/resources/contents/code/extensions/cp.bat

swh/web/tests/resources/contents/code/extensions/database.properties

swh/web/tests/resources/contents/code/extensions/dde_default_alg.jl

swh/web/tests/resources/contents/code/extensions/display.ex

swh/web/tests/resources/contents/code/extensions/engine.js.erb

Add end to end tests for content rendering
ClosedPublic
Actions

Revision Contents
Changeset List