Page MenuHomeSoftware Heritage

webapp: Following getting started fails with node dependencies error
Closed, MigratedEdits Locked

Description

Regarding the webapp, following the getting started on a new machine
results is not enough:

sudo apt install nodejs npm
cd swh-web
npm install

This result in an error about node dependencies:

$ make build-webpack-dev

...

ERROR in ./node_modules/bootstrap-loader/no-op.js (./node_modules/cache-loader/dist/cjs.js!./node_modules/css-loader?sourceMap!./node_modules/postcss-loader/src?sourceMap!./node_modules/resolve-url-loader!./node_modules/sass-loader/lib/loader.js?sourceMap!./node_modules/bootstrap-loader/lib/bootstrap.styles.loader.js?{ "bootstrapVersion":4,"preBootstrapCustomizations": "/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/assets/config/bootstrap-pre-custom   ize.scss","appStyles": "/home/tony/work/inria/repo/swh/swh-environment/swh-web/node_modules/admin-lte/build/scss/AdminLTE-raw.scss","extractStyles":true,"styleLoaders": ["cache-loader","css-loader?sourceMap","postcss-loader?sourceMap","sass-loader?sourceMap"],"styles": ["mixins","print","buttons","code","forms","grid","images","reboot","tables","type","alert","badge","breadcrumb","button-group","card","close","custom-forms","dropdown","input-group","jumbotron","list-group","media","nav","navbar","pagination","progress","transitions","carousel","modal","popover","tooltip","utilities"],"scripts": ["alert","button","carousel","collapse","dropdown","modal","popover","scrollspy","tab","tooltip","util"],"configFilePath": "/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/assets/config/.bootstraprc","bootstrapPath": "/home/tony/work/inria/repo/swh/swh-environment/swh-web/node_modules/bootstrap","bootstrapRelPath": "../bootstrap"}!./node_modules/bootstrap-loader/no-op.js)
Module build failed (from ./node_modules/sass-loader/lib/loader.js):
Error: Missing binding /home/tony/work/inria/repo/swh/swh-environment/swh-web/node_modules/node-sass/vendor/linux-x64-57/binding.node
Node Sass could not find a binding for your current environment: Linux 64-bit with Node.js 8.x

Found bindings for the following environments:
- Linux 64-bit with Node.js 10.x

This usually happens because your environment has changed since running `npm install`.
Run `npm rebuild node-sass` to download the binding for your current environment.
at module.exports (/home/tony/work/inria/repo/swh/swh-environment/swh-web/node_modules/node-sass/lib/binding.js:15:13)
at Object.<anonymous> (/home/tony/work/inria/repo/swh/swh-environment/swh-web/node_modules/node-sass/lib/index.js:14:35)
at Module._compile (/home/tony/work/inria/repo/swh/swh-environment/swh-web/node_modules/v8-compile-cache/v8-compile-cache.js:178:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (/home/tony/work/inria/repo/swh/swh-environment/swh-web/node_modules/v8-compile-cache/v8-compile-cache.js:159:20)
at Object.sassLoader (/home/tony/work/inria/repo/swh/swh-environment/swh-web/node_modules/sass-loader/lib/loader.js:46:72)
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! swh-web@0.0.1 build-dev: `NODE_ENV=development webpack --config ./swh/web/assets/config/webpack.config.development.js  --display-modules --progress --colors`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the swh-web@0.0.1 build-dev script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/tony/.npm/_logs/2018-09-17T10_31_07_962Z-debug.log
make: *** [Makefile.local:3: build-webpack-dev] Error 2

According to this log, we need at least 10.x:
And indeed, in debian testing, i'm currently at 8:

$ node --version
v8.11.2
$ dpkg -l nodejs | grep nodejs
ii  nodejs                                    8.11.2~dfsg-1             amd64                     evented I/O for V8 javascript$

Checking further, all debian sources are at best at that version 8.11
[1].

The official documentation [2] proposes the use of a shell script:

curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
sudo apt-get install -y nodejs

Or, next best thing, use of the official debian repository [3] (which
like i said are late).

Checking further and notably the swh-web readme, there is a manual
approach referenced which sounds better [4] [5].

Basically, i had to add a public key (nodesource's package signing key)

https://deb.nodesource.com/gpgkey/nodesource.gpg.key

and edited a file /etc/apt/sources.list.d/nodesource.list:

deb https://deb.nodesource.com/node10.x buster main

Then:

sudo apt update
sudo apt install -y nodejs

Note: No need for npm in this case, it comes from that nodejs package.

And now, everything worked as documented.

[1] https://packages.debian.org/source/nodejs

[2] https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions

[3] https://packages.debian.org/search?searchon=names&keywords=nodejs

[4] https://forge.softwareheritage.org/source/swh-web/

[5] https://github.com/nodesource/distributions#debmanual

Cheers,

Event Timeline

ardumont triaged this task as Normal priority.Sep 17 2018, 1:45 PM
ardumont created this task.

According to this log, we need at least 10.x:

I've a counter example to this: I've node 8.6.0 and it works for me.

Anyway, I think the root of the issue here is that we should not expect people who just want to hack on the web app locally to be able to build the webpack. AFAICT building the webpack is the only reason why we need node, right? If so, that's ~400+ MB of nodes dependencies just for a speed optimization.

@anlambert: would it be possible to make the web app work without webpack, when in developer/debug mode? Or would it be too much a hurdle?

I've a counter example to this: I've node 8.6.0 and it works for me.

...

Anyway, ...

Quite.

Note that what i have gathered here is from a fresh install (directly to a debian 'testing' for unrelated hardware issues).
I had the issues already a while back with another machine (and i got out of it that way).

I think the root of the issue here is that we should not expect people who just want to hack on the web app locally to be able to build the webpack. AFAICT building the webpack is the only reason why we need node, right?

Possibly, yes.

If so, that's ~400+ MB of nodes dependencies just for a speed optimization.

heh ;)

A speed optimization for development purposes but not only that. IIRC, there are some minification and other nice things as well.

@anlambert: would it be possible to make the web app work without webpack, when in developer/debug mode?

Indeed, that would be easier for newcomers to render those dependencies optionals.
At least, at first, then they could so choose to speed things up a bit using the node things.

Cheers,

Honestly, it would be too much of a hurdle. Using node/npm is quite common in web development today
and users who want to hack on the webapp locally should not be suprised about having to issue a 'npm install'
command before being able to run it (this is also explained in the README).

Node is required for developing swh-web for multiple reasons.

It allows to manage and retrieve the needed dependencies (for instance bootstrap, highlightjs, web fonts, ...) through npm or yarn.
This is much more convenient as storing all of them in the git repository, especially as new versions are released.

It is also needed to run webpack in order to process all those dependencies and organize them into logical bundles.
Webpack is not about speed optimization but rather about frontend development tooling and better code
organization.
Webpack also allows to greatly improve frontend development through the features it offered by its core
and its plugins, notably:

  • can import all javascript module formats: CommonJS, UMD, ES6
  • use of es6 syntax for javascript code (which then gets transpiled to es5 for compatibility with older browsers)
  • use of css postprocessors
  • liniting of js/css code
  • compilation on the fly of sass to css (swh-web overrides some global bootstrap sass variables for instance)
  • development server serving compiled assets from memory with automatic reloading of the application in the browser when modifying source files
  • minification and dead code elimination in production mode

To summarize, webpack is really usefull for better frontend development so I would prefer to not drop its use.

Honestly, it would be too much of a hurdle. Using node/npm is quite common in web development today
and users who want to hack on the webapp locally should not be suprised about having to issue a 'npm install'
command before being able to run it (this is also explained in the README).

Ok, fair enough. But please let's make a point of depending only on the LTS version of node.

Speaking of which: any idea what's causing @ardumont issue here? Because it does work fine for me with node LTS (from Debian testing).

Not really. I am using node LTS 8.x since a while too and I never encountered such an issue with node-sass.

I've a counter example to this: I've node 8.6.0 and it works for me.
Not really. I am using node LTS 8.x since a while too and I never encountered such an issue with node-sass.

We need something like "need reproduction" or something... ;)

zack lowered the priority of this task from Normal to Low.Sep 17 2018, 3:09 PM
zack added a project: Unreproducible.

I found out the issue.

I occurred to me during my ride back that it was all about dangling data.
Among other data, I rsync'ed my swh-web's node_modules/ folder (in .gitignore).

So back to square one, uninstalling all things, installing the initial setup:

sudo apt purge nodejs -y
sudo rm /etc/apt/sources.list.d/nodesource.list
 rm -rf swh-web/node_modules
sudo apt update
sudo apt install nodejs npm
cd swh-web
npm install
make test

And everything is good!

Closing now.

The good news is, we have some stuff to show to people having the same problem \m/

Cheers,

ardumont claimed this task.