Paths

Table of Contentst

Differential D2981

Apply black formatting to Python code and add pre-commit hook
AbandonedPublic
Actions

Authored by anlambert on Apr 8 2020, 4:08 PM.

Details

Reviewers

None

Group Reviewers

Reviewers

Summary

I heard it was time to move to black formatting in all swh components ;-)

That huge diff contains the following:

reformat the whole swh-web Python codebase using black tool

add pre-commit hook to run black on all changed files

update flake8 configuration to match black code style

remove a lot of useless '# noqa' comments after new formatting

Diff Detail

Repository

rDWAPPS Web applications

Branch

blackify-swh-web

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 11686
Build 17727: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 17726: arc lint + arc unit

Event Timeline

anlambert created this revision.Apr 8 2020, 4:08 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptApr 8 2020, 4:08 PM

Build has FAILED

Patch application report for D2981 (id=10596)

Rebasing onto 2251dea545...

Current branch diff-target is up to date.

Changes applied before test

commit 15834b070bf757168bd27deca953867b90ccd7d0
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Wed Apr 8 15:08:29 2020 +0200

    Apply black formatting to Python code and add pre-commit hook
    
    - reformat the whole swh-web Python codebase using black tool
    
    - add pre-commit hook to run black on all changed files
    
    - update flake8 configuration to match black code style
    
    - remove a lot of useless '# noqa' comments after new formatting

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/49/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/49/console

Harbormaster failed remote builds in B11686: Diff 10596!Apr 8 2020, 4:11 PM

Thanks for this.

We also agreed, on irc, to not review such a huge diff with mostly formatting changes.
So, i guess, let the build finish and if everything is fine, you can push ;)

I checked the setup-ed files (setup.cfg, ...) and they look quite similar as in other diffs so fine ;)

In D2981#72126, @ardumont wrote:

Thanks for this.

We also agreed, on irc, to not review such a huge diff with mostly formatting changes.
So, i guess, let the build finish and if everything is fine, you can push ;)

I checked the setup-ed files (setup.cfg, ...) and they look quite similar as in other diffs so.

I can imagine as the content is pure reformatting ;-) I just need a validation on configuration and test CI build.

Build has FAILED

Patch application report for D2981 (id=10596)

Rebasing onto 2251dea545...

Current branch diff-target is up to date.

Changes applied before test

commit 15834b070bf757168bd27deca953867b90ccd7d0
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Wed Apr 8 15:08:29 2020 +0200

    Apply black formatting to Python code and add pre-commit hook
    
    - reformat the whole swh-web Python codebase using black tool
    
    - add pre-commit hook to run black on all changed files
    
    - update flake8 configuration to match black code style
    
    - remove a lot of useless '# noqa' comments after new formatting

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/50/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/50/console

Seems the black changes makes the build fail as this error appeared:

16:16:45    Running setup.py develop for swh.web
16:16:45      ERROR: Command errored out with exit status 1:
16:16:45       command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/var/lib/jenkins/workspace/DWAPPS/tests-on-diff/setup.py'"'"'; __file__='"'"'/var/lib/jenkins/workspace/DWAPPS/tests-on-diff/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix=
16:16:45           cwd: /var/lib/jenkins/workspace/DWAPPS/tests-on-diff/
16:16:45      Complete output (30 lines):
16:16:45      running develop
16:16:45      /tmp/pip-build-env-egnk76io/overlay/lib/python3.7/site-packages/setuptools/dist.py:472: UserWarning: The version specified ('base-revision-50-D2981.post1') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details.
16:16:45        "details." % version
16:16:45      WARNING: The user site-packages directory is disabled.
16:16:45      error: can't create or remove files in install directory
16:16:45      
16:16:45      The following error occurred while trying to add or remove files in the
16:16:45      installation directory:
16:16:45      
16:16:45          [Errno 13] Permission denied: '/usr/local/lib/python3.7/dist-packages/test-easy-install-61.write-test'
16:16:45      
16:16:45      The installation directory you specified (via --install-dir, --prefix, or
16:16:45      the distutils default setting) was:
16:16:45      
16:16:45          /usr/local/lib/python3.7/dist-packages/
16:16:45      
16:16:45      Perhaps your account does not have write access to this directory?  If the
16:16:45      installation directory is a system-owned directory, you may need to sign in
16:16:45      as the administrator or "root" account.  If you do not have administrative
16:16:45      access to this machine, you may wish to choose a different installation
16:16:45      directory, preferably one that is listed in your PYTHONPATH environment
16:16:45      variable.
16:16:45      
16:16:45      For information on other options, you may wish to consult the
16:16:45      documentation at:
16:16:45      
16:16:45        https://setuptools.readthedocs.io/en/latest/easy_install.html
16:16:45      
16:16:45      Please make the appropriate changes for your system and try again.
16:16:45      
16:16:45      ----------------------------------------
16:16:45    ERROR: Can't roll back swh.web; was not uninstalled
16:16:45  ERROR: Command errored out with exit status 1: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/var/lib/jenkins/workspace/DWAPPS/tests-on-diff/setup.py'"'"'; __file__='"'"'/var/lib/jenkins/workspace/DWAPPS/tests-on-diff/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix= Check the logs for full command output.

I need to dig further on this.

Update: add black processing in tox

Build has FAILED

Patch application report for D2981 (id=10598)

Rebasing onto 2251dea545...

Current branch diff-target is up to date.

Changes applied before test

commit f70a400e3f70c10ee6eb5abfc4fa825b21efb659
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Wed Apr 8 15:08:29 2020 +0200

    Apply black formatting to Python code and add pre-commit hook
    
    - reformat the whole swh-web Python codebase using black tool
    
    - add pre-commit hook to run black on all changed files
    
    - update flake8 configuration to match black code style
    
    - remove a lot of useless '# noqa' comments after new formatting
    
    - add black processing in tox

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/52/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/52/console

Harbormaster failed remote builds in B11693: Diff 10598!Apr 8 2020, 4:48 PM

Update: Unblackify setup.py to check if build still fails

Build has FAILED

Patch application report for D2981 (id=10601)

Rebasing onto 2251dea545...

Current branch diff-target is up to date.

Changes applied before test

commit a9a57aa42b018eb6340003d1ecdbfb5b1ff540bb
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Wed Apr 8 17:22:18 2020 +0200

    Apply black formatting to Python code and add pre-commit hook
    
    - reformat the whole swh-web Python codebase using black tool
    
    - add pre-commit hook to run black on all changed files
    
    - update flake8 configuration to match black code style
    
    - remove a lot of useless '# noqa' comments after new formatting
    
    - add black processing in tox

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/53/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/53/console

Harbormaster failed remote builds in B11705: Diff 10601!Apr 8 2020, 5:25 PM

anlambert mentioned this in D2983: requirements: Cleanup, normalize and sort depedencies.Apr 8 2020, 5:54 PM

Remove pyproject.toml file to check if it has side effect on build

Build is green

Patch application report for D2981 (id=10606)

Rebasing onto 2251dea545...

Current branch diff-target is up to date.

Changes applied before test

commit cb9d52fa3de7bbd17e5011264454ffed921601b1
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Wed Apr 8 17:22:18 2020 +0200

    Apply black formatting to Python code and add pre-commit hook
    
    - reformat the whole swh-web Python codebase using black tool
    
    - add pre-commit hook to run black on all changed files
    
    - update flake8 configuration to match black code style
    
    - remove a lot of useless '# noqa' comments after new formatting
    
    - add black processing in tox

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/56/ for more details.

Harbormaster completed remote builds in B11710: Diff 10606.Apr 8 2020, 6:19 PM