Page MenuHomeSoftware Heritage

Add isort pre-commit hook and configuration to all repos
Closed, MigratedEdits Locked

Description

Sorting Python imports manually is quite cumbersome so we should use the isort tool to handle that process
and get consistent imports ordering across all swh modules.

As an example, D3972 adds a new pre-commit hook in swh-web to call isort when committing with a configuration
compatible with black formatter.

That task is here to define the isort configuration that will be used in all modules. Proposed one is the following:

[tool.isort]
multi_line_output = 3
include_trailing_comma = true
force_grid_wrap = 0
use_parentheses = true
ensure_newline_before_comments = true
line_length = 88
force_sort_within_sections = true

Basically it corresponds to the recommended settings when using black plus the addition of force_sort_within_sections = true
(which sorts from x import y before import z instead of grouping from imports together at the bottom)

More options are available here.

Related Objects

Mentioned In
D3519: Add isort configuration
rDWCLI257409e11b64: python: Reorder imports with isort
rDWCLI814057d33bde: pre-commit: Add isort hook and configuration
rDWAPPSa35c45e86248: python: Reorder imports with isort
rDWAPPSd24922fc881c: pre-commit: Add isort hook and configuration
rDVAU2b6cc78112de: python: Reorder imports with isort
rDVAUcf35fb7e1c1c: pre-commit: Add isort hook and configuration
rDSTOb0027abc340e: python: Reorder imports with isort
rDSTOd27a04647b0c: pre-commit: Add isort hook and configuration
rDSEA74e1faf705c6: python: Reorder imports with isort
rDSEA9b7516f4dcdc: pre-commit: Add isort hook and configuration
rDSCH7b0d48fa2429: python: Reorder imports with isort
rDSCH8d8b58f4458e: pre-commit: Add isort hook and configuration
rDTSCNf47dc80067e5: python: Reorder imports with isort
rDTSCN64381b2599a4: pre-commit: Add isort hook and configuration
rDTPLf7cf1023b6ca: python: Reorder imports with isort
rDTPL53ca9e43e5f9: pre-commit: Add isort hook and configuration
rDOBJSRPL91adace75cba: python: Reorder imports with isort
rDOBJSRPL915721a13c47: pre-commit: Add isort hook and configuration
rDOBJSeca35aa11ba8: python: Reorder imports with isort
rDOBJS35a796bfd629: pre-commit: Add isort hook and configuration
rDMODa27371850ac1: python: Reorder imports with isort
rDMOD5bda22fb4827: pre-commit: Add isort hook and configuration
rDLDSVN40f407c718b6: python: Reorder imports with isort
rDLDSVNc1ac93225e22: pre-commit: Add isort hook and configuration
rDLDHG6d22cbebd237: python: Reorder imports with isort
rDLDHG21a66f4fafcf: pre-commit: Add isort hook and configuration
rDLDGd733c4c48420: python: Reorder imports with isort
rDLDG152eece9aff1: pre-commit: Add isort hook and configuration
rDLDBASE7b2c80e708f1: python: Reorder imports with isort
rDLDBASEcc2ffe15369b: pre-commit: Add isort hook and configuration
rDLS22f71812945e: python: Reorder imports with isort
rDLSd24846a95709: pre-commit: Add isort hook and configuration
rDJNLc8a14b0a7c9a: python: Reorder imports with isort
rDJNLb5969a7a3b10: pre-commit: Add isort hook and configuration
rDCIDX27524b20f5ad: python: Reorder imports with isort
rDCIDXc6fcd222d9af: pre-commit: Add isort hook and configuration
rDICPca1f5f7368e6: python: Reorder imports with isort
rDICP832a11a6dcae: pre-commit: Add isort hook and configuration
rDDOC643141a212a0: python: Reorder imports with isort
rDDOCb60869d8116e: pre-commit: Add isort hook and configuration
rDDEPa12b1195b0d2: python: Reorder imports with isort
rDDEP52c057dbf9c5: pre-commit: Add isort hook and configuration
rDDATASET48ba2a402142: python: Reorder imports with isort
rDDATASETc33e8ea7aac5: pre-commit: Add isort hook and configuration
rDCORE28d61c8cf149: python: Reorder imports with isort
rDCOREba8a97830949: pre-commit: Add isort hook and configuration
D3972: pre-commit: Add isort hook and configuration
Mentioned Here
D3972: pre-commit: Add isort hook and configuration

Event Timeline

+1 to the idea (and to the specific config as I have already bikeshed it)

You might need to skip .py files in tests/*/data (eg. swh-web iirc)

LGTM

(I was initially surprised by the mixing together of "import" lines with "from ... import" ones, but upon reflection it makes a lot of sense, because one might have to switch between the two forms, and it's silly to have to move the line back and forth between import blocks when that happens.)

You might need to skip .py files in tests/*/data (eg. swh-web iirc)

Right ! Once global isort config will be pushed to repos, I will update D3972 with specific swh-web configuration only (django import grouping, paths exclusion).

Ok so the plan to integrate isort in all repos will be the following:

  1. Integrate the following diff (manually) in all repos using the change-all-repos script from @vlorentz but do not push to remote yet
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 5ab0e08..df70797 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -26,6 +26,11 @@ repos:
     language: system
     types: [python]
 
+- repo: https://github.com/PyCQA/isort
+  rev: 5.5.2
+  hooks:
+  - id: isort
+
 - repo: https://github.com/python/black
   rev: 19.10b0
   hooks:
diff --git a/pyproject.toml b/pyproject.toml
index b5413f6..206af2c 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,2 +1,12 @@
 [tool.black]
 target-version = ['py37']
+
+[tool.isort]
+multi_line_output = 3
+include_trailing_comma = true
+force_grid_wrap = 0
+use_parentheses = true
+ensure_newline_before_comments = true
+line_length = 88
+force_sort_within_sections = true
+
  1. Run make check in all repos to apply imports ordering
  2. Run make test in all repos to check and fix possible side effects
  3. Once it is done, add a new commit for the imports ordering
  4. Push to remotes

Ok so I applied the above process locally and no bad surprises when running all test suites. I will push the changes to remote then and close that task once it is done.

anlambert claimed this task.

All repositories have been successfully processed, closing this.