Page MenuHomeSoftware Heritage
Differential D4313

Add new example repository generated from script
ClosedPublic
Actions

Authored by acezar on Oct 19 2020, 10:44 PM.

Details

Reviewers
douardda
Group Reviewers
Reviewers
Commits
rDLDHGc8c91ab674a9: Add new example repository generated from script
Summary

First updatable example repository documented by its generation script.

Diff Detail

Repository
rDLDHG Mercurial loader
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

acezar created this revision.Oct 19 2020, 10:44 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptOct 19 2020, 10:44 PM
swh-public-ci added a comment.Oct 19 2020, 10:45 PM

Build has FAILED

Patch application report for D4313 (id=15242)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..1a58b92
Fast-forward
 setup.py                                          |   2 +
 swh/loader/mercurial/identify.py                  | 494 ++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py          | 187 ++++++++
 swh/loader/mercurial/tests/data/example.json      |   1 +
 swh/loader/mercurial/tests/data/example.sh        |  59 +++
 swh/loader/mercurial/tests/data/example.tgz       | Bin 0 -> 51200 bytes
 swh/loader/mercurial/tests/data/hello.json        |   1 +
 swh/loader/mercurial/tests/data/the-sandbox.json  |   1 +
 swh/loader/mercurial/tests/data/transplant.json   |   1 +
 swh/loader/mercurial/tests/dvcs_loader_checker.py | 132 ++++++
 swh/loader/mercurial/tests/test_identify.py       |  88 ++++
 swh/loader/mercurial/tests/test_loader.py         |  12 +
 12 files changed, 978 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/data/example.json
 create mode 100644 swh/loader/mercurial/tests/data/example.sh
 create mode 100644 swh/loader/mercurial/tests/data/example.tgz
 create mode 100644 swh/loader/mercurial/tests/data/hello.json
 create mode 100644 swh/loader/mercurial/tests/data/the-sandbox.json
 create mode 100644 swh/loader/mercurial/tests/data/transplant.json
 create mode 100644 swh/loader/mercurial/tests/dvcs_loader_checker.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit 1a58b927c23d3aeed6d850b5d404b27bf0d5fdbb
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:42:21 2020 +0200

    Add new example repository generated from script

commit d12497d9e6f544fc8c4c56437c719a6e5c6d9b70
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:22:07 2020 +0200

    Add `Hg20BundleLoader` tests from json files
    
    Summary:
    Generated json files with `swh/loader/mercurial/tests/data/build.py` for
    existing repositories and added them to `Hg20BundleLoader` tests.
    
    Reviewers: #reviewers
    
    Differential Revision: https://forge.softwareheritage.org/D4312

commit 4a92800d8ab7cd108ad621b6d9da1533c09886d3
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder

commit b543125f433ab3022cc80c5b76e3a2eab0162b61
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/64/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/64/console

Harbormaster failed remote builds in B16474: Diff 15242!Oct 19 2020, 10:45 PM
acezar added a parent revision: D4312: Add `Hg20BundleLoader` tests from json files.Oct 19 2020, 10:47 PM
douardda requested changes to this revision.Oct 23 2020, 11:20 AM
douardda added a subscriber: douardda.

ok on the diff itself, but why is this new example repo needed for? This should be explained in the commit message. (the "why"! always insist on the "why" rather than (or in addition to) the "what" in your commit messages, please.

This revision now requires changes to proceed.Oct 23 2020, 11:20 AM
acezar updated this revision to Diff 15408.Oct 27 2020, 10:30 AM
acezar edited the summary of this revision. (Show Details)

Updated commit message

swh-public-ci added a comment.Oct 27 2020, 10:31 AM

Build has FAILED

Patch application report for D4313 (id=15408)

Rebasing onto bd914dec39...

Current branch diff-target is up to date.
Changes applied before test
commit 126fc7955a43b1e3440b2f0cb092f169b6ac8fa7
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:42:21 2020 +0200

    Add new example repository generated from script
    
    First updatable example repository documented by its generation script.

commit f5a7a0cdf21fdf9823cff33895883638bb495415
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:22:07 2020 +0200

    Add `Hg20BundleLoader` tests from json files
    
    Generated json files with `swh/loader/mercurial/tests/data/build.py` for
    existing repositories and added them to `Hg20BundleLoader` tests.
    
    Introduce `DVCSLoaderChecker` as a standardized way to test repositories
    against json files.

commit 5b0256683349ad892ffed749e7dc45d8b768e5fb
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder

commit c3d959fe1cf5bee461ff7366697f907a578d9288
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/70/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/70/console

Harbormaster failed remote builds in B16637: Diff 15408!Oct 27 2020, 10:31 AM
acezar updated this revision to Diff 15410.Oct 27 2020, 10:40 AM

too many things in last diff

swh-public-ci added a comment.Oct 27 2020, 10:41 AM

Build has FAILED

Patch application report for D4313 (id=15410)

Rebasing onto bd914dec39...

Current branch diff-target is up to date.
Changes applied before test
commit 126fc7955a43b1e3440b2f0cb092f169b6ac8fa7
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:42:21 2020 +0200

    Add new example repository generated from script
    
    First updatable example repository documented by its generation script.

commit f5a7a0cdf21fdf9823cff33895883638bb495415
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:22:07 2020 +0200

    Add `Hg20BundleLoader` tests from json files
    
    Generated json files with `swh/loader/mercurial/tests/data/build.py` for
    existing repositories and added them to `Hg20BundleLoader` tests.
    
    Introduce `DVCSLoaderChecker` as a standardized way to test repositories
    against json files.

commit 5b0256683349ad892ffed749e7dc45d8b768e5fb
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder

commit c3d959fe1cf5bee461ff7366697f907a578d9288
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/71/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/71/console

Harbormaster failed remote builds in B16639: Diff 15410!Oct 27 2020, 10:41 AM
acezar updated this revision to Diff 15418.Oct 27 2020, 11:08 AM

Fighting arc

swh-public-ci added a comment.Oct 27 2020, 11:10 AM

Build has FAILED

Patch application report for D4313 (id=15418)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..126fc79
Fast-forward
 setup.py                                          |   2 +
 swh/loader/mercurial/identify.py                  | 532 ++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py          | 187 ++++++++
 swh/loader/mercurial/tests/data/example.json      |   1 +
 swh/loader/mercurial/tests/data/example.sh        |  59 +++
 swh/loader/mercurial/tests/data/example.tgz       | Bin 0 -> 51200 bytes
 swh/loader/mercurial/tests/data/hello.json        |   1 +
 swh/loader/mercurial/tests/data/the-sandbox.json  |   1 +
 swh/loader/mercurial/tests/data/transplant.json   |   1 +
 swh/loader/mercurial/tests/dvcs_loader_checker.py | 132 ++++++
 swh/loader/mercurial/tests/test_identify.py       |  88 ++++
 swh/loader/mercurial/tests/test_loader.py         |  12 +
 12 files changed, 1016 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/data/example.json
 create mode 100644 swh/loader/mercurial/tests/data/example.sh
 create mode 100644 swh/loader/mercurial/tests/data/example.tgz
 create mode 100644 swh/loader/mercurial/tests/data/hello.json
 create mode 100644 swh/loader/mercurial/tests/data/the-sandbox.json
 create mode 100644 swh/loader/mercurial/tests/data/transplant.json
 create mode 100644 swh/loader/mercurial/tests/dvcs_loader_checker.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit 126fc7955a43b1e3440b2f0cb092f169b6ac8fa7
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:42:21 2020 +0200

    Add new example repository generated from script
    
    First updatable example repository documented by its generation script.

commit f5a7a0cdf21fdf9823cff33895883638bb495415
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:22:07 2020 +0200

    Add `Hg20BundleLoader` tests from json files
    
    Generated json files with `swh/loader/mercurial/tests/data/build.py` for
    existing repositories and added them to `Hg20BundleLoader` tests.
    
    Introduce `DVCSLoaderChecker` as a standardized way to test repositories
    against json files.

commit 5b0256683349ad892ffed749e7dc45d8b768e5fb
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder

commit c3d959fe1cf5bee461ff7366697f907a578d9288
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/74/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/74/console

Harbormaster failed remote builds in B16645: Diff 15418!Oct 27 2020, 11:10 AM
douardda added a comment.Oct 27 2020, 11:46 AM

I would expect the commit message to be a bot more explanatory: either this new test case adds some tested aspects that were not tested before, and it should mention it, or it does not, and it should also mention it explaining this new test is the base for futures extended ones in a more manageable way (what's the "updatable" stands for, if I get this right).

acezar updated this revision to Diff 15454.Oct 27 2020, 4:51 PM

Update

swh-public-ci added a comment.Oct 27 2020, 4:54 PM

Build has FAILED

Patch application report for D4313 (id=15454)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..03bca9b
Fast-forward
 setup.py                                          |   2 +
 swh/loader/mercurial/identify.py                  | 532 ++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py          | 265 +++++++++++
 swh/loader/mercurial/tests/data/example.json      |   1 +
 swh/loader/mercurial/tests/data/example.sh        |  59 +++
 swh/loader/mercurial/tests/data/example.tgz       | Bin 0 -> 51200 bytes
 swh/loader/mercurial/tests/data/hello.json        |   1 +
 swh/loader/mercurial/tests/data/the-sandbox.json  |   1 +
 swh/loader/mercurial/tests/data/transplant.json   |   1 +
 swh/loader/mercurial/tests/dvcs_loader_checker.py | 144 ++++++
 swh/loader/mercurial/tests/test_identify.py       |  93 ++++
 swh/loader/mercurial/tests/test_loader.py         |  12 +
 12 files changed, 1111 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/data/example.json
 create mode 100644 swh/loader/mercurial/tests/data/example.sh
 create mode 100644 swh/loader/mercurial/tests/data/example.tgz
 create mode 100644 swh/loader/mercurial/tests/data/hello.json
 create mode 100644 swh/loader/mercurial/tests/data/the-sandbox.json
 create mode 100644 swh/loader/mercurial/tests/data/transplant.json
 create mode 100644 swh/loader/mercurial/tests/dvcs_loader_checker.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit 03bca9bcdd13058bfa03a8b80dee1887f47d8404
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:42:21 2020 +0200

    Add new example repository generated from script
    
    First updatable example repository documented by its generation script.

commit c0d13d25aeaf4658a10ef6c1330208332328205f
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:22:07 2020 +0200

    Add `Hg20BundleLoader` tests from json files
    
    Generated json files with `swh/loader/mercurial/tests/data/build.py` for
    existing repositories and added them to `Hg20BundleLoader` tests.
    
    Introduce `DVCSLoaderChecker` as a standardized way to test repositories
    against json files.

commit b4e053fdcdd4cd7cdaef915de79c89829186b067
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    This build script purpose is to create example repositories from bash scripts
    and extract assertion data from them into json files.
    
    Advantages:
    
        - the bash script documents the repository creation
        - automating creation allow easy repository update
        - automation extraction allow easier update of assertion data

commit e75b99287ba58bdbc6cb95703f11303c561a8f34
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/84/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/84/console

Harbormaster failed remote builds in B16682: Diff 15454!Oct 27 2020, 4:54 PM
douardda accepted this revision.Nov 4 2020, 9:29 AM

same as the previous one, ok once green

This revision is now accepted and ready to land.Nov 4 2020, 9:29 AM
acezar updated this revision to Diff 15787.Nov 10 2020, 2:56 PM

Update to make ci pass

swh-public-ci added a comment.Nov 10 2020, 2:58 PM

Build is green

Patch application report for D4313 (id=15787)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..9bc1c57
Fast-forward
 requirements.txt                                  |   1 +
 setup.py                                          |   2 +
 swh/loader/mercurial/identify.py                  | 533 ++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py          | 265 +++++++++++
 swh/loader/mercurial/tests/data/example.json      |   1 +
 swh/loader/mercurial/tests/data/example.sh        |  59 +++
 swh/loader/mercurial/tests/data/example.tgz       | Bin 0 -> 51200 bytes
 swh/loader/mercurial/tests/data/hello.json        |   1 +
 swh/loader/mercurial/tests/data/the-sandbox.json  |   1 +
 swh/loader/mercurial/tests/data/transplant.json   |   1 +
 swh/loader/mercurial/tests/dvcs_loader_checker.py | 144 ++++++
 swh/loader/mercurial/tests/test_identify.py       |  93 ++++
 swh/loader/mercurial/tests/test_loader.py         |  12 +
 13 files changed, 1113 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/data/example.json
 create mode 100644 swh/loader/mercurial/tests/data/example.sh
 create mode 100644 swh/loader/mercurial/tests/data/example.tgz
 create mode 100644 swh/loader/mercurial/tests/data/hello.json
 create mode 100644 swh/loader/mercurial/tests/data/the-sandbox.json
 create mode 100644 swh/loader/mercurial/tests/data/transplant.json
 create mode 100644 swh/loader/mercurial/tests/dvcs_loader_checker.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit 9bc1c571ca996b2a5cf7a77111cf5995821e4b27
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:42:21 2020 +0200

    Add new example repository generated from script
    
    First updatable example repository documented by its generation script.

commit e4eb5c21afc064812c692b6b36fbbbf1801b3552
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:22:07 2020 +0200

    Add `Hg20BundleLoader` tests from json files
    
    Generated json files with `swh/loader/mercurial/tests/data/build.py` for
    existing repositories and added them to `Hg20BundleLoader` tests.
    
    Introduce `DVCSLoaderChecker` as a standardized way to test repositories
    against json files.

commit ea7cc149edbee8778121fc37a099edaedc584f60
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    This build script purpose is to create example repositories from bash scripts
    and extract assertion data from them into json files.
    
    Advantages:
    
        - the bash script documents the repository creation
        - automating creation allow easy repository update
        - automation extraction allow easier update of assertion data

commit 00b709bae8c01854fa9eb37e2b5d9989fa68a17d
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/91/ for more details.

Harbormaster completed remote builds in B16998: Diff 15787.Nov 10 2020, 2:58 PM
acezar updated this revision to Diff 16419.Nov 27 2020, 5:34 PM

Cannot merge because wrong hash (probably edited to remove arc info)

swh-public-ci added a comment.Nov 27 2020, 5:36 PM

Build is green

Patch application report for D4313 (id=16419)

Rebasing onto bc32e1280c...

Current branch diff-target is up to date.
Changes applied before test
commit c8c91ab674a9ade49caacd63a5b507bab67df9dc
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Oct 19 16:42:21 2020 +0200

    Add new example repository generated from script
    
    First updatable example repository documented by its generation script.

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/126/ for more details.

Harbormaster completed remote builds in B17608: Diff 16419.Nov 27 2020, 5:36 PM
acezar added a comment.Nov 27 2020, 5:37 PM

In fact I forgot to push some renaming

Closed by commit rDLDHGc8c91ab674a9: Add new example repository generated from script (authored by acezar). · Explain WhyNov 27 2020, 5:38 PM
This revision was automatically updated to reflect the committed changes.
acezar added a commit: rDLDHGc8c91ab674a9: Add new example repository generated from script.

Revision Contents
Changeset List

PathSize
swh/
loader/
mercurial/
tests/
data/
1 line
59 lines
2 lines

Diff 16421

View Options

swh/loader/mercurial/tests/data/example.json

Loading...
View Options

swh/loader/mercurial/tests/data/example.sh

Loading...
View Options

swh/loader/mercurial/tests/data/example.tgz

Loading...
View Options

swh/loader/mercurial/tests/test_loader.py

Loading...
About · Developer information · Legal