Page MenuHomeSoftware Heritage
Differential D4311

Add testing repository builder
ClosedPublic
Actions

Authored by acezar on Oct 19 2020, 10:30 PM.

Details

Reviewers
douardda
Group Reviewers
Reviewers
Commits
rDLDHGff11f77f1b49: Add testing repository builder
Summary

This build script purpose is to create example repositories from bash scripts
and extract assertion data from them into json files.

Advantages:

  • the bash script documents the repository creation
  • automating creation allow easy repository update
  • automation extraction allow easier update of assertion data

Diff Detail

Repository
rDLDHG Mercurial loader
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

acezar created this revision.Oct 19 2020, 10:30 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptOct 19 2020, 10:30 PM
swh-public-ci added a comment.Oct 19 2020, 10:31 PM

Build has FAILED

Patch application report for D4311 (id=15239)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..07f1b37
Fast-forward
 setup.py                                    |   2 +
 swh/loader/mercurial/identify.py            | 494 ++++++++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py    | 187 +++++++++++
 swh/loader/mercurial/tests/test_identify.py |  88 +++++
 4 files changed, 771 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit 07f1b3730f6659133a2431c45d6b01696fe9da45
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    Depends on D4216

commit b543125f433ab3022cc80c5b76e3a2eab0162b61
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/62/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/62/console

Harbormaster failed remote builds in B16472: Diff 15239!Oct 19 2020, 10:31 PM
acezar added a child revision: D4312: Add `Hg20BundleLoader` tests from json files.Oct 19 2020, 10:47 PM
douardda added a subscriber: douardda.Oct 23 2020, 11:21 AM

please do not put the "depends on Dxxx" line in the git commit message.

douardda added a comment.Oct 23 2020, 11:25 AM

Also the commit message should give a bit more information on what this new script is needed for, maybe with a usage example.
This defines a bunch of commands. When and how should "I" use them?

swh/loader/mercurial/tests/data/build.py
2

why 2018? Is this copied from elsewhere in the swh space?

douardda requested changes to this revision.Oct 23 2020, 11:25 AM
This revision now requires changes to proceed.Oct 23 2020, 11:25 AM
acezar updated this revision to Diff 15402.Oct 26 2020, 4:46 PM
acezar edited the summary of this revision. (Show Details)

Add and fix documentation

swh-public-ci added a comment.Oct 26 2020, 4:47 PM

Build has FAILED

Patch application report for D4311 (id=15402)

Rebasing onto bd914dec39...

Current branch diff-target is up to date.
Changes applied before test
commit 838149f98d435fce7d809fbe1c324c4c42dc2ba7
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    This build script purpose is to create example repositories from bash scripts
    and extract assertion data from them into json files.
    
    Advantages:
    
        - the bash script documents the repository creation
        - automating creation allow easy repository update
        - automation extraction allow easier update of assertion data

commit c3d959fe1cf5bee461ff7366697f907a578d9288
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/67/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/67/console

Harbormaster failed remote builds in B16631: Diff 15402!Oct 26 2020, 4:47 PM
acezar updated this revision to Diff 15416.Oct 27 2020, 11:05 AM

Fighting arc

swh-public-ci added a comment.Oct 27 2020, 11:06 AM

Build has FAILED

Patch application report for D4311 (id=15416)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..5b02566
Fast-forward
 setup.py                                    |   2 +
 swh/loader/mercurial/identify.py            | 532 ++++++++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py    | 187 ++++++++++
 swh/loader/mercurial/tests/test_identify.py |  88 +++++
 4 files changed, 809 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit 5b0256683349ad892ffed749e7dc45d8b768e5fb
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder

commit c3d959fe1cf5bee461ff7366697f907a578d9288
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/72/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/72/console

Harbormaster failed remote builds in B16643: Diff 15416!Oct 27 2020, 11:06 AM
acezar updated this revision to Diff 15427.Oct 27 2020, 2:52 PM

Get back the right commit

swh-public-ci added a comment.Oct 27 2020, 2:53 PM

Build has FAILED

Patch application report for D4311 (id=15427)

Rebasing onto bd914dec39...

Current branch diff-target is up to date.
Changes applied before test
commit 838149f98d435fce7d809fbe1c324c4c42dc2ba7
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    This build script purpose is to create example repositories from bash scripts
    and extract assertion data from them into json files.
    
    Advantages:
    
        - the bash script documents the repository creation
        - automating creation allow easy repository update
        - automation extraction allow easier update of assertion data

commit c3d959fe1cf5bee461ff7366697f907a578d9288
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/76/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/76/console

Harbormaster failed remote builds in B16656: Diff 15427!Oct 27 2020, 2:53 PM
acezar updated this revision to Diff 15428.Oct 27 2020, 2:53 PM

arc fail again...

swh-public-ci added a comment.Oct 27 2020, 2:54 PM

Build has FAILED

Patch application report for D4311 (id=15428)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..838149f
Fast-forward
 setup.py                                    |   2 +
 swh/loader/mercurial/identify.py            | 532 ++++++++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py    | 265 ++++++++++++++
 swh/loader/mercurial/tests/test_identify.py |  88 +++++
 4 files changed, 887 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit 838149f98d435fce7d809fbe1c324c4c42dc2ba7
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    This build script purpose is to create example repositories from bash scripts
    and extract assertion data from them into json files.
    
    Advantages:
    
        - the bash script documents the repository creation
        - automating creation allow easy repository update
        - automation extraction allow easier update of assertion data

commit c3d959fe1cf5bee461ff7366697f907a578d9288
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/77/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/77/console

Harbormaster failed remote builds in B16657: Diff 15428!Oct 27 2020, 2:55 PM
acezar added inline comments.Oct 27 2020, 4:01 PM
swh/loader/mercurial/tests/data/build.py
2

copied from test_loader.py

acezar added inline comments.Oct 27 2020, 4:10 PM
swh/loader/mercurial/tests/data/build.py
2

Some extraction:

  • swh/loader/mercurial/archive_extract.py: # Copyright (C) 2017-2018
  • swh/loader/mercurial/bundle20_reader.py: # Copyright (C) 2017
  • swh/loader/mercurial/chunked_reader.py: # Copyright (C) 2017
  • swh/loader/mercurial/cli.py: # Copyright (C) 2018
  • swh/loader/mercurial/converters.py: # Copyright (C) 2015-2017
  • swh/loader/mercurial/identify.py: # Copyright (C) 2020
  • swh/loader/mercurial/__init__.py: # Copyright (C) 2019
  • swh/loader/mercurial/loader.py: # Copyright (C) 2017-2020
  • swh/loader/mercurial/objects.py: # Copyright (C) 2017-2018
  • swh/loader/mercurial/tasks.py: # Copyright (C) 2017-2019
  • swh/loader/mercurial/tests/conftest.py: # Copyright (C) 2019-2020
  • swh/loader/mercurial/tests/test_converters.py: # Copyright (C) 2018
  • swh/loader/mercurial/tests/test_loader.py: # Copyright (C) 2018-2020
  • swh/loader/mercurial/tests/test_tasks.py: # Copyright (C) 2018-2020
acezar updated this revision to Diff 15448.Oct 27 2020, 4:47 PM

Update

swh-public-ci added a comment.Oct 27 2020, 4:49 PM

Build has FAILED

Patch application report for D4311 (id=15448)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..838149f
Fast-forward
 setup.py                                    |   2 +
 swh/loader/mercurial/identify.py            | 532 ++++++++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py    | 265 ++++++++++++++
 swh/loader/mercurial/tests/test_identify.py |  88 +++++
 4 files changed, 887 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit 838149f98d435fce7d809fbe1c324c4c42dc2ba7
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    This build script purpose is to create example repositories from bash scripts
    and extract assertion data from them into json files.
    
    Advantages:
    
        - the bash script documents the repository creation
        - automating creation allow easy repository update
        - automation extraction allow easier update of assertion data

commit c3d959fe1cf5bee461ff7366697f907a578d9288
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/80/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/80/console

Harbormaster failed remote builds in B16676: Diff 15448!Oct 27 2020, 4:49 PM
acezar updated this revision to Diff 15452.Oct 27 2020, 4:49 PM

Update

swh-public-ci added a comment.Oct 27 2020, 4:51 PM

Build has FAILED

Patch application report for D4311 (id=15452)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..b4e053f
Fast-forward
 setup.py                                    |   2 +
 swh/loader/mercurial/identify.py            | 532 ++++++++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py    | 265 ++++++++++++++
 swh/loader/mercurial/tests/test_identify.py |  93 +++++
 4 files changed, 892 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit b4e053fdcdd4cd7cdaef915de79c89829186b067
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    This build script purpose is to create example repositories from bash scripts
    and extract assertion data from them into json files.
    
    Advantages:
    
        - the bash script documents the repository creation
        - automating creation allow easy repository update
        - automation extraction allow easier update of assertion data

commit e75b99287ba58bdbc6cb95703f11303c561a8f34
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/82/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/82/console

Harbormaster failed remote builds in B16680: Diff 15452!Oct 27 2020, 4:51 PM
douardda added inline comments.Oct 28 2020, 10:09 AM
swh/loader/mercurial/tests/data/build.py
2

This very file is new, thus the copyright starts in 2020. Did I miss something?

acezar marked 2 inline comments as done.Oct 29 2020, 10:11 AM
acezar added inline comments.
swh/loader/mercurial/tests/data/build.py
2

Was my understanding of the issue which was wrong.

swh-public-ci added a comment.Oct 30 2020, 5:11 PM

Build has FAILED

Patch application report for D4311 (id=15452)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..b4e053f
Fast-forward
 setup.py                                    |   2 +
 swh/loader/mercurial/identify.py            | 532 ++++++++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py    | 265 ++++++++++++++
 swh/loader/mercurial/tests/test_identify.py |  93 +++++
 4 files changed, 892 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit b4e053fdcdd4cd7cdaef915de79c89829186b067
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    This build script purpose is to create example repositories from bash scripts
    and extract assertion data from them into json files.
    
    Advantages:
    
        - the bash script documents the repository creation
        - automating creation allow easy repository update
        - automation extraction allow easier update of assertion data

commit e75b99287ba58bdbc6cb95703f11303c561a8f34
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/87/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/87/console

douardda accepted this revision.Nov 4 2020, 9:25 AM

I'm fine with this, can be landed as soon as tests are ok.

This revision is now accepted and ready to land.Nov 4 2020, 9:25 AM
acezar updated this revision to Diff 15777.Nov 10 2020, 2:39 PM
acezar marked an inline comment as done.

Make sure the diff is up to date.

Maybe I messed something with arc again, since the tests works for me and
another dev on 2 different setups: one with docker the other without docker.

swh-public-ci added a comment.Nov 10 2020, 2:41 PM

Build is green

Patch application report for D4311 (id=15777)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..ea7cc14
Fast-forward
 requirements.txt                            |   1 +
 setup.py                                    |   2 +
 swh/loader/mercurial/identify.py            | 533 ++++++++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py    | 265 ++++++++++++++
 swh/loader/mercurial/tests/test_identify.py |  93 +++++
 5 files changed, 894 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit ea7cc149edbee8778121fc37a099edaedc584f60
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    This build script purpose is to create example repositories from bash scripts
    and extract assertion data from them into json files.
    
    Advantages:
    
        - the bash script documents the repository creation
        - automating creation allow easy repository update
        - automation extraction allow easier update of assertion data

commit 00b709bae8c01854fa9eb37e2b5d9989fa68a17d
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/88/ for more details.

Harbormaster completed remote builds in B16988: Diff 15777.Nov 10 2020, 2:41 PM
acezar updated this revision to Diff 15784.Nov 10 2020, 2:52 PM

Update

Seems that I messed my arc diff --update ci should passe with this.

swh-public-ci added a comment.Nov 10 2020, 2:53 PM

Build is green

Patch application report for D4311 (id=15784)

Could not rebase; Attempt merge onto bd914dec39...

Updating bd914de..ea7cc14
Fast-forward
 requirements.txt                            |   1 +
 setup.py                                    |   2 +
 swh/loader/mercurial/identify.py            | 533 ++++++++++++++++++++++++++++
 swh/loader/mercurial/tests/data/build.py    | 265 ++++++++++++++
 swh/loader/mercurial/tests/test_identify.py |  93 +++++
 5 files changed, 894 insertions(+)
 create mode 100644 swh/loader/mercurial/identify.py
 create mode 100755 swh/loader/mercurial/tests/data/build.py
 create mode 100644 swh/loader/mercurial/tests/test_identify.py
Changes applied before test
commit ea7cc149edbee8778121fc37a099edaedc584f60
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    This build script purpose is to create example repositories from bash scripts
    and extract assertion data from them into json files.
    
    Advantages:
    
        - the bash script documents the repository creation
        - automating creation allow easy repository update
        - automation extraction allow easier update of assertion data

commit 00b709bae8c01854fa9eb37e2b5d9989fa68a17d
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Oct 8 18:07:50 2020 +0200

    add swh-hg-identify a cli to identify hg objects

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/89/ for more details.

Harbormaster completed remote builds in B16995: Diff 15784.Nov 10 2020, 2:53 PM
acezar updated this revision to Diff 16416.Nov 27 2020, 5:26 PM

Cannot push the merge because commit message as be changed
(Probably to remove arcanist data)

swh-public-ci added a comment.Nov 27 2020, 5:27 PM

Build is green

Patch application report for D4311 (id=16416)

Rebasing onto a2e9cf1691...

Current branch diff-target is up to date.
Changes applied before test
commit ff11f77f1b493bd1c8ed257e790ded8da276101c
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Fri Oct 16 11:28:35 2020 +0200

    Add testing repository builder
    
    This build script purpose is to create example repositories from bash scripts
    and extract assertion data from them into json files.
    
    Advantages:
    
        - the bash script documents the repository creation
        - automating creation allow easy repository update
        - automation extraction allow easier update of assertion data

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/125/ for more details.

Harbormaster completed remote builds in B17605: Diff 16416.Nov 27 2020, 5:27 PM
Closed by commit rDLDHGff11f77f1b49: Add testing repository builder (authored by acezar). · Explain WhyNov 27 2020, 5:27 PM
This revision was automatically updated to reflect the committed changes.
acezar added a commit: rDLDHGff11f77f1b49: Add testing repository builder.

Revision Contents
Changeset List

PathSize
swh/
loader/
mercurial/
tests/
data/
265 lines

Diff 16417

View Options

swh/loader/mercurial/tests/data/build.py

Loading...
About · Developer information · Legal