Page MenuHomeSoftware Heritage

Reference ingested but discontinued bitbucket mercurial origins
ClosedPublic

Authored by ardumont on Oct 14 2021, 3:42 PM.

Details

Summary

Either that or something more evolved as discussed in the related task.

I'd argue that i'd very much like to close the bitbucket origin ingestion which has been
ongoing for months now. So that naive approach suits me well enough right now.

We can always evolve this a bit more later. There is a dedicated opened task btw to
actually discuss on how to improve the coverage setup and avoid those kind of static
dicts (i don't find back the reference).

For this to evolve here, that'd require at least an evolution in the scheduler 'listers'
table to have a 'discontinued' flag (or something with a better name) that we could lift
to actually query on {lister name, instance name, visit_type}. That'd also require we
reference actual origins properly in listed_origins. And that could be done rather
easily for the bitbucket mercurial origins, but not necessarily for the other ones (in
the modified dict from the diff).

Related to T3658

Diff Detail

Repository
rDWAPPS Web applications
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 24434
Build 38131: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 38130: arc lint + arc unit

Event Timeline

Build was aborted

Patch application report for D6475 (id=23524)

Rebasing onto a200f70f2c...

First, rewinding head to replay your work on top of it...
Applying: tests: Turn content* hypothesis strategies into pytest fixtures
Using index info to reconstruct a base tree...
M	swh/web/tests/api/test_apiresponse.py
M	swh/web/tests/api/test_utils.py
M	swh/web/tests/api/views/test_content.py
M	swh/web/tests/api/views/test_identifiers.py
M	swh/web/tests/api/views/test_release.py
M	swh/web/tests/api/views/test_revision.py
M	swh/web/tests/browse/views/test_content.py
M	swh/web/tests/browse/views/test_identifiers.py
M	swh/web/tests/common/test_archive.py
M	swh/web/tests/common/test_identifiers.py
M	swh/web/tests/conftest.py
M	swh/web/tests/misc/test_badges.py
M	swh/web/tests/misc/test_iframe.py
M	swh/web/tests/strategies.py
Falling back to patching base and 3-way merge...
Auto-merging swh/web/tests/strategies.py
CONFLICT (content): Merge conflict in swh/web/tests/strategies.py
Auto-merging swh/web/tests/misc/test_iframe.py
CONFLICT (content): Merge conflict in swh/web/tests/misc/test_iframe.py
Auto-merging swh/web/tests/misc/test_badges.py
CONFLICT (content): Merge conflict in swh/web/tests/misc/test_badges.py
Auto-merging swh/web/tests/conftest.py
CONFLICT (content): Merge conflict in swh/web/tests/conftest.py
Auto-merging swh/web/tests/common/test_identifiers.py
CONFLICT (content): Merge conflict in swh/web/tests/common/test_identifiers.py
Auto-merging swh/web/tests/common/test_archive.py
CONFLICT (content): Merge conflict in swh/web/tests/common/test_archive.py
Auto-merging swh/web/tests/browse/views/test_identifiers.py
CONFLICT (content): Merge conflict in swh/web/tests/browse/views/test_identifiers.py
Auto-merging swh/web/tests/browse/views/test_content.py
CONFLICT (content): Merge conflict in swh/web/tests/browse/views/test_content.py
Auto-merging swh/web/tests/api/views/test_revision.py
CONFLICT (content): Merge conflict in swh/web/tests/api/views/test_revision.py
Auto-merging swh/web/tests/api/views/test_release.py
CONFLICT (content): Merge conflict in swh/web/tests/api/views/test_release.py
Auto-merging swh/web/tests/api/views/test_identifiers.py
CONFLICT (content): Merge conflict in swh/web/tests/api/views/test_identifiers.py
Auto-merging swh/web/tests/api/test_utils.py
CONFLICT (content): Merge conflict in swh/web/tests/api/test_utils.py
Auto-merging swh/web/tests/api/test_apiresponse.py
CONFLICT (content): Merge conflict in swh/web/tests/api/test_apiresponse.py
Patch failed at 0001 tests: Turn content* hypothesis strategies into pytest fixtures

Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".

Rebase failed (ret=1)!

Could not rebase; Attempt merge onto a200f70f2c...

Already up to date.
Changes applied before test
commit 8f6c7538fbb786e5d895d53e47893bbd07f6f817
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 14 15:33:21 2021 +0200

    Reference discontinued ingested but discontinued bitbucket origins
    
    Related to T3658

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1176/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1176/console

ardumont retitled this revision from Reference discontinued ingested but discontinued bitbucket origins to Reference ingested but discontinued bitbucket mercurial origins.Oct 14 2021, 3:50 PM

Build is green

Patch application report for D6475 (id=23525)

Rebasing onto a200f70f2c...

Current branch diff-target is up to date.
Changes applied before test
commit 8f6c7538fbb786e5d895d53e47893bbd07f6f817
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 14 15:33:21 2021 +0200

    Reference discontinued ingested but discontinued bitbucket origins
    
    Related to T3658

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1177/ for more details.

anlambert added a subscriber: anlambert.

Looks good to me, and yes those static dicts should be moved out of swh-web but I do not have a proper solution in mind so far.

This revision is now accepted and ready to land.Oct 14 2021, 5:10 PM

yes, those static dicts should be moved out of swh-web but I do not have a proper solution in mind so far.

yup, same here ;)

Well, we have a hint iirc, use swh-web's db model or something so we can avoid having to deploy new
swh web version of the code

This revision was landed with ongoing or failed builds.Oct 14 2021, 5:51 PM
This revision was automatically updated to reflect the committed changes.

Build is green

Patch application report for D6475 (id=23534)

Rebasing onto ed2c2dbed9...

Current branch diff-target is up to date.
Changes applied before test
commit 4162a34683bfd74fc9c2af273aa2938cd3073533
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 14 15:33:21 2021 +0200

    Reference ingested but discontinued bitbucket mercurial origins
    
    Related to T3658

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1184/ for more details.