Page MenuHomeSoftware Heritage

visit_stats: Keep a reference to the last snapshot
AbandonedPublicDraft

Authored by ardumont on Jan 13 2021, 3:37 PM.

Details

Summary

To allow more fine-grained deducation on the eventful/uneventful nature of last visits

Related to T2967
Depends on D4852

Test Plan

tox
(failing until swh.model > 0.9 [1] is released)

[1] D4848

Diff Detail

Repository
rDSCH Scheduling utilities
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 18331
Build 28314: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 28313: arc lint + arc unit

Event Timeline

Build has FAILED

Patch application report for D4853 (id=17180)

Could not rebase; Attempt merge onto a62003397d...

Updating a620033..bf43179
Fast-forward
 sql/updates/20.sql                         |  4 ++
 swh/scheduler/backend.py                   | 10 ++-
 swh/scheduler/journal_client.py            | 47 +++++++++++++++
 swh/scheduler/model.py                     |  3 +
 swh/scheduler/sql/10-superuser-init.sql    |  1 +
 swh/scheduler/sql/30-schema.sql            |  5 +-
 swh/scheduler/tests/test_journal_client.py | 97 ++++++++++++++++++++++++++++++
 swh/scheduler/tests/test_scheduler.py      | 18 ++++++
 8 files changed, 181 insertions(+), 4 deletions(-)
 create mode 100644 sql/updates/20.sql
 create mode 100644 swh/scheduler/journal_client.py
 create mode 100644 swh/scheduler/tests/test_journal_client.py
Changes applied before test
commit bf431797f29ea447e82adb3f7c397840300d0e59
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jan 13 15:32:36 2021 +0100

    visit_stats: Keep a reference to the last snapshot
    
    Related to T2967

commit ab9871d3f14ccc4510ef0912af45716259630878
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jan 13 13:03:28 2021 +0100

    Populate origin_visit_stats table out of the origin_visit_status topic
    
    Related to T2967

Link to build: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/102/
See console output for more information: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/102/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 13 2021, 3:39 PM
Harbormaster failed remote builds in B18324: Diff 17180!

Fix blank line, drop spurious "new" term...

Build has FAILED

Patch application report for D4853 (id=17187)

Could not rebase; Attempt merge onto a62003397d...

Updating a620033..1e05e71
Fast-forward
 sql/updates/20.sql                         |  4 ++
 swh/scheduler/backend.py                   | 10 ++-
 swh/scheduler/journal_client.py            | 47 +++++++++++++++
 swh/scheduler/model.py                     |  3 +
 swh/scheduler/sql/10-superuser-init.sql    |  1 +
 swh/scheduler/sql/30-schema.sql            |  4 +-
 swh/scheduler/tests/test_journal_client.py | 97 ++++++++++++++++++++++++++++++
 swh/scheduler/tests/test_scheduler.py      | 18 ++++++
 8 files changed, 181 insertions(+), 3 deletions(-)
 create mode 100644 sql/updates/20.sql
 create mode 100644 swh/scheduler/journal_client.py
 create mode 100644 swh/scheduler/tests/test_journal_client.py
Changes applied before test
commit 1e05e7154156a481f86d08c41e6986b6a3b5ef99
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jan 13 15:32:36 2021 +0100

    visit_stats: Keep a reference to the last snapshot
    
    Related to T2967

commit ab9871d3f14ccc4510ef0912af45716259630878
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jan 13 13:03:28 2021 +0100

    Populate origin_visit_stats table out of the origin_visit_status topic
    
    Related to T2967

Link to build: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/108/
See console output for more information: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/108/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 13 2021, 3:54 PM
Harbormaster failed remote builds in B18331: Diff 17187!
  • Drop pgcrypto extension (added but unused), i planned to defined sha1git but did not so

not useful here.

  • rename 20.sql into 21.sql (i D4855 adding it already, let's not conflict ;)

Build has FAILED

Patch application report for D4853 (id=17195)

Could not rebase; Attempt merge onto a62003397d...

Updating a620033..be20e09
Fast-forward
 sql/updates/21.sql                         |  4 ++
 swh/scheduler/backend.py                   | 10 ++-
 swh/scheduler/journal_client.py            | 47 +++++++++++++++
 swh/scheduler/model.py                     |  3 +
 swh/scheduler/sql/30-schema.sql            |  4 +-
 swh/scheduler/tests/test_journal_client.py | 97 ++++++++++++++++++++++++++++++
 swh/scheduler/tests/test_scheduler.py      | 18 ++++++
 7 files changed, 180 insertions(+), 3 deletions(-)
 create mode 100644 sql/updates/21.sql
 create mode 100644 swh/scheduler/journal_client.py
 create mode 100644 swh/scheduler/tests/test_journal_client.py
Changes applied before test
commit be20e09b877837302e1aeade259be29a93988798
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jan 13 15:32:36 2021 +0100

    visit_stats: Keep a reference to the last snapshot
    
    Related to T2967

commit ab9871d3f14ccc4510ef0912af45716259630878
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jan 13 13:03:28 2021 +0100

    Populate origin_visit_stats table out of the origin_visit_status topic
    
    Related to T2967

Link to build: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/112/
See console output for more information: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/112/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 13 2021, 6:48 PM
Harbormaster failed remote builds in B18339: Diff 17195!