Page MenuHomeSoftware Heritage
Differential D4853

visit_stats: Keep a reference to the last snapshot
AbandonedPublicDraft
Actions

Authored by ardumont on Jan 13 2021, 3:37 PM.

Details

Reviewers
None
Group Reviewers
Reviewers
Maniphest Tasks
T2967: Write journal client subcribed to origin_visit_status topics
Summary

To allow more fine-grained deducation on the eventful/uneventful nature of last visits

Related to T2967
Depends on D4852

Test Plan

tox
(failing until swh.model > 0.9 [1] is released)

[1] D4848

Diff Detail

Event Timeline

ardumont created this revision.Jan 13 2021, 3:37 PM
swh-public-ci added a comment.Jan 13 2021, 3:39 PM

Build has FAILED

Patch application report for D4853 (id=17180)

Could not rebase; Attempt merge onto a62003397d...

Updating a620033..bf43179
Fast-forward
 sql/updates/20.sql                         |  4 ++
 swh/scheduler/backend.py                   | 10 ++-
 swh/scheduler/journal_client.py            | 47 +++++++++++++++
 swh/scheduler/model.py                     |  3 +
 swh/scheduler/sql/10-superuser-init.sql    |  1 +
 swh/scheduler/sql/30-schema.sql            |  5 +-
 swh/scheduler/tests/test_journal_client.py | 97 ++++++++++++++++++++++++++++++
 swh/scheduler/tests/test_scheduler.py      | 18 ++++++
 8 files changed, 181 insertions(+), 4 deletions(-)
 create mode 100644 sql/updates/20.sql
 create mode 100644 swh/scheduler/journal_client.py
 create mode 100644 swh/scheduler/tests/test_journal_client.py
Changes applied before test
commit bf431797f29ea447e82adb3f7c397840300d0e59
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jan 13 15:32:36 2021 +0100

    visit_stats: Keep a reference to the last snapshot
    
    Related to T2967

commit ab9871d3f14ccc4510ef0912af45716259630878
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jan 13 13:03:28 2021 +0100

    Populate origin_visit_stats table out of the origin_visit_status topic
    
    Related to T2967

Link to build: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/102/
See console output for more information: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/102/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 13 2021, 3:39 PM
Harbormaster failed remote builds in B18324: Diff 17180!
ardumont updated this revision to Diff 17187.Jan 13 2021, 3:45 PM

Fix blank line, drop spurious "new" term...

swh-public-ci added a comment.Jan 13 2021, 3:54 PM

Build has FAILED

Patch application report for D4853 (id=17187)

Could not rebase; Attempt merge onto a62003397d...

Updating a620033..1e05e71
Fast-forward
 sql/updates/20.sql                         |  4 ++
 swh/scheduler/backend.py                   | 10 ++-
 swh/scheduler/journal_client.py            | 47 +++++++++++++++
 swh/scheduler/model.py                     |  3 +
 swh/scheduler/sql/10-superuser-init.sql    |  1 +
 swh/scheduler/sql/30-schema.sql            |  4 +-
 swh/scheduler/tests/test_journal_client.py | 97 ++++++++++++++++++++++++++++++
 swh/scheduler/tests/test_scheduler.py      | 18 ++++++
 8 files changed, 181 insertions(+), 3 deletions(-)
 create mode 100644 sql/updates/20.sql
 create mode 100644 swh/scheduler/journal_client.py
 create mode 100644 swh/scheduler/tests/test_journal_client.py
Changes applied before test
commit 1e05e7154156a481f86d08c41e6986b6a3b5ef99
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jan 13 15:32:36 2021 +0100

    visit_stats: Keep a reference to the last snapshot
    
    Related to T2967

commit ab9871d3f14ccc4510ef0912af45716259630878
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jan 13 13:03:28 2021 +0100

    Populate origin_visit_stats table out of the origin_visit_status topic
    
    Related to T2967

Link to build: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/108/
See console output for more information: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/108/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 13 2021, 3:54 PM
Harbormaster failed remote builds in B18331: Diff 17187!
ardumont edited the test plan for this revision. (Show Details)Jan 13 2021, 6:43 PM
ardumont added a child revision: D4859: visit_stats: Update references when per event date is more recent.Jan 13 2021, 6:44 PM
ardumont updated this revision to Diff 17195.Jan 13 2021, 6:47 PM
  • Drop pgcrypto extension (added but unused), i planned to defined sha1git but did not so

not useful here.

  • rename 20.sql into 21.sql (i D4855 adding it already, let's not conflict ;)
swh-public-ci added a comment.Jan 13 2021, 6:48 PM

Build has FAILED

Patch application report for D4853 (id=17195)

Could not rebase; Attempt merge onto a62003397d...

Updating a620033..be20e09
Fast-forward
 sql/updates/21.sql                         |  4 ++
 swh/scheduler/backend.py                   | 10 ++-
 swh/scheduler/journal_client.py            | 47 +++++++++++++++
 swh/scheduler/model.py                     |  3 +
 swh/scheduler/sql/30-schema.sql            |  4 +-
 swh/scheduler/tests/test_journal_client.py | 97 ++++++++++++++++++++++++++++++
 swh/scheduler/tests/test_scheduler.py      | 18 ++++++
 7 files changed, 180 insertions(+), 3 deletions(-)
 create mode 100644 sql/updates/21.sql
 create mode 100644 swh/scheduler/journal_client.py
 create mode 100644 swh/scheduler/tests/test_journal_client.py
Changes applied before test
commit be20e09b877837302e1aeade259be29a93988798
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jan 13 15:32:36 2021 +0100

    visit_stats: Keep a reference to the last snapshot
    
    Related to T2967

commit ab9871d3f14ccc4510ef0912af45716259630878
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jan 13 13:03:28 2021 +0100

    Populate origin_visit_stats table out of the origin_visit_status topic
    
    Related to T2967

Link to build: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/112/
See console output for more information: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/112/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 13 2021, 6:48 PM
Harbormaster failed remote builds in B18339: Diff 17195!
ardumont edited the summary of this revision. (Show Details)Jan 14 2021, 11:17 AM
ardumont added a parent revision: D4852: Start journal client to populate the origin_visit_stats model.
swh-public-ci added a comment.Jan 14 2021, 5:08 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/123/
See console output for more information: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/123/console

ardumont abandoned this revision.Jan 14 2021, 5:48 PM

superseded by D4866

Revision Contents
Changeset List

PathSize
sql/
updates/
4 lines
swh/
scheduler/
10 lines
1 line
3 lines
sql/
4 lines
tests/
18 lines

Diff 17195

View Options

sql/updates/21.sql

Loading...
View Options

swh/scheduler/backend.py

Loading...
View Options

swh/scheduler/journal_client.py

Loading...
View Options

swh/scheduler/model.py

Loading...
View Options

swh/scheduler/sql/30-schema.sql

Loading...
View Options

swh/scheduler/tests/test_scheduler.py

Loading...
About · Developer information · Legal