Depends on D5805.
Details
Details
- Reviewers
aeviso - Group Reviewers
Reviewers - Commits
- rDPROVf3cd239bf8c3: Add a simple out-of-order dataset
Diff Detail
Diff Detail
- Repository
- rDPROV Provenance database
- Branch
- test
- Lint
Lint Skipped - Unit
Unit Tests Skipped - Build Status
Buildable 21762 Build 33837: Phabricator diff pipeline on jenkins Jenkins console · Jenkins Build 33836: arc lint + arc unit
Event Timeline
Comment Actions
Build is green
Patch application report for D5811 (id=20746)
Could not rebase; Attempt merge onto 5aa0314dd7...
Updating 5aa0314..4ba8588 Fast-forward requirements-test.txt | 1 + swh/provenance/model.py | 21 +- swh/provenance/postgresql/provenancedb_base.py | 12 +- .../postgresql/provenancedb_with_path.py | 115 +++------ swh/provenance/provenance.py | 201 ++++++++------- swh/provenance/tests/conftest.py | 26 +- swh/provenance/tests/data/README.md | 138 ++++++++++ swh/provenance/tests/data/cmdbts2.msgpack | Bin 0 -> 17734 bytes swh/provenance/tests/data/cmdbts2_repo.yaml | 80 ++++++ .../tests/data/generate_storage_from_git.py | 115 +++++++++ swh/provenance/tests/data/out-of-order.msgpack | Bin 0 -> 5254 bytes swh/provenance/tests/data/out-of-order_repo.yaml | 29 +++ .../tests/data/synthetic_cmdbts2_lower_1.txt | 91 +++++++ .../tests/data/synthetic_cmdbts2_lower_2.txt | 91 +++++++ .../tests/data/synthetic_cmdbts2_upper_1.txt | 91 +++++++ .../tests/data/synthetic_cmdbts2_upper_2.txt | 91 +++++++ swh/provenance/tests/data/synthetic_lower_1.txt | 91 ------- swh/provenance/tests/data/synthetic_lower_2.txt | 91 ------- .../tests/data/synthetic_out-of-order_lower_1.txt | 31 +++ swh/provenance/tests/data/synthetic_upper_1.txt | 92 ------- swh/provenance/tests/data/synthetic_upper_2.txt | 91 ------- swh/provenance/tests/test_provenance_db.py | 281 ++++++++++++--------- swh/provenance/tests/test_provenance_db_storage.py | 2 +- swh/provenance/tests/test_provenance_heuristics.py | 186 ++++++++++++++ 24 files changed, 1294 insertions(+), 673 deletions(-) create mode 100644 swh/provenance/tests/data/README.md create mode 100644 swh/provenance/tests/data/cmdbts2.msgpack create mode 100644 swh/provenance/tests/data/cmdbts2_repo.yaml create mode 100644 swh/provenance/tests/data/generate_storage_from_git.py create mode 100644 swh/provenance/tests/data/out-of-order.msgpack create mode 100644 swh/provenance/tests/data/out-of-order_repo.yaml create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_lower_1.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_lower_2.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_upper_1.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_upper_2.txt delete mode 100644 swh/provenance/tests/data/synthetic_lower_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_lower_2.txt create mode 100644 swh/provenance/tests/data/synthetic_out-of-order_lower_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_upper_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_upper_2.txt create mode 100644 swh/provenance/tests/test_provenance_heuristics.py
Changes applied before test
commit 4ba858811022b0865f379c04967a25b594b9bdd1
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Jun 1 16:34:57 2021 +0200
Add a simple out-of-order dataset
commit e3d6e0b3c4708d31e9f26c0dc8b415f00bb219aa
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Jun 1 11:47:09 2021 +0200
Remove test_provenance_heuristics_CMDBTS test
since it's redundant with the cmdbts2 test, now generated from a simple
yaml file rather than depending on the original CMDBTS git repo on
github.
The CMDBTS dataset (CMDBTS.msgpack) is kept for now since it's still
used for other tests (e.g. test_provenance_db).
commit f55368813234fd693ad3bb98ce0ed83e53e0ce22
Author: David Douard <david.douard@sdfa3.org>
Date: Mon May 31 16:51:01 2021 +0200
Add a new (git) dataset generation scaffolding for tests
and use it to the generate a 'cmdbts2' test case strictly equivalent
to the CMDBTS repo.
See the swh/provenance/tests/data/README.md file for more details.
Note: this aims at making easy to write more test cases than depending
on the CMDBTS git repo on github. For example, a new test case should
come soon for situations like 'out-of-order' revisions.
commit 19f8bd8f5d7476339d8e7eabd0ba2a8aa800251f
Author: David Douard <david.douard@sdfa3.org>
Date: Mon May 31 16:45:48 2021 +0200
Remove test_provenance_heuristics from tests from ArchvieStorage tests
because it's not that meaningful.
commit 08d8dd6478836ff4ab1c00c67f553b6d705b5a9c
Author: David Douard <david.douard@sdfa3.org>
Date: Tue May 25 14:31:51 2021 +0200
Simplify DB queries in ProvenanceWithPathDB.content_find_(first|all)
the queries should be exactly the same as before (query plans are the
same); just written (hopefully) in a bit more readable manner.
commit fd43523fd594e70ccd002827d379321f52c2b6da
Author: David Douard <david.douard@sdfa3.org>
Date: Tue May 25 14:59:16 2021 +0200
Add a test for content_find_all()
commit d85f2b0ee48aefe03ad32311623e5390f43d7261
Author: David Douard <david.douard@sdfa3.org>
Date: Wed May 12 09:47:03 2021 +0200
Refactor the isochrone graph computation
attempt to simplify a bit this part of the code:
- IsochroneNode are now only used for directories
- FileEntry are stored in a new IsochroneNode.files attribute, so
- IsochroneNode.children only stores IsochroneNode (thus DirectoryEntry)
objects,
- rename IsochroneNode.date as 'dbdate' and clarify its semantics
commit 31d833ec86bf041e100795e7796ce832d00450ef
Author: David Douard <david.douard@sdfa3.org>
Date: Wed May 19 16:16:53 2021 +0200
Add 'ls_files()' and 'ls_dirs()' methods to the DirectoryEntry class
to make it a bit easier to compute the isochrone graph (see following
revisions).
commit 72644b98a218132c0b173f360c503438688ecebb
Author: David Douard <david.douard@sdfa3.org>
Date: Wed May 19 16:14:41 2021 +0200
Add __str__ methods to RevisionEntry, DirectoryEntry and FileEntry
to ease logging and debugging.
commit a71041fbaf3f0d7ec3ea944cbbf04286c57d8b7e
Author: David Douard <david.douard@sdfa3.org>
Date: Wed May 12 12:44:07 2021 +0200
Improve a bit the code of ProvenanceDBBase
commit defcb388ffba0869edb1a126b6626710c396c2ac
Author: David Douard <david.douard@sdfa3.org>
Date: Wed May 12 10:30:23 2021 +0200
Add a test for the build_isochrone_graph() function
this test is far from ideal, since it's mostly the record of what happen
during a "known good" session of revision insertions, but at least it
should allow to refactor code related to the isochrone graph computation
with a bit more confidence...See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/47/ for more details.
Comment Actions
@aeviso I have a question: in this scenario, do we expect the date in the directory table for A/C and A/B/C (sha1 c9cabe7f49012e3fdef6ac6b929efb5654f583cf) directories to be invalidated or updated due to the arrival, at rev R05, of a version of the b file dated earlier than originally seen in R01?
edit: I'm asking because it's unclear to me how the remove_cache is supposed to work with regard to the insert_all() method. When this situation occurs, the directory_invalidate_in_isochrone_frontier() is called, but this will not impact what will happen at db insertion time. It will only have impact on how how directory_get_date(s)_in_isochrone_frontier() behaves. So if I'm not mistaken, the behavior of all this will depend on whether several revisions are inserted at once (in a single transaction) or not, for example.
Comment Actions
Build is green
Patch application report for D5811 (id=20756)
Could not rebase; Attempt merge onto 49e47c3ea7...
Removing swh/provenance/tests/data/synthetic_upper_2.txt Removing swh/provenance/tests/data/synthetic_upper_1.txt Removing swh/provenance/tests/data/synthetic_lower_2.txt Removing swh/provenance/tests/data/synthetic_lower_1.txt Merge made by the 'recursive' strategy. requirements-test.txt | 1 + swh/provenance/model.py | 31 ++- swh/provenance/postgresql/provenancedb_base.py | 12 +- .../postgresql/provenancedb_with_path.py | 115 +++------ swh/provenance/provenance.py | 220 +++++++++------- swh/provenance/tests/conftest.py | 26 +- swh/provenance/tests/data/README.md | 138 ++++++++++ swh/provenance/tests/data/cmdbts2.msgpack | Bin 0 -> 17734 bytes swh/provenance/tests/data/cmdbts2_repo.yaml | 80 ++++++ .../tests/data/generate_storage_from_git.py | 115 +++++++++ swh/provenance/tests/data/out-of-order.msgpack | Bin 0 -> 5254 bytes swh/provenance/tests/data/out-of-order_repo.yaml | 29 +++ .../tests/data/synthetic_cmdbts2_lower_1.txt | 91 +++++++ .../tests/data/synthetic_cmdbts2_lower_2.txt | 91 +++++++ .../tests/data/synthetic_cmdbts2_upper_1.txt | 91 +++++++ .../tests/data/synthetic_cmdbts2_upper_2.txt | 91 +++++++ swh/provenance/tests/data/synthetic_lower_1.txt | 91 ------- swh/provenance/tests/data/synthetic_lower_2.txt | 91 ------- .../tests/data/synthetic_out-of-order_lower_1.txt | 31 +++ swh/provenance/tests/data/synthetic_upper_1.txt | 92 ------- swh/provenance/tests/data/synthetic_upper_2.txt | 91 ------- swh/provenance/tests/test_provenance_db.py | 281 ++++++++++++--------- swh/provenance/tests/test_provenance_db_storage.py | 2 +- swh/provenance/tests/test_provenance_heuristics.py | 186 ++++++++++++++ 24 files changed, 1315 insertions(+), 681 deletions(-) create mode 100644 swh/provenance/tests/data/README.md create mode 100644 swh/provenance/tests/data/cmdbts2.msgpack create mode 100644 swh/provenance/tests/data/cmdbts2_repo.yaml create mode 100644 swh/provenance/tests/data/generate_storage_from_git.py create mode 100644 swh/provenance/tests/data/out-of-order.msgpack create mode 100644 swh/provenance/tests/data/out-of-order_repo.yaml create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_lower_1.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_lower_2.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_upper_1.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_upper_2.txt delete mode 100644 swh/provenance/tests/data/synthetic_lower_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_lower_2.txt create mode 100644 swh/provenance/tests/data/synthetic_out-of-order_lower_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_upper_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_upper_2.txt create mode 100644 swh/provenance/tests/test_provenance_heuristics.py
Changes applied before test
commit aef5b9fcea25d09476fdfca661f5624a463b8fc6
Merge: 49e47c3 21d48d1
Author: Jenkins user <jenkins@localhost>
Date: Wed Jun 2 15:41:18 2021 +0000
Merge branch 'diff-target' into HEAD
commit 21d48d1410d720d7e606b7ddc452aa019cd86c26
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Jun 1 16:34:57 2021 +0200
Add a simple out-of-order dataset
commit 421b2c832a9d37ee6de8b29eebf7c1f65ed01d5a
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Jun 1 11:47:09 2021 +0200
Remove test_provenance_heuristics_CMDBTS test
since it's redundant with the cmdbts2 test, now generated from a simple
yaml file rather than depending on the original CMDBTS git repo on
github.
The CMDBTS dataset (CMDBTS.msgpack) is kept for now since it's still
used for other tests (e.g. test_provenance_db).
commit 10a166272c262f7725d041fc0b15219868eedacb
Author: David Douard <david.douard@sdfa3.org>
Date: Mon May 31 16:51:01 2021 +0200
Add a new (git) dataset generation scaffolding for tests
and use it to the generate a 'cmdbts2' test case strictly equivalent
to the CMDBTS repo.
See the swh/provenance/tests/data/README.md file for more details.
Note: this aims at making easy to write more test cases than depending
on the CMDBTS git repo on github. For example, a new test case should
come soon for situations like 'out-of-order' revisions.
commit ddc2ae0583db4b317c04d97386d18d2c17ae00d7
Author: David Douard <david.douard@sdfa3.org>
Date: Mon May 31 16:45:48 2021 +0200
Remove test_provenance_heuristics from tests from ArchvieStorage tests
because it's not that meaningful.
commit 16bab3c60c2a3a80782273f1aaff796826e7dc2c
Author: David Douard <david.douard@sdfa3.org>
Date: Tue May 25 14:31:51 2021 +0200
Simplify DB queries in ProvenanceWithPathDB.content_find_(first|all)
the queries should be exactly the same as before (query plans are the
same); just written (hopefully) in a bit more readable manner.
commit 024cc9ce93e545782a980f8e81d5d09651b8231b
Author: David Douard <david.douard@sdfa3.org>
Date: Tue May 25 14:59:16 2021 +0200
Add a test for content_find_all()
commit af15ad65f4a34e7703bfec80666102a6403cb505
Author: David Douard <david.douard@sdfa3.org>
Date: Wed May 12 09:47:03 2021 +0200
Refactor the isochrone graph computation
attempt to simplify a bit this part of the code:
- IsochroneNode are now only used for directories
- FileEntry are used directly from IsochroneNode.entry.files (no need
for creating new FileEntry instances), so
- IsochroneNode.children only stores IsochroneNode (thus DirectoryEntry)
objects,
- rename IsochroneNode.date as 'dbdate' and clarify its semantics,
- attempt to document (comments) a bit more the algorithm and semantics
of several attributes/variables used in there.
commit 1f49fdc967a2854d3a68dec34886b824fdf045f6
Author: David Douard <david.douard@sdfa3.org>
Date: Wed May 19 16:16:53 2021 +0200
Replace 'DirectoryEntry.ls()' method by 'files' and 'dirs' properties
and make the retrieval of children from the archive explicit in a
dedicated retrieve_children() method.
commit 72644b98a218132c0b173f360c503438688ecebb
Author: David Douard <david.douard@sdfa3.org>
Date: Wed May 19 16:14:41 2021 +0200
Add __str__ methods to RevisionEntry, DirectoryEntry and FileEntry
to ease logging and debugging.
commit a71041fbaf3f0d7ec3ea944cbbf04286c57d8b7e
Author: David Douard <david.douard@sdfa3.org>
Date: Wed May 12 12:44:07 2021 +0200
Improve a bit the code of ProvenanceDBBase
commit defcb388ffba0869edb1a126b6626710c396c2ac
Author: David Douard <david.douard@sdfa3.org>
Date: Wed May 12 10:30:23 2021 +0200
Add a test for the build_isochrone_graph() function
this test is far from ideal, since it's mostly the record of what happen
during a "known good" session of revision insertions, but at least it
should allow to refactor code related to the isochrone graph computation
with a bit more confidence...See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/55/ for more details.
Comment Actions
Build is green
Patch application report for D5811 (id=20769)
Could not rebase; Attempt merge onto 49e47c3ea7...
Updating 49e47c3..296229e Fast-forward requirements-test.txt | 1 + swh/provenance/model.py | 31 ++- swh/provenance/postgresql/provenancedb_base.py | 12 +- .../postgresql/provenancedb_with_path.py | 115 +++------ swh/provenance/provenance.py | 220 +++++++++------- swh/provenance/tests/conftest.py | 26 +- swh/provenance/tests/data/README.md | 138 ++++++++++ swh/provenance/tests/data/cmdbts2.msgpack | Bin 0 -> 17734 bytes swh/provenance/tests/data/cmdbts2_repo.yaml | 80 ++++++ .../tests/data/generate_storage_from_git.py | 115 +++++++++ swh/provenance/tests/data/out-of-order.msgpack | Bin 0 -> 5254 bytes swh/provenance/tests/data/out-of-order_repo.yaml | 29 +++ .../tests/data/synthetic_cmdbts2_lower_1.txt | 91 +++++++ .../tests/data/synthetic_cmdbts2_lower_2.txt | 91 +++++++ .../tests/data/synthetic_cmdbts2_upper_1.txt | 91 +++++++ .../tests/data/synthetic_cmdbts2_upper_2.txt | 91 +++++++ swh/provenance/tests/data/synthetic_lower_1.txt | 91 ------- swh/provenance/tests/data/synthetic_lower_2.txt | 91 ------- .../tests/data/synthetic_out-of-order_lower_1.txt | 31 +++ swh/provenance/tests/data/synthetic_upper_1.txt | 92 ------- swh/provenance/tests/data/synthetic_upper_2.txt | 91 ------- swh/provenance/tests/test_provenance_db.py | 281 ++++++++++++--------- swh/provenance/tests/test_provenance_db_storage.py | 2 +- swh/provenance/tests/test_provenance_heuristics.py | 186 ++++++++++++++ 24 files changed, 1315 insertions(+), 681 deletions(-) create mode 100644 swh/provenance/tests/data/README.md create mode 100644 swh/provenance/tests/data/cmdbts2.msgpack create mode 100644 swh/provenance/tests/data/cmdbts2_repo.yaml create mode 100644 swh/provenance/tests/data/generate_storage_from_git.py create mode 100644 swh/provenance/tests/data/out-of-order.msgpack create mode 100644 swh/provenance/tests/data/out-of-order_repo.yaml create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_lower_1.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_lower_2.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_upper_1.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_upper_2.txt delete mode 100644 swh/provenance/tests/data/synthetic_lower_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_lower_2.txt create mode 100644 swh/provenance/tests/data/synthetic_out-of-order_lower_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_upper_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_upper_2.txt create mode 100644 swh/provenance/tests/test_provenance_heuristics.py
Changes applied before test
commit 296229e3ffddb05c516a4208c25a6155f24314b4
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Jun 1 16:34:57 2021 +0200
Add a simple out-of-order dataset
commit 46c7c9df7beaae63f4dc1089498c64c5658d3bf5
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Jun 1 11:47:09 2021 +0200
Remove test_provenance_heuristics_CMDBTS test
since it's redundant with the cmdbts2 test, now generated from a simple
yaml file rather than depending on the original CMDBTS git repo on
github.
The CMDBTS dataset (CMDBTS.msgpack) is kept for now since it's still
used for other tests (e.g. test_provenance_db).
commit b3279560fea0c3a84002516cd25d3c3ce86491c6
Author: David Douard <david.douard@sdfa3.org>
Date: Mon May 31 16:51:01 2021 +0200
Add a new (git) dataset generation scaffolding for tests
and use it to the generate a 'cmdbts2' test case strictly equivalent
to the CMDBTS repo.
See the swh/provenance/tests/data/README.md file for more details.
Note: this aims at making easy to write more test cases than depending
on the CMDBTS git repo on github. For example, a new test case should
come soon for situations like 'out-of-order' revisions.
commit 56f0ae8e12990006b1faec62bb8f61b9eed84955
Author: David Douard <david.douard@sdfa3.org>
Date: Mon May 31 16:45:48 2021 +0200
Remove test_provenance_heuristics from tests from ArchvieStorage tests
because it's not that meaningful.
commit de30f332f219e4edb299bb50a0b808a779c57d85
Author: David Douard <david.douard@sdfa3.org>
Date: Tue May 25 14:31:51 2021 +0200
Simplify DB queries in ProvenanceWithPathDB.content_find_(first|all)
the queries should be exactly the same as before (query plans are the
same); just written (hopefully) in a bit more readable manner.
commit ee8e4b0b7ce6a85eac0665a916a37b2d63e3bb4d
Author: David Douard <david.douard@sdfa3.org>
Date: Tue May 25 14:59:16 2021 +0200
Add a test for content_find_all()
commit 94598b3ce8c49eb6dfe5308b47b74271a7f9d625
Author: David Douard <david.douard@sdfa3.org>
Date: Wed May 12 09:47:03 2021 +0200
Refactor the isochrone graph computation
attempt to simplify a bit this part of the code:
- IsochroneNode are now only used for directories
- FileEntry are used directly from IsochroneNode.entry.files (no need
for creating new FileEntry instances), so
- IsochroneNode.children only stores IsochroneNode (thus DirectoryEntry)
objects,
- rename IsochroneNode.date as 'dbdate' and clarify its semantics,
- attempt to document (comments) a bit more the algorithm and semantics
of several attributes/variables used in there.
commit 9d110b93e9c39d65bf2986b148c4bf3467b0efa3
Author: David Douard <david.douard@sdfa3.org>
Date: Wed May 19 16:16:53 2021 +0200
Replace 'DirectoryEntry.ls()' method by 'files' and 'dirs' properties
and make the retrieval of children from the archive explicit in a
dedicated retrieve_children() method.
commit fcfbb250e688a4ade6849522714832ec49238a8d
Author: David Douard <david.douard@sdfa3.org>
Date: Wed May 19 16:14:41 2021 +0200
Add __str__ methods to RevisionEntry, DirectoryEntry and FileEntry
to ease logging and debugging.
commit 1f823ac01491ee0f27eac685d32322f8558c26bc
Author: David Douard <david.douard@sdfa3.org>
Date: Wed May 12 12:44:07 2021 +0200
Improve a bit the code of ProvenanceDBBase
commit cb623cb0e7dd9a2a568b6d2645e89c4d86ba0a66
Author: David Douard <david.douard@sdfa3.org>
Date: Wed May 12 10:30:23 2021 +0200
Add a test for the build_isochrone_graph() function
this test is far from ideal, since it's mostly the record of what happen
during a "known good" session of revision insertions, but at least it
should allow to refactor code related to the isochrone graph computation
with a bit more confidence...See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/63/ for more details.
Comment Actions
For example, imagine I extend this repo a little bit by adding to out-of-order_repo.yaml file the following content:
- msg: R06
date: 1000000050
content:
A/B/C/a: "content a"
A/B/C/b: "content b"
A/B/c: "content c"what is the expected synthetic representation for this?
My first shot would have been:
1000000050.0 53519b5a5e8cf12a4f81f82e489f95c1d04d5314 R06
R06 | | | R 53519b5a5e8cf12a4f81f82e489f95c1d04d5314 | 1000000050.0
| R---C | A/B/c | C fa08654474ae2ddc4f61ee3a43d017ba65a439c3 | 0
| R-D | A/B/C | D c9cabe7f49012e3fdef6ac6b929efb5654f583cf | -45
| | + a | C 20329687bb9c1231a7e05afe86160343ad49b494 | -50
| | + b | C 50e9cdb03f9719261dd39d7f2920b906db3711a3 | -45But this fails the tests because the R-D added in this section does not take into consideration, with current implementation, the fact the b file is now timestamped at 1000000005 (due to revision R05), so the timestamp for the directory c9cabe7f49012e3fdef6ac6b929efb5654f583cf remains 1000000010 .
Comment Actions
Build has FAILED
Patch application report for D5811 (id=20788)
Could not rebase; Attempt merge onto 08344d3f76...
Updating 08344d3..71d39de Fast-forward requirements-test.txt | 1 + swh/provenance/tests/conftest.py | 26 +-- swh/provenance/tests/data/README.md | 138 +++++++++++++++ swh/provenance/tests/data/cmdbts2.msgpack | Bin 0 -> 17734 bytes swh/provenance/tests/data/cmdbts2_repo.yaml | 80 +++++++++ .../tests/data/generate_storage_from_git.py | 115 +++++++++++++ swh/provenance/tests/data/out-of-order.msgpack | Bin 0 -> 6653 bytes swh/provenance/tests/data/out-of-order_repo.yaml | 35 ++++ .../tests/data/synthetic_cmdbts2_lower_1.txt | 91 ++++++++++ .../tests/data/synthetic_cmdbts2_lower_2.txt | 91 ++++++++++ .../tests/data/synthetic_cmdbts2_upper_1.txt | 91 ++++++++++ .../tests/data/synthetic_cmdbts2_upper_2.txt | 91 ++++++++++ swh/provenance/tests/data/synthetic_lower_1.txt | 91 ---------- swh/provenance/tests/data/synthetic_lower_2.txt | 91 ---------- .../tests/data/synthetic_out-of-order_lower_1.txt | 38 +++++ swh/provenance/tests/data/synthetic_upper_1.txt | 92 ---------- swh/provenance/tests/data/synthetic_upper_2.txt | 91 ---------- swh/provenance/tests/test_provenance_db.py | 132 --------------- swh/provenance/tests/test_provenance_db_storage.py | 2 +- swh/provenance/tests/test_provenance_heuristics.py | 186 +++++++++++++++++++++ 20 files changed, 974 insertions(+), 508 deletions(-) create mode 100644 swh/provenance/tests/data/README.md create mode 100644 swh/provenance/tests/data/cmdbts2.msgpack create mode 100644 swh/provenance/tests/data/cmdbts2_repo.yaml create mode 100644 swh/provenance/tests/data/generate_storage_from_git.py create mode 100644 swh/provenance/tests/data/out-of-order.msgpack create mode 100644 swh/provenance/tests/data/out-of-order_repo.yaml create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_lower_1.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_lower_2.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_upper_1.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_upper_2.txt delete mode 100644 swh/provenance/tests/data/synthetic_lower_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_lower_2.txt create mode 100644 swh/provenance/tests/data/synthetic_out-of-order_lower_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_upper_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_upper_2.txt create mode 100644 swh/provenance/tests/test_provenance_heuristics.py
Changes applied before test
commit 71d39de612ef5b156887dfca9bf491649e17bdde
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Jun 1 16:34:57 2021 +0200
Add a simple out-of-order dataset
commit 9854a75c8f5426836c561bd9c1b9bad7c85494e0
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Jun 1 11:47:09 2021 +0200
Remove test_provenance_heuristics_CMDBTS test
since it's redundant with the cmdbts2 test, now generated from a simple
yaml file rather than depending on the original CMDBTS git repo on
github.
The CMDBTS dataset (CMDBTS.msgpack) is kept for now since it's still
used for other tests (e.g. test_provenance_db).
commit 4f7b0eadd10c55318f64688abfe391ead4bcc3af
Author: David Douard <david.douard@sdfa3.org>
Date: Mon May 31 16:51:01 2021 +0200
Add a new (git) dataset generation scaffolding for tests
and use it to the generate a 'cmdbts2' test case strictly equivalent
to the CMDBTS repo.
See the swh/provenance/tests/data/README.md file for more details.
Note: this aims at making easy to write more test cases than depending
on the CMDBTS git repo on github. For example, a new test case should
come soon for situations like 'out-of-order' revisions.
commit fd373add1762c515070d39dc1cc1b58c09d3e8e4
Author: David Douard <david.douard@sdfa3.org>
Date: Mon May 31 16:45:48 2021 +0200
Remove test_provenance_heuristics from tests from ArchvieStorage tests
because it's not that meaningful.Link to build: https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/70/
See console output for more information: https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/70/console
Comment Actions
Fix the synth file...
but note the discrepency in the timestamps of R06, the ts of the D-C frontier (directory c9cabe7f49012e3fdef6ac6b929efb5654f583cf), -40, is NOT the maxdate of its content (the b file has now a -45 delta), due to the fact the timestamp of a known frontier is not updated when its invalidated by an out-of-order revision being processed.
Comment Actions
Build is green
Patch application report for D5811 (id=20795)
Could not rebase; Attempt merge onto 08344d3f76...
Updating 08344d3..242726f Fast-forward requirements-test.txt | 1 + swh/provenance/tests/conftest.py | 26 +-- swh/provenance/tests/data/README.md | 138 +++++++++++++++ swh/provenance/tests/data/cmdbts2.msgpack | Bin 0 -> 17734 bytes swh/provenance/tests/data/cmdbts2_repo.yaml | 80 +++++++++ .../tests/data/generate_storage_from_git.py | 115 +++++++++++++ swh/provenance/tests/data/out-of-order.msgpack | Bin 0 -> 6653 bytes swh/provenance/tests/data/out-of-order_repo.yaml | 35 ++++ .../tests/data/synthetic_cmdbts2_lower_1.txt | 91 ++++++++++ .../tests/data/synthetic_cmdbts2_lower_2.txt | 91 ++++++++++ .../tests/data/synthetic_cmdbts2_upper_1.txt | 91 ++++++++++ .../tests/data/synthetic_cmdbts2_upper_2.txt | 91 ++++++++++ swh/provenance/tests/data/synthetic_lower_1.txt | 91 ---------- swh/provenance/tests/data/synthetic_lower_2.txt | 91 ---------- .../tests/data/synthetic_out-of-order_lower_1.txt | 38 +++++ swh/provenance/tests/data/synthetic_upper_1.txt | 92 ---------- swh/provenance/tests/data/synthetic_upper_2.txt | 91 ---------- swh/provenance/tests/test_provenance_db.py | 132 --------------- swh/provenance/tests/test_provenance_db_storage.py | 2 +- swh/provenance/tests/test_provenance_heuristics.py | 186 +++++++++++++++++++++ 20 files changed, 974 insertions(+), 508 deletions(-) create mode 100644 swh/provenance/tests/data/README.md create mode 100644 swh/provenance/tests/data/cmdbts2.msgpack create mode 100644 swh/provenance/tests/data/cmdbts2_repo.yaml create mode 100644 swh/provenance/tests/data/generate_storage_from_git.py create mode 100644 swh/provenance/tests/data/out-of-order.msgpack create mode 100644 swh/provenance/tests/data/out-of-order_repo.yaml create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_lower_1.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_lower_2.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_upper_1.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_upper_2.txt delete mode 100644 swh/provenance/tests/data/synthetic_lower_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_lower_2.txt create mode 100644 swh/provenance/tests/data/synthetic_out-of-order_lower_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_upper_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_upper_2.txt create mode 100644 swh/provenance/tests/test_provenance_heuristics.py
Changes applied before test
commit 242726f6980b6c98c7cd9942fd0b1e1ee21e034f
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Jun 1 16:34:57 2021 +0200
Add a simple out-of-order dataset
commit 9854a75c8f5426836c561bd9c1b9bad7c85494e0
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Jun 1 11:47:09 2021 +0200
Remove test_provenance_heuristics_CMDBTS test
since it's redundant with the cmdbts2 test, now generated from a simple
yaml file rather than depending on the original CMDBTS git repo on
github.
The CMDBTS dataset (CMDBTS.msgpack) is kept for now since it's still
used for other tests (e.g. test_provenance_db).
commit 4f7b0eadd10c55318f64688abfe391ead4bcc3af
Author: David Douard <david.douard@sdfa3.org>
Date: Mon May 31 16:51:01 2021 +0200
Add a new (git) dataset generation scaffolding for tests
and use it to the generate a 'cmdbts2' test case strictly equivalent
to the CMDBTS repo.
See the swh/provenance/tests/data/README.md file for more details.
Note: this aims at making easy to write more test cases than depending
on the CMDBTS git repo on github. For example, a new test case should
come soon for situations like 'out-of-order' revisions.
commit fd373add1762c515070d39dc1cc1b58c09d3e8e4
Author: David Douard <david.douard@sdfa3.org>
Date: Mon May 31 16:45:48 2021 +0200
Remove test_provenance_heuristics from tests from ArchvieStorage tests
because it's not that meaningful.See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/73/ for more details.
Comment Actions
Build is green
Patch application report for D5811 (id=20803)
Could not rebase; Attempt merge onto 08344d3f76...
Updating 08344d3..7092416 Fast-forward requirements-test.txt | 1 + swh/provenance/tests/conftest.py | 28 ++-- swh/provenance/tests/data/README.md | 138 +++++++++++++++ swh/provenance/tests/data/cmdbts2.msgpack | Bin 0 -> 17734 bytes swh/provenance/tests/data/cmdbts2_repo.yaml | 80 +++++++++ .../tests/data/generate_storage_from_git.py | 115 +++++++++++++ swh/provenance/tests/data/out-of-order.msgpack | Bin 0 -> 6653 bytes swh/provenance/tests/data/out-of-order_repo.yaml | 35 ++++ .../tests/data/synthetic_cmdbts2_lower_1.txt | 91 ++++++++++ .../tests/data/synthetic_cmdbts2_lower_2.txt | 91 ++++++++++ .../tests/data/synthetic_cmdbts2_upper_1.txt | 91 ++++++++++ .../tests/data/synthetic_cmdbts2_upper_2.txt | 91 ++++++++++ swh/provenance/tests/data/synthetic_lower_1.txt | 91 ---------- swh/provenance/tests/data/synthetic_lower_2.txt | 91 ---------- .../tests/data/synthetic_out-of-order_lower_1.txt | 42 +++++ swh/provenance/tests/data/synthetic_upper_1.txt | 92 ---------- swh/provenance/tests/data/synthetic_upper_2.txt | 91 ---------- swh/provenance/tests/test_provenance_db.py | 132 --------------- swh/provenance/tests/test_provenance_db_storage.py | 2 +- swh/provenance/tests/test_provenance_heuristics.py | 186 +++++++++++++++++++++ 20 files changed, 979 insertions(+), 509 deletions(-) create mode 100644 swh/provenance/tests/data/README.md create mode 100644 swh/provenance/tests/data/cmdbts2.msgpack create mode 100644 swh/provenance/tests/data/cmdbts2_repo.yaml create mode 100644 swh/provenance/tests/data/generate_storage_from_git.py create mode 100644 swh/provenance/tests/data/out-of-order.msgpack create mode 100644 swh/provenance/tests/data/out-of-order_repo.yaml create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_lower_1.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_lower_2.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_upper_1.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_upper_2.txt delete mode 100644 swh/provenance/tests/data/synthetic_lower_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_lower_2.txt create mode 100644 swh/provenance/tests/data/synthetic_out-of-order_lower_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_upper_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_upper_2.txt create mode 100644 swh/provenance/tests/test_provenance_heuristics.py
Changes applied before test
commit 70924164c5b251f6e8b3e23f691bd77d723b843e
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Jun 1 16:34:57 2021 +0200
Add a simple out-of-order dataset
commit 19ef0ba9f5c86d26b595aaa5dc64390994551b64
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Jun 1 11:47:09 2021 +0200
Remove test_provenance_heuristics_CMDBTS test
since it's redundant with the cmdbts2 test, now generated from a simple
yaml file rather than depending on the original CMDBTS git repo on
github.
The CMDBTS dataset (CMDBTS.msgpack) is kept for now since it's still
used for other tests (e.g. test_provenance_db).
commit f534e558645ecc9384dfb1781e94266feac683f1
Author: David Douard <david.douard@sdfa3.org>
Date: Mon May 31 16:51:01 2021 +0200
Add a new (git) dataset generation scaffolding for tests
and use it to the generate a 'cmdbts2' test case strictly equivalent
to the CMDBTS repo.
See the swh/provenance/tests/data/README.md file for more details.
Note: this aims at making easy to write more test cases than depending
on the CMDBTS git repo on github. For example, a new test case should
come soon for situations like 'out-of-order' revisions.
commit fd373add1762c515070d39dc1cc1b58c09d3e8e4
Author: David Douard <david.douard@sdfa3.org>
Date: Mon May 31 16:45:48 2021 +0200
Remove test_provenance_heuristics from tests from ArchvieStorage tests
because it's not that meaningful.See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/78/ for more details.
Comment Actions
Build is green
Patch application report for D5811 (id=20807)
Could not rebase; Attempt merge onto 08344d3f76...
Updating 08344d3..f3cd239 Fast-forward requirements-test.txt | 1 + swh/provenance/tests/conftest.py | 28 ++-- swh/provenance/tests/data/README.md | 138 +++++++++++++++ swh/provenance/tests/data/cmdbts2.msgpack | Bin 0 -> 17734 bytes swh/provenance/tests/data/cmdbts2_repo.yaml | 80 +++++++++ .../tests/data/generate_storage_from_git.py | 115 +++++++++++++ swh/provenance/tests/data/out-of-order.msgpack | Bin 0 -> 6653 bytes swh/provenance/tests/data/out-of-order_repo.yaml | 35 ++++ .../tests/data/synthetic_cmdbts2_lower_1.txt | 91 ++++++++++ .../tests/data/synthetic_cmdbts2_lower_2.txt | 91 ++++++++++ .../tests/data/synthetic_cmdbts2_upper_1.txt | 91 ++++++++++ .../tests/data/synthetic_cmdbts2_upper_2.txt | 91 ++++++++++ swh/provenance/tests/data/synthetic_lower_1.txt | 91 ---------- swh/provenance/tests/data/synthetic_lower_2.txt | 91 ---------- .../tests/data/synthetic_out-of-order_lower_1.txt | 42 +++++ swh/provenance/tests/data/synthetic_upper_1.txt | 92 ---------- swh/provenance/tests/data/synthetic_upper_2.txt | 91 ---------- swh/provenance/tests/test_provenance_db.py | 132 --------------- swh/provenance/tests/test_provenance_db_storage.py | 2 +- swh/provenance/tests/test_provenance_heuristics.py | 186 +++++++++++++++++++++ 20 files changed, 979 insertions(+), 509 deletions(-) create mode 100644 swh/provenance/tests/data/README.md create mode 100644 swh/provenance/tests/data/cmdbts2.msgpack create mode 100644 swh/provenance/tests/data/cmdbts2_repo.yaml create mode 100644 swh/provenance/tests/data/generate_storage_from_git.py create mode 100644 swh/provenance/tests/data/out-of-order.msgpack create mode 100644 swh/provenance/tests/data/out-of-order_repo.yaml create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_lower_1.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_lower_2.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_upper_1.txt create mode 100644 swh/provenance/tests/data/synthetic_cmdbts2_upper_2.txt delete mode 100644 swh/provenance/tests/data/synthetic_lower_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_lower_2.txt create mode 100644 swh/provenance/tests/data/synthetic_out-of-order_lower_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_upper_1.txt delete mode 100644 swh/provenance/tests/data/synthetic_upper_2.txt create mode 100644 swh/provenance/tests/test_provenance_heuristics.py
Changes applied before test
commit f3cd239bf8c3241b297c6481beca266bfd47eb25
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Jun 1 16:34:57 2021 +0200
Add a simple out-of-order dataset
commit 5f9e5b53a5b0547cfdbe1676e2b09648f4f359f1
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Jun 1 11:47:09 2021 +0200
Remove test_provenance_heuristics_CMDBTS test
since it's redundant with the cmdbts2 test, now generated from a simple
yaml file rather than depending on the original CMDBTS git repo on
github.
The CMDBTS dataset (CMDBTS.msgpack) is kept for now since it's still
used for other tests (e.g. test_provenance_db).
commit 9fe096b3905495bff534649f2e5e0ecb8802217d
Author: David Douard <david.douard@sdfa3.org>
Date: Mon May 31 16:51:01 2021 +0200
Add a new (git) dataset generation scaffolding for tests
and use it to the generate a 'cmdbts2' test case strictly equivalent
to the CMDBTS repo.
See the swh/provenance/tests/data/README.md file for more details.
Note: this aims at making easy to write more test cases than depending
on the CMDBTS git repo on github. For example, a new test case should
come soon for situations like 'out-of-order' revisions.
commit fd373add1762c515070d39dc1cc1b58c09d3e8e4
Author: David Douard <david.douard@sdfa3.org>
Date: Mon May 31 16:45:48 2021 +0200
Remove test_provenance_heuristics from tests from ArchvieStorage tests
because it's not that meaningful.See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/82/ for more details.