In memory kafkfa writer lacks tests.
I'm adding two new tests that ensure the proper handling of privileged objects in case of in memory implementation.
These tests also cover the scenario mentioned in T2823#63777
Related T2823
Details
Details
- Reviewers
vlorentz - Group Reviewers
Reviewers - Commits
- rDJNL2972c7a694aa: tests/inmemory: Ensure privileged objects are stored properly
Diff Detail
Diff Detail
- Repository
- rDJNL Journal infrastructure
- Branch
- master
- Lint
No Linters Available - Unit
No Unit Test Coverage - Build Status
Buildable 21206 Build 32921: Phabricator diff pipeline on jenkins Jenkins console · Jenkins Build 32920: arc lint + arc unit
Event Timeline
Comment Actions
Build is green
Patch application report for D5618 (id=20030)
Rebasing onto 236a00262e...
Current branch diff-target is up to date.
Changes applied before test
commit 3dd4b0d29548db9977aa203061c7ffe8015ea9d3
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date: Tue Apr 27 01:16:28 2021 +0530
tests/inmemory: Ensure privileged objects are stored properlySee https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/158/ for more details.
Comment Actions
Compare the expected values directly:
assert sorted(expected) == sorted(writer.objects) assert writer.privileged_objects == []
Comment Actions
Build is green
Patch application report for D5618 (id=20143)
Rebasing onto dd59b67c7f...
First, rewinding head to replay your work on top of it... Applying: tests/inmemory: Ensure privileged objects are stored properly
Changes applied before test
commit 5664eb23ff2c6df6320e4d4c02e834ffa7ee9c04
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date: Tue Apr 27 01:16:28 2021 +0530
tests/inmemory: Ensure privileged objects are stored properlySee https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/160/ for more details.
Comment Actions
assert sorted(expected) == sorted(writer.objects)
These are of type ImmutableDict and hence can't be sorted directly
assert writer.privileged_objects == []
throws an error because writer.privileged_objects is of type ListProxy and can't be directly compared to a list
So, I've tried using content swhid for sorting. I had to limit the test to content because TEST_OBJECTS contains objects which don't have swhid function implemented. But IMO, it's fine. Because we're testing whether the object is properly stored in writer.privileged_objects and writer.objects or not. What do you think?
Comment Actions
What is? It works for me.
assert writer.privileged_objects == []throws an error because writer.privileged_objects is of type ListProxy and can't be directly compared to a list
Then convert them to lists.
Comment Actions
- Convert ListProxy to List and compare with []
- Use sorted function and populate writer with TEST_OBJECTS
Comment Actions
Build has FAILED
Patch application report for D5618 (id=20180)
Rebasing onto dd59b67c7f...
First, rewinding head to replay your work on top of it... Applying: tests/inmemory: Ensure privileged objects are stored properly
Changes applied before test
commit 93f594b7c96153bcf00e3cd6e2f4e5aad3d976fc
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date: Tue Apr 27 01:16:28 2021 +0530
tests/inmemory: Ensure privileged objects are stored properlyLink to build: https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/161/
See console output for more information: https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/161/console
Comment Actions
Jenkins is also reporting errors when I'm using this code.
But assert sorted(expected, key=lambda x : x[0]) == sorted(writer.objects, key=lambda x : x[0]) doesn't throw any error.
Comment Actions
Ok, my bad. key=lambda x : x[0] is not good enough because it does fully order everything (there are pairs of x where x[0] is equal, but not x[1]); so your initial approach was the correct one.
Just use .id instead of .swhid(), we don't need a full SWHID here.
Comment Actions
Actually no, we can do better: assert set(expected) == set(writer.objects) should work
Comment Actions
Build is green
Patch application report for D5618 (id=20213)
Rebasing onto dd59b67c7f...
First, rewinding head to replay your work on top of it... Applying: tests/inmemory: Ensure privileged objects are stored properly
Changes applied before test
commit 0c1ecfb0f6baca3cffb4c6a991b66ca773d78756
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date: Tue Apr 27 01:16:28 2021 +0530
tests/inmemory: Ensure privileged objects are stored properlySee https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/162/ for more details.
Comment Actions
Build is green
Patch application report for D5618 (id=20216)
Rebasing onto dd59b67c7f...
Current branch diff-target is up to date.
Changes applied before test
commit 2972c7a694aa09f635f793bcb902de5fd78f1cde
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date: Tue Apr 27 01:16:28 2021 +0530
tests/inmemory: Ensure privileged objects are stored properlySee https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/163/ for more details.