Page MenuHomeSoftware Heritage

tests/inmemory: Ensure privileged objects are stored properly
ClosedPublic

Authored by KShivendu on Apr 26 2021, 9:52 PM.

Details

Summary

In memory kafkfa writer lacks tests.
I'm adding two new tests that ensure the proper handling of privileged objects in case of in memory implementation.
These tests also cover the scenario mentioned in T2823#63777

Related T2823

Diff Detail

Repository
rDJNL Journal infrastructure
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D5618 (id=20030)

Rebasing onto 236a00262e...

Current branch diff-target is up to date.
Changes applied before test
commit 3dd4b0d29548db9977aa203061c7ffe8015ea9d3
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Apr 27 01:16:28 2021 +0530

    tests/inmemory: Ensure privileged objects are stored properly

See https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/158/ for more details.

Compare the expected values directly:

assert sorted(expected) == sorted(writer.objects)
assert writer.privileged_objects == []
This revision now requires changes to proceed.Apr 27 2021, 9:31 AM

Improve time complexity using sorting

Build is green

Patch application report for D5618 (id=20143)

Rebasing onto dd59b67c7f...

First, rewinding head to replay your work on top of it...
Applying: tests/inmemory: Ensure privileged objects are stored properly
Changes applied before test
commit 5664eb23ff2c6df6320e4d4c02e834ffa7ee9c04
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Apr 27 01:16:28 2021 +0530

    tests/inmemory: Ensure privileged objects are stored properly

See https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/160/ for more details.

assert sorted(expected) == sorted(writer.objects)

These are of type ImmutableDict and hence can't be sorted directly

assert writer.privileged_objects == []

throws an error because writer.privileged_objects is of type ListProxy and can't be directly compared to a list

So, I've tried using content swhid for sorting. I had to limit the test to content because TEST_OBJECTS contains objects which don't have swhid function implemented. But IMO, it's fine. Because we're testing whether the object is properly stored in writer.privileged_objects and writer.objects or not. What do you think?

assert sorted(expected) == sorted(writer.objects)

These are of type ImmutableDict and hence can't be sorted directly

What is? It works for me.

assert writer.privileged_objects == []

throws an error because writer.privileged_objects is of type ListProxy and can't be directly compared to a list

Then convert them to lists.

  • Convert ListProxy to List and compare with []
  • Use sorted function and populate writer with TEST_OBJECTS

Build has FAILED

Patch application report for D5618 (id=20180)

Rebasing onto dd59b67c7f...

First, rewinding head to replay your work on top of it...
Applying: tests/inmemory: Ensure privileged objects are stored properly
Changes applied before test
commit 93f594b7c96153bcf00e3cd6e2f4e5aad3d976fc
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Apr 27 01:16:28 2021 +0530

    tests/inmemory: Ensure privileged objects are stored properly

Link to build: https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/161/
See console output for more information: https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/161/console

assert sorted(expected) == sorted(writer.objects)

These are of type ImmutableDict and hence can't be sorted directly

What is? It works for me.

Jenkins is also reporting errors when I'm using this code.
But assert sorted(expected, key=lambda x : x[0]) == sorted(writer.objects, key=lambda x : x[0]) doesn't throw any error.

Ok, my bad. key=lambda x : x[0] is not good enough because it does fully order everything (there are pairs of x where x[0] is equal, but not x[1]); so your initial approach was the correct one.

Just use .id instead of .swhid(), we don't need a full SWHID here.

Actually no, we can do better: assert set(expected) == set(writer.objects) should work

Build is green

Patch application report for D5618 (id=20213)

Rebasing onto dd59b67c7f...

First, rewinding head to replay your work on top of it...
Applying: tests/inmemory: Ensure privileged objects are stored properly
Changes applied before test
commit 0c1ecfb0f6baca3cffb4c6a991b66ca773d78756
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Apr 27 01:16:28 2021 +0530

    tests/inmemory: Ensure privileged objects are stored properly

See https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/162/ for more details.

This revision is now accepted and ready to land.May 3 2021, 10:39 AM

Build is green

Patch application report for D5618 (id=20216)

Rebasing onto dd59b67c7f...

Current branch diff-target is up to date.
Changes applied before test
commit 2972c7a694aa09f635f793bcb902de5fd78f1cde
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Apr 27 01:16:28 2021 +0530

    tests/inmemory: Ensure privileged objects are stored properly

See https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/163/ for more details.