Page MenuHomeSoftware Heritage

Small improvements to the mirror tests
ClosedPublic

Authored by lunar on Sep 27 2022, 5:48 PM.

Details

Summary
  • Add a listing of Python dependencies needed for the tests
  • Add a canonical way to environment variables for the tests
  • Use logging to record tests progression
  • Raise timeouts for replaying, shut down and storage access
  • Use a dedicated network for Docker swarm
  • Remove an unused Docker image from the generation process
  • Add troubleshooting tips to the test documentation

Diff Detail

Repository
rCDFP Deployment tools for hosting a mirror
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 32000
Build 50096: arc lint + arc unit

Event Timeline

lunar requested review of this revision.Sep 27 2022, 5:48 PM
ardumont added a subscriber: ardumont.

lgtm

one non-blocking suggestion inline.

tests/test_graph_replayer.py
326

not on you. Still, would be best to let the log instruction do the formatting (instead of f-strings).

This revision is now accepted and ready to land.Sep 28 2022, 7:00 PM
tests/test_graph_replayer.py
326

Could you explain your rationale? Is it because of extra computation costs if logging is disabled? I find f-strings more readable than other interpolation format so I favored legibility over minor gains in CPU cycles in this case.

tests/test_graph_replayer.py
326

It's about the logging level you set yes. But also about uniformisation in the base
code. If we were to set the log level to warn in production, that'd be still computed
indeed.

If you look at any of our other modules, you'll see we do use f-string for some string
computations but not for the log ones. We let logging do its job.

Also it happens also that some objects (SWHID comes to mind but i may be wrong) used in
f-string could trigger some heavy computations too.

tests/test_graph_replayer.py
326

Although now that i've said that and given that I re-realized it's test code, it may be fine to keep it that way.

To be clear, it's non-blocking otherwise i would have requested changes btw.

lunar marked an inline comment as not done.

Have the log instruction perform the formatting intead of using f-strings