Page MenuHomeSoftware Heritage
Feed Advanced Search

Oct 14 2021

anlambert updated the summary of D6476: tests: Ensure failing test with random fixture data can be reproduced.
Oct 14 2021, 4:53 PM
anlambert requested review of D6476: tests: Ensure failing test with random fixture data can be reproduced.
Oct 14 2021, 4:37 PM
anlambert requested review of D6471: tests/conftest: Put filtered input data in cache.
Oct 14 2021, 1:55 PM
anlambert closed D6463: tests: Turn remaining hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM
anlambert closed T3603: Replace stateful hypothesis strategies by pytest fixtures as Resolved by committing rDWAPPSa200f70f2ce9: tests: Turn remaining hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM · Web app
anlambert committed rDWAPPSa200f70f2ce9: tests: Turn remaining hypothesis strategies into pytest fixtures (authored by anlambert).
tests: Turn remaining hypothesis strategies into pytest fixtures
Oct 14 2021, 11:04 AM
anlambert closed D6461: tests: Turn snapshot* hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM
anlambert closed D6460: tests: Turn revision* hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM
anlambert closed D6459: tests: Turn release* hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM
anlambert committed rDWAPPSa91200cbe1e5: tests: Turn snapshot* hypothesis strategies into pytest fixtures (authored by anlambert).
tests: Turn snapshot* hypothesis strategies into pytest fixtures
Oct 14 2021, 11:04 AM
anlambert committed rDWAPPS341454015ded: tests: Turn revision* hypothesis strategies into pytest fixtures (authored by anlambert).
tests: Turn revision* hypothesis strategies into pytest fixtures
Oct 14 2021, 11:04 AM
anlambert committed rDWAPPSc4fed93688b1: tests: Turn release* hypothesis strategies into pytest fixtures (authored by anlambert).
tests: Turn release* hypothesis strategies into pytest fixtures
Oct 14 2021, 11:04 AM
anlambert closed D6458: tests: Turn origin* hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM
anlambert closed D6457: tests: Turn directory* hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM
anlambert committed rDWAPPS81486f517526: tests: Turn directory* hypothesis strategies into pytest fixtures (authored by anlambert).
tests: Turn directory* hypothesis strategies into pytest fixtures
Oct 14 2021, 11:04 AM
anlambert committed rDWAPPSc2d10c90c3bd: tests: Turn origin* hypothesis strategies into pytest fixtures (authored by anlambert).
tests: Turn origin* hypothesis strategies into pytest fixtures
Oct 14 2021, 11:04 AM
anlambert closed D6455: tests: Turn content* hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM
anlambert committed rDWAPPS23a8dc33453f: tests: Turn content* hypothesis strategies into pytest fixtures (authored by anlambert).
tests: Turn content* hypothesis strategies into pytest fixtures
Oct 14 2021, 11:04 AM
anlambert added a comment to D6458: tests: Turn origin* hypothesis strategies into pytest fixtures.
In D6458#167772, @olasd wrote:
In D6458#167771, @olasd wrote:

Yeah, sure, I don't have a problem with that.

(That is, I don't have a problem with these changes landing first, as long as we make sure that eventually we have a proper way of reproducing test failures that have come out of "random" fixtures.)

Oct 14 2021, 11:03 AM

Oct 13 2021

anlambert updated the diff for D6463: tests: Turn remaining hypothesis strategies into pytest fixtures.

Rebase

Oct 13 2021, 4:53 PM
anlambert updated the diff for D6461: tests: Turn snapshot* hypothesis strategies into pytest fixtures.

Add missing tests_data parameter to snapshot_swhid fixture

Oct 13 2021, 4:51 PM
anlambert added a comment to D6458: tests: Turn origin* hypothesis strategies into pytest fixtures.
In D6458#167702, @olasd wrote:

Thanks for working on reducing the number of hypothesis fixtures!

I'm a bit concerned about the reproducibility of test results, given fixtures that pull random list elements, with no control on the sequence of test executions and on the seed of the python random module when the fixture is called. (Now that I've looked at swh.web.tests.data, I'm even more concerned :-))

I don't have an answer about "what to use?", unfortunately, except just going for exhaustive tests (i.e. running the test functions for *all* values of the origins in the test data set), which doesn't sound very compelling unless the size of the sample dataset is small, which doesn't look to be the case.

https://github.com/pytest-dev/pytest/issues/5463 has some background about concerns with respect to random seeding in tests.

Apart from that, I see that some of the function-level fixtures are doing ""heavy"" querying on the test data for information that is, in effect, static (e.g. the list of origins with more than two visits, etc.). I wonder if it would be possible to extract this logic to only run it once on initialization of the test data?

I initially wrote: we may want to initialize a single, module scoped seed_storage fixture with all data inserted, and make the storage fixture used by tests a function-scoped fixture which would clone this seed storage instance - I assume some tests have to *write* to the storage, so you can't just have one global read only storage fixture - but I now see that's what swh.web.tests.data does. Maybe _init_tests_data could be turned into that seed_storage module-scoped pytest fixture, instead of the current ad-hoc logic? This would also help us control the random seed used for generating the test data (allowing us to override it to reproduce test results)?

Oct 13 2021, 4:10 PM

Oct 12 2021

anlambert requested review of D6463: tests: Turn remaining hypothesis strategies into pytest fixtures.
Oct 12 2021, 5:43 PM
anlambert added a revision to T3603: Replace stateful hypothesis strategies by pytest fixtures: D6463: tests: Turn remaining hypothesis strategies into pytest fixtures.
Oct 12 2021, 5:25 PM · Web app
anlambert requested review of D6461: tests: Turn snapshot* hypothesis strategies into pytest fixtures.
Oct 12 2021, 4:08 PM
anlambert requested review of D6460: tests: Turn revision* hypothesis strategies into pytest fixtures.
Oct 12 2021, 3:57 PM
anlambert added a revision to T3603: Replace stateful hypothesis strategies by pytest fixtures: D6461: tests: Turn snapshot* hypothesis strategies into pytest fixtures.
Oct 12 2021, 3:53 PM · Web app
anlambert added a revision to T3603: Replace stateful hypothesis strategies by pytest fixtures: D6460: tests: Turn revision* hypothesis strategies into pytest fixtures.
Oct 12 2021, 3:27 PM · Web app
anlambert requested review of D6459: tests: Turn release* hypothesis strategies into pytest fixtures.
Oct 12 2021, 2:27 PM
anlambert added a revision to T3603: Replace stateful hypothesis strategies by pytest fixtures: D6459: tests: Turn release* hypothesis strategies into pytest fixtures.
Oct 12 2021, 2:09 PM · Web app
anlambert requested review of D6458: tests: Turn origin* hypothesis strategies into pytest fixtures.
Oct 12 2021, 2:04 PM
anlambert added a revision to T3603: Replace stateful hypothesis strategies by pytest fixtures: D6458: tests: Turn origin* hypothesis strategies into pytest fixtures.
Oct 12 2021, 1:49 PM · Web app
anlambert requested review of D6457: tests: Turn directory* hypothesis strategies into pytest fixtures.
Oct 12 2021, 12:03 PM
anlambert added a revision to T3603: Replace stateful hypothesis strategies by pytest fixtures: D6457: tests: Turn directory* hypothesis strategies into pytest fixtures.
Oct 12 2021, 11:47 AM · Web app
anlambert updated the diff for D6455: tests: Turn content* hypothesis strategies into pytest fixtures.

Forgot to remove content_swhid strategy

Oct 12 2021, 11:27 AM
anlambert updated the diff for D6455: tests: Turn content* hypothesis strategies into pytest fixtures.

Fix typo in function name

Oct 12 2021, 11:01 AM
anlambert updated the diff for D6455: tests: Turn content* hypothesis strategies into pytest fixtures.

Rebase

Oct 12 2021, 10:54 AM
anlambert requested changes to D6441: Deprecate /browse/origin/log/ URLs.

Still some work to do before I can accept the diff:

  • add missing tests
  • remove the use of the browse-origin-log view in codebase (grep for it)
Oct 12 2021, 10:50 AM
anlambert added a comment to D6455: tests: Turn content* hypothesis strategies into pytest fixtures.

lgtm

I suppose you'll align the other hypothesis strategies the same way in other diffs?

Oct 12 2021, 10:38 AM

Oct 11 2021

anlambert requested review of D6455: tests: Turn content* hypothesis strategies into pytest fixtures.
Oct 11 2021, 5:13 PM
anlambert added a revision to T3603: Replace stateful hypothesis strategies by pytest fixtures: D6455: tests: Turn content* hypothesis strategies into pytest fixtures.
Oct 11 2021, 4:59 PM · Web app
anlambert retitled D6441: Deprecate /browse/origin/log/ URLs from Deprecate most of the /browse/origin/.* URLs to Deprecate /browse/origin/log/ URLs.
Oct 11 2021, 3:48 PM
anlambert requested changes to D6441: Deprecate /browse/origin/log/ URLs.

It seems that the browse-snapshot-log view is missing tests so we should create the file swh/web/tests/browse/views/test_snapshot.py
and implement tests here.

Oct 11 2021, 3:47 PM
anlambert accepted D6449: Navigation to directory(code) is broken in snapshot view.

Looks good to me, thanks !

Oct 11 2021, 3:35 PM
anlambert accepted D6453: converters: Prevent zero dates from being converted to null dates..

Looks good to me.

Oct 11 2021, 2:26 PM
anlambert requested changes to D6449: Navigation to directory(code) is broken in snapshot view.

Looks good for the code but still some issues in the commit message (typo + formatting),
I would have done like this for instance.

Oct 11 2021, 2:17 PM
anlambert closed T3618: Reschedule loading of dumb git origins submitted to "Save code now" service as Resolved.

Issues related to git dumb loading have been handled and the 5 dumb origins that were failing have been resubmitted through save code now and successfully loaded, closing this.

Oct 11 2021, 1:51 PM · Save Code Now, Archive coverage, Git loader
anlambert added inline comments to D6441: Deprecate /browse/origin/log/ URLs.
Oct 11 2021, 1:44 PM
anlambert requested changes to D6449: Navigation to directory(code) is broken in snapshot view.

Looks good but some small changes are required.

Oct 11 2021, 1:33 PM
anlambert requested changes to D6441: Deprecate /browse/origin/log/ URLs.

I added some inline comments to handle.

Oct 11 2021, 1:22 PM
anlambert added a comment to T3641: Top level tabs not working in snapshot view.

Looks like the bug comes from that piece of javascript code. We should test if the pathname ends with /branches/ or /releases/ instead to activate the correct tab.

Oct 11 2021, 12:00 PM · Web app

Oct 7 2021

anlambert accepted D6437: Rename imports of swh.model.identifiers to fix deprecation warnings..
Oct 7 2021, 3:37 PM
anlambert closed D6436: Replace use of deprecated swh.model.identifiers module.
Oct 7 2021, 3:17 PM
anlambert committed rDWAPPS264c0bc84bee: Replace use of deprecated swh.model.identifiers module (authored by anlambert).
Replace use of deprecated swh.model.identifiers module
Oct 7 2021, 3:17 PM
anlambert requested review of D6436: Replace use of deprecated swh.model.identifiers module.
Oct 7 2021, 3:12 PM
anlambert closed D6434: faq: Fix broken links related to Web API.
Oct 7 2021, 12:06 PM
anlambert committed rDDOC60f9bb1d6d91: faq: Fix broken links related to Web API (authored by anlambert).
faq: Fix broken links related to Web API
Oct 7 2021, 12:06 PM
anlambert accepted D6435: Replace imports/references to swh.model.identifiers, it's deprecated..
Oct 7 2021, 12:06 PM
anlambert accepted D6429: docs: Update for the new API + remove references to deprecated module swh.model.identifiers.

Looks good to me.

Oct 7 2021, 12:04 PM
anlambert requested review of D6434: faq: Fix broken links related to Web API.
Oct 7 2021, 11:56 AM
anlambert closed D6432: keycloak: Add swh.web.api.save_origin role to swh-web client.
Oct 7 2021, 11:35 AM
anlambert committed rSPSITE8bf99a574a06: keycloak: Add swh.web.api.save_origin role to swh-web client (authored by anlambert).
keycloak: Add swh.web.api.save_origin role to swh-web client
Oct 7 2021, 11:35 AM
anlambert added a comment to T3608: Deprecate most of the /browse/origin/.* URLs.

Yes we can, the snapshot should be provided as query parameter in other views in that case.

Thanks. Then we can delete most of the code in https://forge.softwareheritage.org/source/swh-web/browse/master/swh/web/browse/snapshot_context.py

Testing will be a bit tricky, I will try to add some generic cypress test.

Oct 7 2021, 11:27 AM · Web app
anlambert requested review of D6432: keycloak: Add swh.web.api.save_origin role to swh-web client.
Oct 7 2021, 11:19 AM
anlambert closed D6425: origin_save: Lift save request creation restrictions with permission.
Oct 7 2021, 11:16 AM
anlambert committed rDWAPPS0da4e46ae3c1: origin_save: Lift save request creation restrictions with permission (authored by anlambert).
origin_save: Lift save request creation restrictions with permission
Oct 7 2021, 11:16 AM
anlambert added a comment to T3608: Deprecate most of the /browse/origin/.* URLs.

@anlambert do you think we can deprecate following routes as well? I think they can be redirected to the corresponding swh/web/browse/views/<object_type>.py routes.

  • /snapshot/(?P<snapshot_id>[0-9a-f]+)/content/
  • /snapshot/(?P<snapshot_id>[0-9a-f]+)/directory/(?P<path>.+)
  • / snapshot/(?P<snapshot_id>[0-9a-f]+)/directory/
  • `/snapshot/(?P<snapshot_id>[0-9a-f]+)/content/(?P<path>.+)/'

we can delete a lot of code from snapshot_context.py module then

Oct 7 2021, 11:09 AM · Web app
anlambert added a comment to T3608: Deprecate most of the /browse/origin/.* URLs.

Just to be clear, you're looking to keep these URL working, but turn them into redirects over to swhid-centric URLs with context parameters (and drop the original view code from these URLs), correct?

Oct 7 2021, 11:08 AM · Web app

Oct 6 2021

anlambert requested review of D6425: origin_save: Lift save request creation restrictions with permission.
Oct 6 2021, 5:39 PM
anlambert added a project to T3286: Use journal clients for webapp and deposit to subscribe to events: Save Code Now.
Oct 6 2021, 5:14 PM · Save Code Now, SWORD deposit, Web app
anlambert committed rDWAPPS6c72612902a6: settings/tests: ensure sqlite db file has sqlite3 extension (authored by anlambert).
settings/tests: ensure sqlite db file has sqlite3 extension
Oct 6 2021, 4:33 PM
anlambert committed rDWAPPScc60a4a5c88e: assets, cypress: Remove debug logs (authored by anlambert).
assets, cypress: Remove debug logs
Oct 6 2021, 4:33 PM
anlambert committed rDWAPPSc8d26ac59080: package.json: Upgrade dependencies (authored by anlambert).
package.json: Upgrade dependencies
Oct 6 2021, 4:33 PM
anlambert closed D6422: assets/origin/bundles: Restore visualizations state on page reload.
Oct 6 2021, 4:33 PM
anlambert committed rDWAPPS2bcf33c2e527: assets/origin/bundles: Restore visualizations state on page reload (authored by anlambert).
assets/origin/bundles: Restore visualizations state on page reload
Oct 6 2021, 4:33 PM
anlambert closed D6421: assets/bundles/origin: Add missing statuses to visits reporting.
Oct 6 2021, 4:33 PM
anlambert committed rDWAPPSd394d9452dbd: assets/bundles/origin: Add missing statuses to visits reporting (authored by anlambert).
assets/bundles/origin: Add missing statuses to visits reporting
Oct 6 2021, 4:33 PM
anlambert requested review of D6422: assets/origin/bundles: Restore visualizations state on page reload.
Oct 6 2021, 3:24 PM
anlambert requested review of D6421: assets/bundles/origin: Add missing statuses to visits reporting.
Oct 6 2021, 3:21 PM

Oct 5 2021

anlambert closed D6416: dumb: Handle missing or corrupted pack file.
Oct 5 2021, 5:58 PM
anlambert committed rDLDGd3d60421423a: dumb: Handle missing or corrupted pack file (authored by anlambert).
dumb: Handle missing or corrupted pack file
Oct 5 2021, 5:58 PM
anlambert closed D6412: browse/snapshot_context: Fix empty snapshot browsing for staff users.
Oct 5 2021, 5:30 PM
anlambert committed rDWAPPSc8c0ac1eab92: browse/snapshot_context: Fix empty snapshot browsing for staff users (authored by anlambert).
browse/snapshot_context: Fix empty snapshot browsing for staff users
Oct 5 2021, 5:30 PM
anlambert updated the diff for D6412: browse/snapshot_context: Fix empty snapshot browsing for staff users.

Rebase

Oct 5 2021, 5:03 PM
anlambert closed D6311: admin/origin_save: Parse optional note from rejection POST request body.
Oct 5 2021, 5:02 PM
anlambert committed rDWAPPS84e550f6bf68: admin/origin_save: Parse optional note from rejection POST request body (authored by anlambert).
admin/origin_save: Parse optional note from rejection POST request body
Oct 5 2021, 5:02 PM
anlambert closed D6309: common/models: Add note text field to SaveOriginRequest model.
Oct 5 2021, 5:02 PM
anlambert committed rDWAPPS86601bceb57b: common/models: Add note text field to SaveOriginRequest model (authored by anlambert).
common/models: Add note text field to SaveOriginRequest model
Oct 5 2021, 5:02 PM
anlambert updated the diff for D6311: admin/origin_save: Parse optional note from rejection POST request body.

Fix test after rebase

Oct 5 2021, 4:45 PM
anlambert updated the diff for D6311: admin/origin_save: Parse optional note from rejection POST request body.

Rebase

Oct 5 2021, 4:34 PM
anlambert updated the diff for D6309: common/models: Add note text field to SaveOriginRequest model.

Rebase

Oct 5 2021, 4:32 PM
anlambert requested review of D6416: dumb: Handle missing or corrupted pack file.
Oct 5 2021, 4:16 PM
anlambert added a revision to T3618: Reschedule loading of dumb git origins submitted to "Save code now" service: D6416: dumb: Handle missing or corrupted pack file.
Oct 5 2021, 4:14 PM · Save Code Now, Archive coverage, Git loader
anlambert closed D6415: dumb: Handle empty repository edge case.
Oct 5 2021, 4:14 PM
anlambert committed rDLDG6b21441da65b: dumb: Handle empty repository edge case (authored by anlambert).
dumb: Handle empty repository edge case
Oct 5 2021, 4:14 PM
anlambert requested review of D6415: dumb: Handle empty repository edge case.
Oct 5 2021, 3:39 PM
anlambert added a revision to T3618: Reschedule loading of dumb git origins submitted to "Save code now" service: D6415: dumb: Handle empty repository edge case.
Oct 5 2021, 3:37 PM · Save Code Now, Archive coverage, Git loader
anlambert accepted D6414: Rename imports of swh.model.identifiers to fix deprecation warnings..
Oct 5 2021, 3:34 PM
anlambert requested review of D6412: browse/snapshot_context: Fix empty snapshot browsing for staff users.
Oct 5 2021, 3:14 PM
anlambert closed D6411: tests: Wrap users creation in pytest fixtures.
Oct 5 2021, 2:57 PM