User Details
- User Since
- Sep 1 2019, 9:45 AM (271 w, 3 d)
Aug 27 2020
@samplet wow! that's pretty cool! Thank you ;)
Aug 2 2020
I'm currently using the following regex to filter the exposed urls .tar.gz$|.zip$|tar.bz2$|.tbz$|.tar.xz$|.tgz$|.tar$ but I'm pretty sure it could be improved.
Jun 17 2020
We need a name for this origin type, one of the hardest problem in CS :-)
Where is it used? Is it a new attribute?
We actually had to choose a name for the visit type, and with a lot of inspiration, we choose nixguix :-/
Do you mean filter the unsupported urls for the field "urls" in the "type": "url"?
Or do you mean only export "type": "url" and remove all the other types from 'sources.json', for instance "git"?
Jun 15 2020
The nixguix loader is working well since 2 weeks on the nixpkgs sources.json file!
So, we can now consider the sources.json file format as stable and you could make the required changes on your sources.json file. A new SHW origin should then be added.
Jun 5 2020
Fix ardumont comment and rebase
Jun 2 2020
May 27 2020
May 25 2020
I'm wondering if we should not catch the exception on a missing metadatas instead. It is much more simple and I think we could have more branches like the evaluation one in the future. If we add git sources, the git loader will be in charge of loading the source and the nixguix loader will still add a pending branch to the snapshot.
May 14 2020
What's the status of this patch series? Would be great to deploy it. :-)
Apr 29 2020
Rebased
Apr 15 2020
Rebased
Apr 3 2020
Fix flake8
Addressed @ardumont comments. Thanks for your review!
Apr 2 2020
@olasd I addressed all of your comments. Thanks for your review.
Mar 31 2020
Remove the second loader instanciation in test_loader_incremental.
Mar 30 2020
@olasd Another drawback is I would need to make one API call per release in order to list all integrity attributes of a snapshot. Even with a cache on the client side, the first run would require ~15000 calls to the SWH API. Do you have a rate limiting on these API endpoints?
Reduce commit diff :/
No longer use the integrity as branch name but the url instead.
Thanks for your review. I'm applying your suggestion which looks nice.
Mar 27 2020
- nixguix: rename the test file
Mar 25 2020
I don't know if nix_guix would be better than nixguix for the module name!
Mar 24 2020
Mar 23 2020
UPdate cmomit message to only mention the test
Rebased
What about this diff? I think it's still valuable since it catch more precisely errors and add a test case;)
WDYT?
Mar 20 2020
Fix a typo
I added a test for the EOFError.
I actually succeeded with a bigger truncated file actually: 70KB
Mar 19 2020
@ardumont I don't understand why the storage doesn't contains 2 contents (a tarball + a file). The test test_loader_one_visit is failing because there is only 1 content in the storage.
- WIP
Rebased on master
Mar 18 2020
- functional: create a branch named evaluation pointing to the evaluation commit
- package.loader: add extra_branches method
- functional: create a branch named evaluation pointing to the evaluation commit
Mar 17 2020
@olasd Thanks for your comments. I addressed all of them.
- Add the functional loader
I'm not so sure about refactoring the base package loader to have two url and origin arguments that are the same most of the time but not all the time; In concrete terms, this means that we should refactor all loaders to use self.origin in the places where they currently use self.url.
For instance, we have the same issue with the pypi loader, where we generate an api url (which we're downloading) from the url of the origin (which is just a user-facing link).
All in all, I think we need to think about doing a refactoring to do this consistently, but I don't think this diff is the right place to do this refactoring; For now, we can just have a hardcoded map from origin url to index url.
Mar 16 2020
- package.loader: add extra_branches method
- functional: create a branch named evaluation pointing to the evaluation commit
@ardumont I squashed almost all commits and added a task test.
- Add the functional loader
I think I need to implement a test for the task also. WDYT?
- package.loader: add extra_branches method
- functional: create a branch named evaluation pointing to the evaluation commit
- package.loader: ignore non tarball source
- package.loader: add origin argument
- Add the functional loader
- cli: add the functional loader in the cli tests
- functional: add test for non json sources files
- functional: improve test_loader_two_visits
- functional: add test_loader_incremental
- functional: move retrieve_sources out of the FunctionalLoader class
- functional: test the origin_visit
- functional: add test_uncompress_failure
- functional: minor fixes in tests
- package.loader: add extra_branches method
- functional: create a branch named evaluation pointing to the evaluation commit
Mar 13 2020
- package.loader: add origin argument
- Add the functional loader
- cli: add the functional loader in the cli tests
- functional: add test for non json sources files
- functional: improve test_loader_two_visits
- functional: add test_loader_incremental
- functional: move retrieve_sources out of the FunctionalLoader class
- functional: test the origin_visit
- functional: add test_uncompress_failure
- functional: minor fixes in tests
So here, i'm missing:
- loading scenario which test the incremental nature of the visit (that'd test resolve_revision_from)
- functional: improve test_loader_two_visits
- functional: add test_loader_incremental
- functional: move retrieve_sources out of the FunctionalLoader class
- functional: test the origin_visit
- functional: add test_uncompress_failure
Mar 12 2020
I actually don't know how should I manage errors. It seems exception are generally just emitted, without a dedicated log message.
How do you catch errors on production?
- functional: add test for non json sources files
Mar 11 2020
Try to base on functional-loader branch
- functional: create a branch named evaluation pointing to the evaluation commit
- functional: create a branch named evaluation pointing to the evaluation commit
- package.loader: ignore non tarball source
- package.loader: add origin argument
- Add the functional loader
- cli: add the functional loader in the cli tests
- package.loader: add hook_branches method
- functional: create a branch named evaluation pointing to the evaluation commit