Page MenuHomeSoftware Heritage
Feed Advanced Search

Sep 30 2022

Alphare updated the diff for D8569: Add rubygems loader.

Use the built_at date instead of publication date

Sep 30 2022, 10:26 AM

Sep 29 2022

Alphare added a revision to T4581: RubyGems loader: D8569: Add rubygems loader.
Sep 29 2022, 10:37 AM · RubyGems loader, Archive coverage
Alphare added a task to D8569: Add rubygems loader: T4581: RubyGems loader.
Sep 29 2022, 10:37 AM
Alphare triaged T4581: RubyGems loader as Normal priority.
Sep 29 2022, 10:37 AM · RubyGems loader, Archive coverage
Alphare added inline comments to D8539: Add random directory sampling policy.
Sep 29 2022, 10:29 AM

Sep 28 2022

Alphare updated the diff for D8539: Add random directory sampling policy.

Extract swhid computation

Sep 28 2022, 8:53 PM
Alphare added a comment to D8569: Add rubygems loader.

This is the initial plan that I was going for with regards to what to load inside the .gem.

Sep 28 2022, 6:35 PM
Alphare requested review of D8569: Add rubygems loader.
Sep 28 2022, 6:35 PM
Alphare requested review of D8539: Add random directory sampling policy.
Sep 28 2022, 6:08 PM
Alphare closed D8538: Setup async interface for discovery module.
Sep 28 2022, 5:54 PM
Alphare committed rDLDBASE1facea3cd215: Setup async interface for discovery module (authored by Alphare).
Setup async interface for discovery module
Sep 28 2022, 5:54 PM
Alphare updated the diff for D8538: Setup async interface for discovery module.

Remove ugly hack

Sep 28 2022, 5:40 PM
Alphare added a comment to D8538: Setup async interface for discovery module.

I still don't understand why you need the _all_contents hack instead of doing this:

async def skipped_content_missing(
    self, skipped_contents: List[Sha1Git]
) -> Iterable[Sha1Git]:
    """List skipped content missing from the archive by sha1"""
    contents = [{"sha1_git": s} s in skipped_contents]
    return (d["sha1_git"] for d in self.storage.skipped_content_missing(contents))
Sep 28 2022, 5:17 PM

Sep 27 2022

Alphare updated the diff for D8538: Setup async interface for discovery module.

Add docstring, abstract method decorators, explain hack

Sep 27 2022, 4:53 PM

Sep 26 2022

Alphare updated the diff for D8538: Setup async interface for discovery module.

Use ABC

Sep 26 2022, 10:56 PM
Alphare added a comment to D8538: Setup async interface for discovery module.

One question for reviewers: I've tried using Protocol for ArchiveDiscoveryInterface (should I?), but I'm not sure if it's possible to define a concrete method within the protocol, namely if __init__ could be implemented by default in ArchiveDiscoveryInterface.

Sep 26 2022, 6:35 PM
Alphare requested review of D8538: Setup async interface for discovery module.
Sep 26 2022, 6:33 PM
Alphare closed D8521: Use a Merkle discovery algorithm with archives.
Sep 26 2022, 5:39 PM
Alphare committed rDLDBASE798f749e66c8: Use a Merkle discovery algorithm with archives (authored by Alphare).
Use a Merkle discovery algorithm with archives
Sep 26 2022, 5:39 PM
Alphare updated the diff for D8521: Use a Merkle discovery algorithm with archives.

Rebase

Sep 26 2022, 5:32 PM
Alphare updated the diff for D8521: Use a Merkle discovery algorithm with archives.

Add assertion for data consistency in tests

Sep 26 2022, 5:24 PM
Alphare updated the diff for D8521: Use a Merkle discovery algorithm with archives.

Add coverage of random sampling and skipped content

Sep 26 2022, 4:02 PM
Alphare updated the diff for D8521: Use a Merkle discovery algorithm with archives.

(maybe) fix CI?

Sep 26 2022, 2:13 PM
Alphare updated the diff for D8521: Use a Merkle discovery algorithm with archives.

Address comments

Sep 26 2022, 1:31 PM
Alphare added a comment to D8521: Use a Merkle discovery algorithm with archives.

There seems to be two chunks in get_sample not covered by tests. Could you add some coverage for them?

Sep 26 2022, 1:31 PM

Sep 23 2022

Alphare updated the diff for D8521: Use a Merkle discovery algorithm with archives.

Address nits

Sep 23 2022, 10:01 AM

Sep 22 2022

Alphare updated the diff for D8521: Use a Merkle discovery algorithm with archives.

Fix import

Sep 22 2022, 6:33 PM
Alphare added inline comments to D8521: Use a Merkle discovery algorithm with archives.
Sep 22 2022, 6:14 PM
Alphare updated the diff for D8521: Use a Merkle discovery algorithm with archives.

Optimize loops, add types, improve docs and fix a bug

Sep 22 2022, 6:14 PM
Alphare requested review of D8521: Use a Merkle discovery algorithm with archives.
Sep 22 2022, 12:32 PM

Aug 30 2022

Alphare closed D8298: Add incremental function to Golang Lister.
Aug 30 2022, 2:35 PM
Alphare closed D7329: Non-incremental Golang module lister.
Aug 30 2022, 2:35 PM · Lister
Alphare committed rDLSc6ce862d3250: Add incremental function to Golang Lister (authored by Alphare).
Add incremental function to Golang Lister
Aug 30 2022, 2:35 PM
Alphare committed rDLS60405e78aefd: Add non-incremental Golang modules lister (authored by Alphare).
Add non-incremental Golang modules lister
Aug 30 2022, 2:35 PM
Alphare closed D8296: Golang module loader.
Aug 30 2022, 2:35 PM
Alphare committed rDLDBASE68e68e3f92c0: Golang module loader (authored by Alphare).
Golang module loader
Aug 30 2022, 2:35 PM
Alphare updated the diff for D8296: Golang module loader.

Rebase

Aug 30 2022, 2:35 PM
Alphare updated the diff for D7329: Non-incremental Golang module lister.

Rebase

Aug 30 2022, 2:33 PM · Lister
Alphare updated the diff for D8298: Add incremental function to Golang Lister.

Rebase

Aug 30 2022, 2:33 PM

Aug 29 2022

Alphare updated the diff for D8298: Add incremental function to Golang Lister.

Remove state save when full listing

Aug 29 2022, 4:52 PM
Alphare updated the diff for D8298: Add incremental function to Golang Lister.

Add incremental celery task and its test

Aug 29 2022, 12:36 PM
Alphare added a comment to D8298: Add incremental function to Golang Lister.

You also need to add the celery task for the incremental lister in tasks.py

Aug 29 2022, 12:28 PM
Alphare updated the diff for D8298: Add incremental function to Golang Lister.

Incremental mode, better testing

Aug 29 2022, 12:20 PM
Alphare updated the diff for D7329: Non-incremental Golang module lister.

Fix infinite loop

Aug 29 2022, 12:20 PM · Lister
Alphare updated the diff for D8296: Golang module loader.

Fix loader task

Aug 29 2022, 12:19 PM

Aug 25 2022

Alphare updated the diff for D8296: Golang module loader.

Adapt from lister changes

Aug 25 2022, 12:05 PM
Alphare updated the diff for D8298: Add incremental function to Golang Lister.

Rebase

Aug 25 2022, 12:00 PM
Alphare updated the diff for D7329: Non-incremental Golang module lister.

Use pkg.go.dev instead of the proxy

Aug 25 2022, 11:58 AM · Lister
Alphare added inline comments to D8298: Add incremental function to Golang Lister.
Aug 25 2022, 11:30 AM

Aug 24 2022

Alphare updated the diff for D8296: Golang module loader.

Reflect the changes from the lister

Aug 24 2022, 6:36 PM
Alphare updated the diff for D7329: Non-incremental Golang module lister.

List per package, not per version, prefix with proxy URL

Aug 24 2022, 6:36 PM · Lister
Alphare updated the diff for D8298: Add incremental function to Golang Lister.

List per package, not per version, prefix with proxy URL

Aug 24 2022, 6:35 PM
Alphare retitled D8296: Golang module loader from Non-incremental Golang module loader to Golang module loader.
Aug 24 2022, 4:11 PM
Alphare added a comment to D8296: Golang module loader.

Hmm... okay then. I guess for reproducibility it's more useful to get the same thing go get than the backend. What do origin URLs look like, though?

Aug 24 2022, 3:26 PM
Alphare updated the diff for D8296: Golang module loader.

Fix release names and update description

Aug 24 2022, 3:24 PM
Alphare updated the diff for D8296: Golang module loader.

Remove useless cached_property and document releases

Aug 24 2022, 2:09 PM
Alphare added a comment to D8296: Golang module loader.

Do we really need to use the Go proxy? Can't/Shouldn't we access the real origins directly?

Aug 24 2022, 2:07 PM
Alphare closed D8302: Add a small note about scripts for database setup.
Aug 24 2022, 12:06 PM
Alphare committed rDDOC99264b0b91c9: Add a small note about scripts for database setup (authored by Alphare).
Add a small note about scripts for database setup
Aug 24 2022, 12:06 PM
Alphare updated the diff for D8302: Add a small note about scripts for database setup.

Move to the end and add a note about it being optional

Aug 24 2022, 11:55 AM
Alphare requested review of D8302: Add a small note about scripts for database setup.
Aug 24 2022, 11:49 AM
Alphare updated the diff for D8298: Add incremental function to Golang Lister.

Rebase

Aug 24 2022, 11:13 AM
Alphare added inline comments to D8298: Add incremental function to Golang Lister.
Aug 24 2022, 11:12 AM
Alphare updated the diff for D7329: Non-incremental Golang module lister.

Add docstring

Aug 24 2022, 11:09 AM · Lister

Aug 23 2022

Alphare requested review of D8298: Add incremental function to Golang Lister.
Aug 23 2022, 5:15 PM
Alphare requested review of D8296: Golang module loader.
Aug 23 2022, 4:16 PM

Aug 22 2022

Alphare updated the diff for D7329: Non-incremental Golang module lister.

Update README, sort setup.py and use f-strings

Aug 22 2022, 3:59 PM · Lister

Apr 28 2022

Alphare closed D7708: Update unresolved issues.
Apr 28 2022, 11:40 AM
Alphare committed rDLDBZRf9d1fe63466a: Update unresolved issues (authored by Alphare).
Update unresolved issues
Apr 28 2022, 11:40 AM
Alphare closed D7707: Add suggestions from current bzr maintainer.
Apr 28 2022, 11:39 AM
Alphare committed rDLDBZR85af3e074199: Add suggestions from current bzr maintainer (authored by Alphare).
Add suggestions from current bzr maintainer
Apr 28 2022, 11:39 AM
Alphare requested review of D7708: Update unresolved issues.
Apr 28 2022, 11:25 AM
Alphare requested review of D7707: Add suggestions from current bzr maintainer.
Apr 28 2022, 11:24 AM

Apr 27 2022

Alphare created T4201: Support empty directories.
Apr 27 2022, 4:38 PM · Core Loader, BZR loader

Apr 14 2022

Alphare added inline comments to D7501: Rust lang, Crates loader.
Apr 14 2022, 2:17 PM

Mar 24 2022

Alphare accepted D7367: Start rust crates lister.

Looks good to me now. There still remains the question of whether we want to store the yanked info for each release somewhere. Off the top of my head it would make the (upcoming) incremental listing/loading less efficient. Maybe that's out of scope for now. :)

Mar 24 2022, 9:12 AM

Mar 21 2022

Alphare requested changes to D7367: Start rust crates lister.

Oops, it seems like this got out of draft too fast. arc says --update and --draft are mutually exclusive for some reason so it's not obvious to me how to tell Phabricator that we still want a diff to stay draft even if the CI passes.

Mar 21 2022, 6:06 PM

Mar 16 2022

Alphare updated the diff for D7329: Non-incremental Golang module lister.

Add test_tasks.py

Mar 16 2022, 10:17 AM · Lister

Mar 14 2022

Alphare added a comment to D7329: Non-incremental Golang module lister.

LGTM

Mar 14 2022, 5:51 PM · Lister

Mar 11 2022

Alphare updated the diff for D7329: Non-incremental Golang module lister.

Remove use of dateutil and add small fixes

Mar 11 2022, 2:51 PM · Lister

Mar 10 2022

Alphare published D7329: Non-incremental Golang module lister for review.
Mar 10 2022, 4:16 PM · Lister

Feb 21 2022

Alphare closed D7208: Support new repositories with old branches.
Feb 21 2022, 4:30 PM
Alphare committed rDLDBZRd77083d7735e: Support new repositories with old branches (authored by Alphare).
Support new repositories with old branches
Feb 21 2022, 4:30 PM
Alphare updated the diff for D7208: Support new repositories with old branches.

Incorporate agreed upon changes

Feb 21 2022, 4:25 PM
Alphare added a comment to D7208: Support new repositories with old branches.

Thanks, how about those?

Feb 21 2022, 4:17 PM
Alphare updated the diff for D7208: Support new repositories with old branches.

Add typing and change commit description

Feb 21 2022, 4:09 PM
Alphare requested review of D7208: Support new repositories with old branches.
Feb 21 2022, 4:02 PM
Alphare accepted D7202: loader: Upgrade repository with old formats prior loading.

Oh, I didn't realize that my approval was needed.

Feb 21 2022, 2:03 PM
Alphare added a comment to D7202: loader: Upgrade repository with old formats prior loading.

Should this be folded with the patch in T3963#79379?

Feb 21 2022, 1:57 PM
Alphare added a task to D7202: loader: Upgrade repository with old formats prior loading: Unknown Object (Maniphest Task).
Feb 21 2022, 1:45 PM

Feb 17 2022

Alphare accepted D7193: launchpad: Allow bzr origins listing.

This looks good, considering the... interesting launchpad API.

Feb 17 2022, 10:48 AM

Feb 14 2022

Alphare closed D7172: sourceforge: fix support for listing bzr origins.
Feb 14 2022, 3:05 PM
Alphare committed rDLS31b4429ced3d: sourceforge: fix support for listing bzr origins (authored by Alphare).
sourceforge: fix support for listing bzr origins
Feb 14 2022, 3:05 PM
Alphare requested review of D7172: sourceforge: fix support for listing bzr origins.
Feb 14 2022, 2:59 PM

Feb 10 2022

Alphare closed D7147: Add URL and directory to CLI loader status echo.
Feb 10 2022, 4:20 PM
Alphare committed rDLDBASEd89c5378c7cf: Add URL and directory to CLI loader status echo (authored by Alphare).
Add URL and directory to CLI loader status echo
Feb 10 2022, 4:20 PM
Alphare accepted D7148: Add type annotations, and fix issues found by mypy.
Feb 10 2022, 4:18 PM
Alphare added a comment to T3915: Deploy Bzr loader to the staging environment.

D7130 has been committed, so a new version of the loader for faster ingestion will need to be released.

Feb 10 2022, 3:28 PM · System administration, Staging environment, BZR loader
Alphare closed D7130: Add fast-path to the bzr loader.
Feb 10 2022, 3:22 PM
Alphare committed rDLDBZR0ffc124709cf: Add fast-path to the bzr loader (authored by Alphare).
Add fast-path to the bzr loader
Feb 10 2022, 3:22 PM