Page MenuHomeSoftware Heritage

KShivendu (Kumar Shivendu)
User

Projects

User does not belong to any projects.

User Details

User Since
Feb 23 2021, 11:48 AM (92 w, 1 d)

Recent Activity

Yesterday

KShivendu requested review of D8907: feat: Add Hex.pm lister.
Wed, Nov 30, 8:22 PM
KShivendu added a revision to T4687: Hex.pm lister (Erlang package manager): D8907: feat: Add Hex.pm lister.
Wed, Nov 30, 8:17 PM · Lister

Wed, Nov 16

KShivendu closed D8753: feat: Introduce RPM loader.
Wed, Nov 16, 2:19 PM
KShivendu committed rDLDBASEa196c85d5ab1: feat: Incremental RPM loader implementation (authored by KShivendu).
feat: Incremental RPM loader implementation
Wed, Nov 16, 2:19 PM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • Fix license headers
  • Fix test cases to match Fedora lister output
Wed, Nov 16, 2:09 PM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • Use intrinsic version (and other suggestions by @anlambert)
  • Add tests for the same
Wed, Nov 16, 9:51 AM

Tue, Nov 15

KShivendu added inline comments to D8753: feat: Introduce RPM loader.
Tue, Nov 15, 3:43 PM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • Rebase against master
Tue, Nov 15, 3:35 PM
KShivendu updated the task description for T4687: Hex.pm lister (Erlang package manager).
Tue, Nov 15, 3:30 PM · Lister
KShivendu added a comment to T4687: Hex.pm lister (Erlang package manager).

I have a question: The .tar files obtained from this have a contents.tar.gz which contains the real source code. Is there a way to extract this internal .tar file using the current tar loader?

Tue, Nov 15, 3:21 PM · Lister
KShivendu updated the task description for T4687: Hex.pm lister (Erlang package manager).
Tue, Nov 15, 3:19 PM · Lister
KShivendu created T4687: Hex.pm lister (Erlang package manager).
Tue, Nov 15, 3:17 PM · Lister
KShivendu closed D8386: feat(fedora): Introduce fedora lister.
Tue, Nov 15, 11:31 AM
KShivendu committed rDLS6ad61aec2394: feat(fedora): Introduce fedora lister (authored by KShivendu).
feat(fedora): Introduce fedora lister
Tue, Nov 15, 11:31 AM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • Rebase against master
Tue, Nov 15, 11:24 AM
KShivendu created T4686: Duplicated branch names are rendered by the webapp.
Tue, Nov 15, 11:21 AM · Web app
KShivendu updated the diff for D8753: feat: Introduce RPM loader.

Minor fixes in the loader docstrings

Tue, Nov 15, 9:22 AM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • Add tests for handling of HTTP errors and sha1 checksum (increase test coverage)
Tue, Nov 15, 9:10 AM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • Extract .tar.gz as a seperate branch (and other suggestions made by @anlambert)
  • Remove .tar.gz extraction logic from extract_rpm_package func
  • Updating relevant tests for the same
Tue, Nov 15, 8:00 AM

Fri, Nov 11

KShivendu added a comment to D8386: feat(fedora): Introduce fedora lister.

I noticed that https://archive.softwareheritage.org/browse/origin/directory/?origin_url=deb://Ubuntu/packages/nginx has duplicate branch names, which is very confusing. In fact, even the default branch is repeated twice and I see two check marks. If we use branch names like 0.3.9-15.fc26, won't the same happen with Fedora listers? It doesn't seem to differentiate between the editions. (or does it?)

Fri, Nov 11, 8:29 AM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • Send origins in the last iteration
  • Change origin url format
  • Use intrinsic version for pkg version key
Fri, Nov 11, 7:04 AM

Thu, Nov 10

KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
Thu, Nov 10, 6:25 AM

Tue, Nov 8

KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • remove unused code
Tue, Nov 8, 1:37 PM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • remove ununsed code
  • rebase with master
Tue, Nov 8, 1:35 PM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.

feat: Remove microseconds from buildTime metadata to match real values from fedora lister

Tue, Nov 8, 1:08 PM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.

feat: Parse buildTime as UTC and store in ISO format

Tue, Nov 8, 12:43 PM

Sun, Nov 6

KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • Use list comprehension for side_effect
  • Trigger rebuild
Sun, Nov 6, 8:07 AM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.

feat: Add edition and build time in package metadata

Sun, Nov 6, 7:10 AM

Sat, Nov 5

KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • Squash commits
  • Improve comments, fix vars, and add docs
Sat, Nov 5, 8:27 PM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • feat: Make the lister incremental and use build_time as release date
  • tests: Refactor to use fewer lines of code and improve readability
Sat, Nov 5, 8:06 PM

Thu, Nov 3

KShivendu added inline comments to D8753: feat: Introduce RPM loader.
Thu, Nov 3, 6:43 AM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • feat: Use subprocess.check_call and extract .tar obtained from .rpm
  • Remove extra TODOs and FIXMEs
  • Add missing copyrights
Thu, Nov 3, 6:38 AM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • feat: Use subprocess.check_call and extract .tar obtained from .rpm
Thu, Nov 3, 6:28 AM

Oct 27 2022

KShivendu requested review of D8753: feat: Introduce RPM loader.
Oct 27 2022, 3:32 PM
KShivendu closed D8784: base/buster: add rpm2cpio dependency.
Oct 27 2022, 12:12 PM
KShivendu committed rCDFJ2b7e64f0ff39: base/buster: add rpm2cpio dependency (authored by KShivendu).
base/buster: add rpm2cpio dependency
Oct 27 2022, 12:12 PM

Oct 26 2022

KShivendu requested review of D8784: base/buster: add rpm2cpio dependency.
Oct 26 2022, 6:34 PM

Oct 23 2022

KShivendu added a revision to T4648: Introduce RPM Loader: D8753: feat: Introduce RPM loader.
Oct 23 2022, 8:23 AM · RPM loader

Oct 21 2022

KShivendu updated subscribers of T4648: Introduce RPM Loader.
Oct 21 2022, 9:40 AM · RPM loader
KShivendu triaged T4648: Introduce RPM Loader as Normal priority.
Oct 21 2022, 9:40 AM · RPM loader

Oct 13 2022

KShivendu added a comment to T4627: Specify the query language in PEG.

P1090 should be helpful

Oct 13 2022, 10:48 AM · Archive search
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • refactor(fedora): Improve helper functions in test_lister.py
  • squash commits
Oct 13 2022, 8:00 AM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.

tests: Mock urllib.request.urlopen instead of repomd.load function

Oct 13 2022, 7:33 AM

Oct 12 2022

KShivendu added inline comments to D8386: feat(fedora): Introduce fedora lister.
Oct 12 2022, 10:42 AM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • feat: Add release field in extra_loader_args and hardcode test cases
  • rebase against master
Oct 12 2022, 8:28 AM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • feat: Add release field in extra_loader_args and hardcode test cases
Oct 12 2022, 8:17 AM

Oct 7 2022

KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.

Support listing multiple editions and old releases via archives.fedoraprojects.org

  • Add support for listing multiple editions based on release
  • Migrate to archives.fedoraproject.org (contains rpms for old releases)
  • Seperate full and incremental listing tests
Oct 7 2022, 7:48 AM

Oct 5 2022

KShivendu added a comment to T4251: [swh-search] Investigate long search queries response time.

Please keep it open since the search isn't fast enough at the moment.

Oct 5 2022, 10:52 AM · System administration, Archive search

Sep 22 2022

KShivendu added a comment to D8386: feat(fedora): Introduce fedora lister.

Some observations:

Sep 22 2022, 1:57 PM

Sep 20 2022

KShivendu closed D8509: gogs: Skip pages with error 500.
Sep 20 2022, 3:43 PM
KShivendu committed rDLSbd35d54398bf: gogs: Skip pages with error 500 (authored by KShivendu).
gogs: Skip pages with error 500
Sep 20 2022, 3:43 PM
KShivendu updated the diff for D8509: gogs: Skip pages with error 500.
  • gitea: fix test name
  • squash and reword commits
Sep 20 2022, 3:37 PM
KShivendu updated the diff for D8509: gogs: Skip pages with error 500.
  • gitea: skip on status code 500
Sep 20 2022, 2:56 PM
KShivendu added inline comments to D8509: gogs: Skip pages with error 500.
Sep 20 2022, 2:10 PM
KShivendu requested review of D8509: gogs: Skip pages with error 500.
Sep 20 2022, 2:08 PM
KShivendu added a comment to T4478: Deploy Gogs lister to staging.

@ardumont because of T4423 we will face 500 for https://try.gogs.io/api/v1/repos/search?page=17 no matter how many times we retry (until it is fixed by Gogs maintainers). So should we not ignore the page and move on to the next one?

Sep 20 2022, 11:50 AM · Gogs lister, Origin-Gitea/Gogs, System administration

Sep 16 2022

KShivendu added a comment to T4478: Deploy Gogs lister to staging.

thanks. will try to reproduce this locally.

Sep 16 2022, 1:32 PM · Gogs lister, Origin-Gitea/Gogs, System administration
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • test_lister: remove unused code
  • squash commits
Sep 16 2022, 1:31 PM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • remove defusedxml from requirements.txt
  • use docstrings for lister vars
  • directly pass default value of releases
  • fix typo in tasks.py
  • compare scheduler_origins with pkg names
Sep 16 2022, 1:19 PM

Sep 7 2022

KShivendu retitled D8386: feat(fedora): Introduce fedora lister from WIP: feat(fedora): Introduce fedora lister to feat(fedora): Introduce fedora lister.
Sep 7 2022, 4:54 PM
KShivendu updated the summary of D8386: feat(fedora): Introduce fedora lister.
Sep 7 2022, 4:54 PM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • refactor(fedora): Improve vars, update comments, and update tests
Sep 7 2022, 2:28 PM

Sep 6 2022

KShivendu abandoned D5164: Fix snapshot page title.
Sep 6 2022, 8:34 AM

Sep 5 2022

KShivendu requested review of D8386: feat(fedora): Introduce fedora lister.
Sep 5 2022, 8:46 AM

Sep 1 2022

KShivendu added a subtask for T1721: Implementation of Gogs Lister: T4478: Deploy Gogs lister to staging.
Sep 1 2022, 10:03 AM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister
KShivendu added a parent task for T4478: Deploy Gogs lister to staging: T1721: Implementation of Gogs Lister.
Sep 1 2022, 10:03 AM · Gogs lister, Origin-Gitea/Gogs, System administration
KShivendu created T4478: Deploy Gogs lister to staging.
Sep 1 2022, 10:03 AM · Gogs lister, Origin-Gitea/Gogs, System administration

Aug 22 2022

KShivendu accepted D8144: Add NuGet Mapping.
Aug 22 2022, 2:14 PM

Aug 20 2022

KShivendu updated the task description for T4448: Implementation of Fedora Lister.
Aug 20 2022, 10:17 AM · RPM lister, Archive coverage
KShivendu updated the task description for T4448: Implementation of Fedora Lister.
Aug 20 2022, 10:11 AM · RPM lister, Archive coverage
KShivendu triaged T4448: Implementation of Fedora Lister as Normal priority.
Aug 20 2022, 9:58 AM · RPM lister, Archive coverage

Aug 17 2022

KShivendu updated the task description for T1721: Implementation of Gogs Lister.
Aug 17 2022, 11:38 AM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister
KShivendu closed D8218: feat: Make the Gogs lister incremental.
Aug 17 2022, 11:38 AM
KShivendu committed rDLS6a53a6ad06d5: feat: Make the Gogs lister incremental (authored by KShivendu).
feat: Make the Gogs lister incremental
Aug 17 2022, 11:38 AM
KShivendu updated the diff for D8218: feat: Make the Gogs lister incremental.

Squash commits and rebase

Aug 17 2022, 11:32 AM
KShivendu updated the diff for D8218: feat: Make the Gogs lister incremental.
  • Introduce 4th page in incremental listing test
  • Improve comments
Aug 17 2022, 10:08 AM

Aug 16 2022

KShivendu updated the diff for D8218: feat: Make the Gogs lister incremental.
  • Introduce last_seen_repo_id in GogsListerState to properly set the value of self.updated in`finalize()`
  • Update incremental listing test to test happy flow (instead of the buggy T4423)
Aug 16 2022, 5:07 AM

Aug 12 2022

KShivendu updated subscribers of D8218: feat: Make the Gogs lister incremental.
Aug 12 2022, 1:11 PM

Aug 11 2022

KShivendu added inline comments to D8218: feat: Make the Gogs lister incremental.
Aug 11 2022, 2:33 PM
KShivendu added inline comments to D8218: feat: Make the Gogs lister incremental.
Aug 11 2022, 2:19 PM
KShivendu updated the diff for D8218: feat: Make the Gogs lister incremental.

test: fix test failures

  • make last_seen_next_link = None if the last page is encountered
  • in test_gogs_incremental_lister, throw 400 in the 1st listing attempt and succeed in the 2nd one.
Aug 11 2022, 7:28 AM
KShivendu updated the diff for D8218: feat: Make the Gogs lister incremental.

fix: Improvements for edge cases and coverage

  • state.last_seen_next_link should be None if last page is encountered (no next link)
  • assert page.repos is not None in get_origins_from_page
  • Improve comments and formatting
Aug 11 2022, 6:52 AM
KShivendu updated the diff for D8218: feat: Make the Gogs lister incremental.

test: Add test for incremental gogs lister and improve existing tests

  • Add test for incremental listing in gogs
  • Use 3 mocked pages instead of 2 in existing tests
  • In test_gogs_list_http_error, successfully parse 1 page before throwing HTTPError
  • Minor improvement in variables names (P1, P2, P3)
Aug 11 2022, 6:24 AM

Aug 9 2022

KShivendu updated the task description for T1721: Implementation of Gogs Lister.
Aug 9 2022, 8:16 AM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister
KShivendu updated the diff for D8218: feat: Make the Gogs lister incremental.
  • test: Improve variable names and comments in gogs lister tests
Aug 9 2022, 8:07 AM
KShivendu requested review of D8218: feat: Make the Gogs lister incremental.
Aug 9 2022, 7:50 AM
KShivendu added a revision to T1721: Implementation of Gogs Lister: D8218: feat: Make the Gogs lister incremental.
Aug 9 2022, 7:45 AM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister

Aug 4 2022

KShivendu added a comment to T1721: Implementation of Gogs Lister.

worth opening a dedicated forge issue

Done. T4423

Aug 4 2022, 3:03 PM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister
KShivendu added a parent task for T4423: Gogs pagination API breaks because of fatal repos: T1721: Implementation of Gogs Lister.
Aug 4 2022, 2:51 PM · Gogs lister, Origin-Gitea/Gogs, Lister, Archive coverage
KShivendu added a subtask for T1721: Implementation of Gogs Lister: T4423: Gogs pagination API breaks because of fatal repos.
Aug 4 2022, 2:51 PM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister
KShivendu triaged T4423: Gogs pagination API breaks because of fatal repos as Low priority.
Aug 4 2022, 2:51 PM · Gogs lister, Origin-Gitea/Gogs, Lister, Archive coverage
KShivendu added a comment to T1721: Implementation of Gogs Lister.

I just ran the lister using swh lister run -l gogs url=https://try.gogs.io/api/v1/ api_token=xxx

Aug 4 2022, 6:30 AM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister

Aug 3 2022

KShivendu added inline comments to D8144: Add NuGet Mapping.
Aug 3 2022, 1:28 PM
KShivendu closed D8160: gogs: Introduce Gogs lister.
Aug 3 2022, 1:08 PM
KShivendu committed rDLSd34a6232a693: gogs: Introduce Gogs lister (authored by KShivendu).
gogs: Introduce Gogs lister
Aug 3 2022, 1:08 PM
KShivendu updated the diff for D8160: gogs: Introduce Gogs lister.
  • minor update to GogsLister docstring
  • squash commit (again)
Aug 3 2022, 12:53 PM
KShivendu updated the diff for D8160: gogs: Introduce Gogs lister.
  • update GogsLister docstring
  • squash commits
Aug 3 2022, 10:36 AM
KShivendu added a comment to T1721: Implementation of Gogs Lister.

Maybe it'd be worth opening an issue upstream regarding this behavior.

Done https://github.com/gogs/docs-api/issues/34

Aug 3 2022, 6:25 AM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister

Aug 2 2022

KShivendu added inline comments to D8144: Add NuGet Mapping.
Aug 2 2022, 8:41 AM
KShivendu requested changes to D8144: Add NuGet Mapping.
Aug 2 2022, 8:09 AM