Page MenuHomeSoftware Heritage

KShivendu (Kumar Shivendu)
User

Projects

User does not belong to any projects.

User Details

User Since
Feb 23 2021, 11:48 AM (164 w, 4 d)

Recent Activity

Dec 12 2022

KShivendu added a comment to D8907: feat: Add Hex.pm lister.

Created these in Hex.pm official github repo:

Dec 12 2022, 12:01 PM

Dec 6 2022

KShivendu added a comment to D8907: feat: Add Hex.pm lister.

order sounds best. Do you want to do it?

Dec 6 2022, 6:21 PM
KShivendu added a comment to D8907: feat: Add Hex.pm lister.

but if you change it, then pagination is unusable because it's offset-based.

Dec 6 2022, 6:14 PM

Nov 30 2022

KShivendu requested review of D8907: feat: Add Hex.pm lister.
Nov 30 2022, 8:22 PM
KShivendu added a revision to T4687: Hex.pm lister (Erlang package manager): D8907: feat: Add Hex.pm lister.
Nov 30 2022, 8:17 PM · Lister

Nov 16 2022

KShivendu closed D8753: feat: Introduce RPM loader.
Nov 16 2022, 2:19 PM
KShivendu committed rDLDBASEa196c85d5ab1: feat: Incremental RPM loader implementation (authored by KShivendu).
feat: Incremental RPM loader implementation
Nov 16 2022, 2:19 PM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • Fix license headers
  • Fix test cases to match Fedora lister output
Nov 16 2022, 2:09 PM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • Use intrinsic version (and other suggestions by @anlambert)
  • Update tests for the same
Nov 16 2022, 9:51 AM

Nov 15 2022

KShivendu added inline comments to D8753: feat: Introduce RPM loader.
Nov 15 2022, 3:43 PM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • Rebase against master
Nov 15 2022, 3:35 PM
KShivendu updated the task description for T4687: Hex.pm lister (Erlang package manager).
Nov 15 2022, 3:30 PM · Lister
KShivendu added a comment to T4687: Hex.pm lister (Erlang package manager).

I have a question: The .tar files obtained from this have a contents.tar.gz which contains the real source code. Is there a way to extract this internal .tar file using the current tar loader?

Nov 15 2022, 3:21 PM · Lister
KShivendu updated the task description for T4687: Hex.pm lister (Erlang package manager).
Nov 15 2022, 3:19 PM · Lister
KShivendu created T4687: Hex.pm lister (Erlang package manager).
Nov 15 2022, 3:17 PM · Lister
KShivendu closed D8386: feat(fedora): Introduce fedora lister.
Nov 15 2022, 11:31 AM
KShivendu committed rDLS6ad61aec2394: feat(fedora): Introduce fedora lister (authored by KShivendu).
feat(fedora): Introduce fedora lister
Nov 15 2022, 11:31 AM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • Rebase against master
Nov 15 2022, 11:24 AM
KShivendu created T4686: Duplicated branch names are rendered by the webapp.
Nov 15 2022, 11:21 AM · Web app
KShivendu updated the diff for D8753: feat: Introduce RPM loader.

Minor fixes in the loader docstrings

Nov 15 2022, 9:22 AM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • Add tests for handling of HTTP errors and sha1 checksum (increase test coverage)
Nov 15 2022, 9:10 AM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • Extract .tar.gz as a seperate branch (and other suggestions made by @anlambert)
  • Remove .tar.gz extraction logic from extract_rpm_package function. Previously, I was just replacing .tar.gz with its extracted folder but now we are creating a separate branch as well.
  • Updating relevant tests for the same
Nov 15 2022, 8:00 AM

Nov 11 2022

KShivendu added a comment to D8386: feat(fedora): Introduce fedora lister.

I noticed that https://archive.softwareheritage.org/browse/origin/directory/?origin_url=deb://Ubuntu/packages/nginx has duplicate branch names, which is very confusing. In fact, even the default branch is repeated twice and I see two check marks. If we use branch names like 0.3.9-15.fc26, won't the same happen with Fedora listers? It doesn't seem to differentiate between the editions. (or does it?)

Nov 11 2022, 8:29 AM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • Send origins in the last iteration
  • Change origin url format
  • Use intrinsic version for pkg version key
Nov 11 2022, 7:04 AM

Nov 10 2022

KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
Nov 10 2022, 6:25 AM

Nov 8 2022

KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • remove unused code
Nov 8 2022, 1:37 PM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • remove ununsed code
  • rebase with master
Nov 8 2022, 1:35 PM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.

feat: Remove microseconds from buildTime metadata to match real values from fedora lister

Nov 8 2022, 1:08 PM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.

feat: Parse buildTime as UTC and store in ISO format

Nov 8 2022, 12:43 PM

Nov 6 2022

KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • Use list comprehension for side_effect
  • Trigger rebuild
Nov 6 2022, 8:07 AM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.

feat: Add edition and build time in package metadata

Nov 6 2022, 7:10 AM

Nov 5 2022

KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • Squash commits
  • Improve comments, fix vars, and add docs
Nov 5 2022, 8:27 PM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • feat: Make the lister incremental and use build_time as release date
  • tests: Refactor to use fewer lines of code and improve readability
Nov 5 2022, 8:06 PM

Nov 3 2022

KShivendu added inline comments to D8753: feat: Introduce RPM loader.
Nov 3 2022, 6:43 AM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • feat: Use subprocess.check_call and extract .tar obtained from .rpm
  • Remove extra TODOs and FIXMEs
  • Add missing copyrights
Nov 3 2022, 6:38 AM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • feat: Use subprocess.check_call and extract .tar obtained from .rpm
Nov 3 2022, 6:28 AM

Oct 27 2022

KShivendu requested review of D8753: feat: Introduce RPM loader.
Oct 27 2022, 3:32 PM
KShivendu closed D8784: base/buster: add rpm2cpio dependency.
Oct 27 2022, 12:12 PM
KShivendu committed rCDFJ2b7e64f0ff39: base/buster: add rpm2cpio dependency (authored by KShivendu).
base/buster: add rpm2cpio dependency
Oct 27 2022, 12:12 PM

Oct 26 2022

KShivendu requested review of D8784: base/buster: add rpm2cpio dependency.
Oct 26 2022, 6:34 PM

Oct 23 2022

KShivendu added a revision to T4648: Introduce RPM Loader: D8753: feat: Introduce RPM loader.
Oct 23 2022, 8:23 AM · RPM loader

Oct 21 2022

KShivendu updated subscribers of T4648: Introduce RPM Loader.
Oct 21 2022, 9:40 AM · RPM loader
KShivendu triaged T4648: Introduce RPM Loader as Normal priority.
Oct 21 2022, 9:40 AM · RPM loader

Oct 13 2022

KShivendu added a comment to T4627: Specify the query language in PEG.

P1090 should be helpful

Oct 13 2022, 10:48 AM · Archive search
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • refactor(fedora): Improve helper functions in test_lister.py
  • squash commits
Oct 13 2022, 8:00 AM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.

tests: Mock urllib.request.urlopen instead of repomd.load function

Oct 13 2022, 7:33 AM

Oct 12 2022

KShivendu added inline comments to D8386: feat(fedora): Introduce fedora lister.
Oct 12 2022, 10:42 AM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • feat: Add release field in extra_loader_args and hardcode test cases
  • rebase against master
Oct 12 2022, 8:28 AM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • feat: Add release field in extra_loader_args and hardcode test cases
Oct 12 2022, 8:17 AM

Oct 7 2022

KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.

Support listing multiple editions and old releases via archives.fedoraprojects.org

  • Add support for listing multiple editions based on release
  • Migrate to archives.fedoraproject.org (contains rpms for old releases)
  • Seperate full and incremental listing tests
Oct 7 2022, 7:48 AM

Oct 5 2022

KShivendu added a comment to T4251: [swh-search] Investigate long search queries response time.

Please keep it open since the search isn't fast enough at the moment.

Oct 5 2022, 10:52 AM · System administration, Archive search

Sep 22 2022

KShivendu added a comment to D8386: feat(fedora): Introduce fedora lister.

Some observations:

Sep 22 2022, 1:57 PM

Sep 20 2022

KShivendu closed D8509: gogs: Skip pages with error 500.
Sep 20 2022, 3:43 PM
KShivendu committed rDLSbd35d54398bf: gogs: Skip pages with error 500 (authored by KShivendu).
gogs: Skip pages with error 500
Sep 20 2022, 3:43 PM
KShivendu updated the diff for D8509: gogs: Skip pages with error 500.
  • gitea: fix test name
  • squash and reword commits
Sep 20 2022, 3:37 PM
KShivendu updated the diff for D8509: gogs: Skip pages with error 500.
  • gitea: skip on status code 500
Sep 20 2022, 2:56 PM
KShivendu added inline comments to D8509: gogs: Skip pages with error 500.
Sep 20 2022, 2:10 PM
KShivendu requested review of D8509: gogs: Skip pages with error 500.
Sep 20 2022, 2:08 PM
KShivendu added a comment to T4478: Deploy Gogs lister to staging.

@ardumont because of T4423 we will face 500 for https://try.gogs.io/api/v1/repos/search?page=17 no matter how many times we retry (until it is fixed by Gogs maintainers). So should we not ignore the page and move on to the next one?

Sep 20 2022, 11:50 AM · Gogs lister, Origin-Gitea/Gogs, System administration

Sep 16 2022

KShivendu added a comment to T4478: Deploy Gogs lister to staging.

thanks. will try to reproduce this locally.

Sep 16 2022, 1:32 PM · Gogs lister, Origin-Gitea/Gogs, System administration
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • test_lister: remove unused code
  • squash commits
Sep 16 2022, 1:31 PM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • remove defusedxml from requirements.txt
  • use docstrings for lister vars
  • directly pass default value of releases
  • fix typo in tasks.py
  • compare scheduler_origins with pkg names
Sep 16 2022, 1:19 PM

Sep 7 2022

KShivendu retitled D8386: feat(fedora): Introduce fedora lister from WIP: feat(fedora): Introduce fedora lister to feat(fedora): Introduce fedora lister.
Sep 7 2022, 4:54 PM
KShivendu updated the summary of D8386: feat(fedora): Introduce fedora lister.
Sep 7 2022, 4:54 PM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • refactor(fedora): Improve vars, update comments, and update tests
Sep 7 2022, 2:28 PM

Sep 6 2022

KShivendu abandoned D5164: Fix snapshot page title.
Sep 6 2022, 8:34 AM

Sep 5 2022

KShivendu requested review of D8386: feat(fedora): Introduce fedora lister.
Sep 5 2022, 8:46 AM

Sep 1 2022

KShivendu added a subtask for T1721: Implementation of Gogs Lister: T4478: Deploy Gogs lister to staging.
Sep 1 2022, 10:03 AM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister
KShivendu added a parent task for T4478: Deploy Gogs lister to staging: T1721: Implementation of Gogs Lister.
Sep 1 2022, 10:03 AM · Gogs lister, Origin-Gitea/Gogs, System administration
KShivendu created T4478: Deploy Gogs lister to staging.
Sep 1 2022, 10:03 AM · Gogs lister, Origin-Gitea/Gogs, System administration

Aug 22 2022

KShivendu accepted D8144: Add NuGet Mapping.
Aug 22 2022, 2:14 PM

Aug 20 2022

KShivendu updated the task description for T4448: Implementation of Fedora Lister.
Aug 20 2022, 10:17 AM · RPM lister, Archive coverage
KShivendu updated the task description for T4448: Implementation of Fedora Lister.
Aug 20 2022, 10:11 AM · RPM lister, Archive coverage
KShivendu triaged T4448: Implementation of Fedora Lister as Normal priority.
Aug 20 2022, 9:58 AM · RPM lister, Archive coverage

Aug 17 2022

KShivendu updated the task description for T1721: Implementation of Gogs Lister.
Aug 17 2022, 11:38 AM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister
KShivendu closed D8218: feat: Make the Gogs lister incremental.
Aug 17 2022, 11:38 AM
KShivendu committed rDLS6a53a6ad06d5: feat: Make the Gogs lister incremental (authored by KShivendu).
feat: Make the Gogs lister incremental
Aug 17 2022, 11:38 AM
KShivendu updated the diff for D8218: feat: Make the Gogs lister incremental.

Squash commits and rebase

Aug 17 2022, 11:32 AM
KShivendu updated the diff for D8218: feat: Make the Gogs lister incremental.
  • Introduce 4th page in incremental listing test
  • Improve comments
Aug 17 2022, 10:08 AM

Aug 16 2022

KShivendu updated the diff for D8218: feat: Make the Gogs lister incremental.
  • Introduce last_seen_repo_id in GogsListerState to properly set the value of self.updated in`finalize()`
  • Update incremental listing test to test happy flow (instead of the buggy T4423)
Aug 16 2022, 5:07 AM

Aug 12 2022

KShivendu updated subscribers of D8218: feat: Make the Gogs lister incremental.
Aug 12 2022, 1:11 PM

Aug 11 2022

KShivendu added inline comments to D8218: feat: Make the Gogs lister incremental.
Aug 11 2022, 2:33 PM
KShivendu added inline comments to D8218: feat: Make the Gogs lister incremental.
Aug 11 2022, 2:19 PM
KShivendu updated the diff for D8218: feat: Make the Gogs lister incremental.

test: fix test failures

  • make last_seen_next_link = None if the last page is encountered
  • in test_gogs_incremental_lister, throw 400 in the 1st listing attempt and succeed in the 2nd one.
Aug 11 2022, 7:28 AM
KShivendu updated the diff for D8218: feat: Make the Gogs lister incremental.

fix: Improvements for edge cases and coverage

  • state.last_seen_next_link should be None if last page is encountered (no next link)
  • assert page.repos is not None in get_origins_from_page
  • Improve comments and formatting
Aug 11 2022, 6:52 AM
KShivendu updated the diff for D8218: feat: Make the Gogs lister incremental.

test: Add test for incremental gogs lister and improve existing tests

  • Add test for incremental listing in gogs
  • Use 3 mocked pages instead of 2 in existing tests
  • In test_gogs_list_http_error, successfully parse 1 page before throwing HTTPError
  • Minor improvement in variables names (P1, P2, P3)
Aug 11 2022, 6:24 AM

Aug 9 2022

KShivendu updated the task description for T1721: Implementation of Gogs Lister.
Aug 9 2022, 8:16 AM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister
KShivendu updated the diff for D8218: feat: Make the Gogs lister incremental.
  • test: Improve variable names and comments in gogs lister tests
Aug 9 2022, 8:07 AM
KShivendu requested review of D8218: feat: Make the Gogs lister incremental.
Aug 9 2022, 7:50 AM
KShivendu added a revision to T1721: Implementation of Gogs Lister: D8218: feat: Make the Gogs lister incremental.
Aug 9 2022, 7:45 AM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister

Aug 4 2022

KShivendu added a comment to T1721: Implementation of Gogs Lister.

worth opening a dedicated forge issue

Done. T4423

Aug 4 2022, 3:03 PM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister
KShivendu added a parent task for T4423: Gogs pagination API breaks because of fatal repos: T1721: Implementation of Gogs Lister.
Aug 4 2022, 2:51 PM · Gogs lister, Origin-Gitea/Gogs, Lister, Archive coverage
KShivendu added a subtask for T1721: Implementation of Gogs Lister: T4423: Gogs pagination API breaks because of fatal repos.
Aug 4 2022, 2:51 PM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister
KShivendu triaged T4423: Gogs pagination API breaks because of fatal repos as Low priority.
Aug 4 2022, 2:51 PM · Gogs lister, Origin-Gitea/Gogs, Lister, Archive coverage
KShivendu added a comment to T1721: Implementation of Gogs Lister.

I just ran the lister using swh lister run -l gogs url=https://try.gogs.io/api/v1/ api_token=xxx

Aug 4 2022, 6:30 AM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister

Aug 3 2022

KShivendu added inline comments to D8144: Add NuGet Mapping.
Aug 3 2022, 1:28 PM
KShivendu closed D8160: gogs: Introduce Gogs lister.
Aug 3 2022, 1:08 PM
KShivendu committed rDLSd34a6232a693: gogs: Introduce Gogs lister (authored by KShivendu).
gogs: Introduce Gogs lister
Aug 3 2022, 1:08 PM
KShivendu updated the diff for D8160: gogs: Introduce Gogs lister.
  • minor update to GogsLister docstring
  • squash commit (again)
Aug 3 2022, 12:53 PM
KShivendu updated the diff for D8160: gogs: Introduce Gogs lister.
  • update GogsLister docstring
  • squash commits
Aug 3 2022, 10:36 AM