Page MenuHomeSoftware Heritage

GitHub listerFolder
ActivePublic

Members

  • This project does not have any members.
  • View All

Watchers

  • This project does not have any watchers.
  • View All

Details

Recent Activity

Sun, Jan 8

gitlab-migration closed T1110: document GitHub caseness caveats as Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 10:21 PM · GitHub lister, Documentation
gitlab-migration changed the status of T1740: fetch extrinsic origin metadata from GitHub, a subtask of T833: When listing an origin, add origin level metadata to RMD storage, from Resolved to Migrated.
Sun, Jan 8, 9:59 PM · Extrinsic metadata, Restricted Project, GitHub lister
gitlab-migration changed the status of T1344: Write specs about metadata workflow , a subtask of T833: When listing an origin, add origin level metadata to RMD storage, from Resolved to Migrated.
Sun, Jan 8, 9:58 PM · Extrinsic metadata, Restricted Project, GitHub lister
gitlab-migration changed the status of T1138: Migrate remaining modules to remote scheduler api use from Resolved to Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 9:57 PM · Vault, SWORD deposit, GitHub lister, Scheduling utilities
gitlab-migration closed T2099: GitHub lister: better handling of wrong credentials as Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 5:01 PM · GitHub lister
gitlab-migration changed the status of T2014: Github lister erroneously delete some loading tasks from Resolved to Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 4:28 PM · GitHub lister
gitlab-migration changed the status of T2001: Github incremental lister is broken from Resolved to Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 4:28 PM · GitHub lister
gitlab-migration changed the status of T1625: Repository with trailing slash are archived separately from Wontfix to Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 4:26 PM · GitHub lister
gitlab-migration changed the status of T1590: lister: Update readme instructions to be able to run listers after a git clone from Resolved to Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 4:26 PM · GitHub lister
gitlab-migration changed the status of T833: When listing an origin, add origin level metadata to RMD storage from Wontfix to Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 4:23 PM · Extrinsic metadata, Restricted Project, GitHub lister
gitlab-migration changed the status of T586: Collect task ids and origin ids in the github lister data from Resolved to Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 4:20 PM · GitHub lister
gitlab-migration changed the status of T587: Make GitHub lister use celery tasks instead of the command-line from Resolved to Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 4:20 PM · GitHub lister
gitlab-migration changed the status of T588: Package up the common logic for the GitHub lister in a common class from Resolved to Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 4:20 PM · GitHub lister
gitlab-migration changed the status of T587: Make GitHub lister use celery tasks instead of the command-line, a subtask of T585: Re-engineer github lister, from Resolved to Migrated.
Sun, Jan 8, 4:20 PM · GitHub lister
gitlab-migration changed the status of T586: Collect task ids and origin ids in the github lister data, a subtask of T585: Re-engineer github lister, from Resolved to Migrated.
Sun, Jan 8, 4:20 PM · GitHub lister
gitlab-migration changed the status of T588: Package up the common logic for the GitHub lister in a common class, a subtask of T585: Re-engineer github lister, from Resolved to Migrated.
Sun, Jan 8, 4:20 PM · GitHub lister
gitlab-migration changed the status of T585: Re-engineer github lister from Resolved to Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 4:20 PM · GitHub lister
gitlab-migration changed the status of T313: Retrieve fork information for github repositories in swh.lister.github from Wontfix to Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 4:18 PM · GitHub lister
gitlab-migration changed the status of T164: Port GitHub lister to swh.storage instead of sqlalchemy from Resolved to Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 4:17 PM · GitHub lister
gitlab-migration changed the status of T163: Add entry points to create / request entities in swh.storage, a subtask of T164: Port GitHub lister to swh.storage instead of sqlalchemy, from Resolved to Migrated.
Sun, Jan 8, 4:17 PM · GitHub lister
gitlab-migration changed the status of T30: port ghlister to swh.* namespace and conventions, a subtask of T37: port ghlister to swh task interface - daily updates, from Resolved to Migrated.
Sun, Jan 8, 4:16 PM · GitHub lister
gitlab-migration changed the status of T38: port ghlister to swh task interface - list all / catch up from Resolved to Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 4:16 PM · GitHub lister
gitlab-migration changed the status of T37: port ghlister to swh task interface - daily updates from Resolved to Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 4:16 PM · GitHub lister
gitlab-migration changed the status of T30: port ghlister to swh.* namespace and conventions from Resolved to Migrated.

This task has been migrated to GitLab.

Sun, Jan 8, 4:16 PM · GitHub lister
gitlab-migration changed the status of T30: port ghlister to swh.* namespace and conventions, a subtask of T38: port ghlister to swh task interface - list all / catch up, from Resolved to Migrated.
Sun, Jan 8, 4:16 PM · GitHub lister

Jul 5 2022

vlorentz closed T833: When listing an origin, add origin level metadata to RMD storage as Wontfix.

replaced loader-based metadata loading (T4188 / T4186)

Jul 5 2022, 6:33 PM · Extrinsic metadata, Restricted Project, GitHub lister

May 3 2022

vlorentz closed T1740: fetch extrinsic origin metadata from GitHub, a subtask of T833: When listing an origin, add origin level metadata to RMD storage, as Resolved.
May 3 2022, 11:08 AM · Extrinsic metadata, Restricted Project, GitHub lister

Apr 28 2022

vlorentz edited projects for T3273: Use "fork" relationships to speed-up initial load of large repositories, added: Origin-GitLab; removed GitLab migration.
Apr 28 2022, 3:27 PM · Origin-GitHub, Origin-GitLab, Git loader, Extrinsic metadata, Core Loader
vlorentz added projects to T3273: Use "fork" relationships to speed-up initial load of large repositories: GitHub lister, GitLab migration.
Apr 28 2022, 3:27 PM · Origin-GitHub, Origin-GitLab, Git loader, Extrinsic metadata, Core Loader

Jan 13 2022

vsellier removed a revision from T833: When listing an origin, add origin level metadata to RMD storage: D6946: netbox: use the centralized admin db.
Jan 13 2022, 4:24 PM · Extrinsic metadata, Restricted Project, GitHub lister
vsellier added a revision to T833: When listing an origin, add origin level metadata to RMD storage: D6946: netbox: use the centralized admin db.
Jan 13 2022, 4:21 PM · Extrinsic metadata, Restricted Project, GitHub lister

Oct 21 2021

vlorentz closed T1344: Write specs about metadata workflow , a subtask of T833: When listing an origin, add origin level metadata to RMD storage, as Resolved.
Oct 21 2021, 2:12 PM · Extrinsic metadata, Restricted Project, GitHub lister

Jun 15 2021

anlambert added a revision to T1110: document GitHub caseness caveats: D5877: assets/save: Ensure to use canonical github repo URL as origin URL.
Jun 15 2021, 5:58 PM · GitHub lister, Documentation

Nov 3 2020

ardumont moved T1138: Migrate remaining modules to remote scheduler api use from Backlog to Archived on the SWORD deposit board.
Nov 3 2020, 4:06 PM · Vault, SWORD deposit, GitHub lister, Scheduling utilities

Sep 18 2020

moranegg added a subtask for T833: When listing an origin, add origin level metadata to RMD storage: T1740: fetch extrinsic origin metadata from GitHub.
Sep 18 2020, 2:36 PM · Extrinsic metadata, Restricted Project, GitHub lister
moranegg added a parent task for T833: When listing an origin, add origin level metadata to RMD storage: T2202: Collect extrinsic metadata.
Sep 18 2020, 2:35 PM · Extrinsic metadata, Restricted Project, GitHub lister
moranegg renamed T833: When listing an origin, add origin level metadata to RMD storage from When listing an origin, add origin level metadata to storage to When listing an origin, add origin level metadata to RMD storage.
Sep 18 2020, 2:31 PM · Extrinsic metadata, Restricted Project, GitHub lister
moranegg moved T833: When listing an origin, add origin level metadata to RMD storage from Backlog to Implementation on the Metadata workflow board.
Sep 18 2020, 1:49 PM · Extrinsic metadata, Restricted Project, GitHub lister

Jan 16 2020

ardumont added a comment to P584 github token creation routine.

Tried this in python but for some reason (come to think of it, most probably because of session issue, it did not work...)

Jan 16 2020, 1:34 PM · GitHub lister, Lister
ardumont created P584 github token creation routine.
Jan 16 2020, 1:32 PM · GitHub lister, Lister

Nov 19 2019

olasd triaged T2099: GitHub lister: better handling of wrong credentials as Normal priority.
Nov 19 2019, 4:33 PM · GitHub lister

Sep 19 2019

anlambert closed T2014: Github lister erroneously delete some loading tasks as Resolved by committing rDLS04d8fdf8df45: github/lister: Prevent erroneous scheduler tasks disabling.
Sep 19 2019, 2:36 PM · GitHub lister
anlambert triaged T2014: Github lister erroneously delete some loading tasks as Normal priority.
Sep 19 2019, 1:59 PM · GitHub lister

Sep 18 2019

anlambert closed T2001: Github incremental lister is broken as Resolved by committing rDLS7572228f7c89: listers: Ensure run can be called without bounds arguments.
Sep 18 2019, 11:10 AM · GitHub lister

Sep 17 2019

anlambert triaged T2001: Github incremental lister is broken as High priority.
Sep 17 2019, 11:22 AM · GitHub lister

Aug 27 2019

ardumont closed T1590: lister: Update readme instructions to be able to run listers after a git clone as Resolved.
Aug 27 2019, 12:11 PM · GitHub lister

May 23 2019

vlorentz removed a parent task for T833: When listing an origin, add origin level metadata to RMD storage: T1102: Handle all GitHub elements.
May 23 2019, 12:07 PM · Extrinsic metadata, Restricted Project, GitHub lister
vlorentz added a subtask for T833: When listing an origin, add origin level metadata to RMD storage: T1344: Write specs about metadata workflow .
May 23 2019, 11:58 AM · Extrinsic metadata, Restricted Project, GitHub lister
vlorentz added a comment to T833: When listing an origin, add origin level metadata to RMD storage.

@zack Yes, rate limit. And I determined this based on our current listing rate (@olasd said 2 to 3 days to fully list GitHub with 500 repos per request; so an API call for each repo would take a total of 500 times 2 or 3 days).

May 23 2019, 11:34 AM · Extrinsic metadata, Restricted Project, GitHub lister

May 20 2019

zack added a comment to T833: When listing an origin, add origin level metadata to RMD storage.
  • sending a request for each repository would need ~2 to 3 years for a full pass over github. That's with our current infrastructure, so it's not a hard limit.
May 20 2019, 3:45 PM · Extrinsic metadata, Restricted Project, GitHub lister