Page MenuHomeSoftware Heritage
Feed All Stories

Oct 27 2022

vlorentz added a comment to D8759: model: Add payload to ExtID class.

Looks good, thanks!

Oct 27 2022, 9:44 AM
swh-public-ci added a comment to D8759: model: Add payload to ExtID class.

Build is green

Oct 27 2022, 1:14 AM
samplet updated the diff for D8759: model: Add payload to ExtID class.

This is a duplicate of the last update. I'm hoping that Arcanist
is now happy with my Git remote settings and will be able to push
the changes to the staging repo to be picked up by CI.

Oct 27 2022, 1:12 AM
Harbormaster failed remote builds in B32612: Diff 31658 for D8759: model: Add payload to ExtID class!
Oct 27 2022, 12:48 AM
swh-public-ci added a comment to D8759: model: Add payload to ExtID class.

Build has FAILED

Oct 27 2022, 12:48 AM
samplet updated the diff for D8759: model: Add payload to ExtID class.

Use a Git SHA-1 for the payload.

Oct 27 2022, 12:48 AM

Oct 26 2022

swh-public-ci added a comment to D8785: package.json: Pin cy2 to 2.1.0 to fix Jenkins jobs.

Build is green

Oct 26 2022, 7:16 PM
anlambert closed D8785: package.json: Pin cy2 to 2.1.0 to fix Jenkins jobs.
Oct 26 2022, 7:16 PM
anlambert committed rDWAPPSaee1c7ccb620: package.json: Pin cy2 to 2.1.0 to fix Jenkins jobs (authored by anlambert).
package.json: Pin cy2 to 2.1.0 to fix Jenkins jobs
Oct 26 2022, 7:16 PM
ardumont accepted D8784: base/buster: add rpm2cpio dependency.
Oct 26 2022, 7:11 PM
KShivendu requested review of D8784: base/buster: add rpm2cpio dependency.
Oct 26 2022, 6:34 PM
Harbormaster failed to build B32608: rDWAPPS8db9fd3217a9: package.json: Upgrade dependencies for rDWAPPS8db9fd3217a9: package.json: Upgrade dependencies!
Oct 26 2022, 5:51 PM
ardumont updated the task description for T3781: Replace the Nixguix loader with a lister.
Oct 26 2022, 5:48 PM · Data Model, Nixguix loader
anlambert closed D8781: utils/exc: Ensure exception text is escaped in HTML error page.
Oct 26 2022, 5:39 PM
anlambert committed rDWAPPS8db9fd3217a9: package.json: Upgrade dependencies (authored by anlambert).
package.json: Upgrade dependencies
Oct 26 2022, 5:39 PM
anlambert committed rDWAPPSdf0bb952e96d: utils/exc: Ensure exception text is escaped in HTML error page (authored by anlambert).
utils/exc: Ensure exception text is escaped in HTML error page
Oct 26 2022, 5:39 PM
anlambert closed D8780: utils/exc: Add typing.
Oct 26 2022, 5:39 PM
anlambert committed rDWAPPSd2c653d3e9c3: utils/exc: Add typing (authored by anlambert).
utils/exc: Add typing
Oct 26 2022, 5:39 PM
ardumont closed D8783: lister: Make sure lister that requires github tokens can use it.
Oct 26 2022, 5:38 PM
ardumont committed rDLS92d494261f26: lister: Make sure lister that requires github tokens can use it (authored by ardumont).
lister: Make sure lister that requires github tokens can use it
Oct 26 2022, 5:38 PM
ardumont updated the test plan for D8783: lister: Make sure lister that requires github tokens can use it.
Oct 26 2022, 5:37 PM
anlambert accepted D8783: lister: Make sure lister that requires github tokens can use it.

Better like this indeed, could you add a test with a sample credentials config file to check github_session object has loaded credentials ?

Oct 26 2022, 5:37 PM
ardumont updated the summary of D8783: lister: Make sure lister that requires github tokens can use it.
Oct 26 2022, 5:36 PM
ardumont requested review of D8783: lister: Make sure lister that requires github tokens can use it.
Oct 26 2022, 5:35 PM
anlambert updated the summary of D8782: templates/stage-cypress-tests: Prevent race condition with support file.
Oct 26 2022, 5:24 PM
anlambert updated the summary of D8782: templates/stage-cypress-tests: Prevent race condition with support file.
Oct 26 2022, 5:24 PM
anlambert requested review of D8782: templates/stage-cypress-tests: Prevent race condition with support file.
Oct 26 2022, 5:20 PM
jayeshv accepted D8781: utils/exc: Ensure exception text is escaped in HTML error page.
Oct 26 2022, 5:14 PM
anlambert requested review of D8781: utils/exc: Ensure exception text is escaped in HTML error page.
Oct 26 2022, 5:12 PM
vlorentz added inline comments to D8779: codemeta: Fix malformed dates that used to be allowed by the deposit.
Oct 26 2022, 4:59 PM
vlorentz added inline comments to D8772: metadata_dictionary: Systematically check input URLs before adding to graph.
Oct 26 2022, 4:56 PM
vlorentz added a parent task for T4663: ObjectFormatException: not enough values to unpack (expected 2, got 1): T4659: Fix all crashes of the git loader caused by malformed git objects.
Oct 26 2022, 4:45 PM · Git loader
vlorentz added a subtask for T4659: Fix all crashes of the git loader caused by malformed git objects: T4663: ObjectFormatException: not enough values to unpack (expected 2, got 1).
Oct 26 2022, 4:45 PM · meta-task, Git loader
vlorentz placed T4663: ObjectFormatException: not enough values to unpack (expected 2, got 1) up for grabs.
Oct 26 2022, 4:45 PM · Git loader
swh-sentry-integration assigned T4663: ObjectFormatException: not enough values to unpack (expected 2, got 1) to vlorentz.
Oct 26 2022, 4:45 PM · Git loader
anlambert requested review of D8780: utils/exc: Add typing.
Oct 26 2022, 3:34 PM
anlambert accepted D8779: codemeta: Fix malformed dates that used to be allowed by the deposit.

LGTM

Oct 26 2022, 3:22 PM
anlambert accepted D8778: codemeta: Fix incorrect output namespace for dates and URLs.

LGTM

Oct 26 2022, 3:15 PM
anlambert accepted D8772: metadata_dictionary: Systematically check input URLs before adding to graph.

LGTM, added some nitpicks about typing as inline comments.

Oct 26 2022, 3:14 PM
jayeshv created P1518 getRevisionParents.
Oct 26 2022, 2:58 PM · GraphQL API
jayeshv created P1517 resolveSwhid.
Oct 26 2022, 2:58 PM · GraphQL API
jayeshv created P1516 getDirEntry.
Oct 26 2022, 2:57 PM · GraphQL API
jayeshv created P1515 checkFileExists.
Oct 26 2022, 2:56 PM · GraphQL API
jayeshv created P1514 getOriginVisits.
Oct 26 2022, 2:55 PM · GraphQL API
jayeshv created P1513 getLatestStatus.
Oct 26 2022, 2:54 PM · GraphQL API
vlorentz planned changes to D8151: [RFC] Add 'evolve' method to BaseModel objects.
Oct 26 2022, 2:47 PM
ardumont committed rDSNIPdb50b4777309: nixguix/analyze-result: Make --obj-type a multiple option (authored by ardumont).
nixguix/analyze-result: Make --obj-type a multiple option
Oct 26 2022, 2:45 PM
ardumont added a comment to T3781: Replace the Nixguix loader with a lister.

With latest diffs, the filtering seems to sort properly the files and tarballs for the guix manifest:

Oct 26 2022, 2:44 PM · Data Model, Nixguix loader
ardumont updated the task description for T3781: Replace the Nixguix loader with a lister.
Oct 26 2022, 2:34 PM · Data Model, Nixguix loader
ardumont closed T4662: staging: Deploy nixguix lister and loader, a subtask of T3781: Replace the Nixguix loader with a lister, as Invalid.
Oct 26 2022, 2:33 PM · Data Model, Nixguix loader
ardumont closed T4662: staging: Deploy nixguix lister and loader as Invalid.

/me sighs

Oct 26 2022, 2:33 PM · Data Model, Nixguix loader
ardumont triaged T4662: staging: Deploy nixguix lister and loader as Normal priority.
Oct 26 2022, 2:31 PM · Data Model, Nixguix loader
ardumont updated the task description for T3781: Replace the Nixguix loader with a lister.
Oct 26 2022, 2:29 PM · Data Model, Nixguix loader
ardumont closed D8774: nixguix: Use content-disposition from http head request if provided.
Oct 26 2022, 2:28 PM
ardumont committed rDLS81688ca17e66: nixguix: Use content-disposition from http head request if provided (authored by ardumont).
nixguix: Use content-disposition from http head request if provided
Oct 26 2022, 2:28 PM
vlorentz requested review of D8779: codemeta: Fix malformed dates that used to be allowed by the deposit.
Oct 26 2022, 2:24 PM
vlorentz requested review of D8778: codemeta: Fix incorrect output namespace for dates and URLs.
Oct 26 2022, 2:22 PM
vlorentz added a revision to T4654: swh-indexer produces dates not supported by swh-search/ElasticSearch: D8779: codemeta: Fix malformed dates that used to be allowed by the deposit.
Oct 26 2022, 2:20 PM · Archive search, Indexer
olasd added a comment to D8775: test_converters: Rename 'raw_manifest' to 'raw_string'.

Dulwich calls it "raw_string", so I went with that

Oct 26 2022, 2:16 PM
vlorentz added a revision to T4654: swh-indexer produces dates not supported by swh-search/ElasticSearch: D8778: codemeta: Fix incorrect output namespace for dates and URLs.
Oct 26 2022, 2:15 PM · Archive search, Indexer
vlorentz added a comment to D8775: test_converters: Rename 'raw_manifest' to 'raw_string'.

Dulwich calls it "raw_string", so I went with that

Oct 26 2022, 12:14 PM
vlorentz closed D8776: converters: Replace '/' with '_' in directory entries.
Oct 26 2022, 12:14 PM
vlorentz committed rDLDG44f70a704eee: converters: Replace '/' with '_' in directory entries (authored by vlorentz).
converters: Replace '/' with '_' in directory entries
Oct 26 2022, 12:14 PM
vlorentz closed D8775: test_converters: Rename 'raw_manifest' to 'raw_string'.
Oct 26 2022, 12:13 PM
vlorentz committed rDLDG4e501c7ba4ac: test_converters: Rename 'raw_manifest' to 'raw_string' (authored by vlorentz).
test_converters: Rename 'raw_manifest' to 'raw_string'
Oct 26 2022, 12:13 PM
anlambert added a comment to T4548: Add a public API endpoint and documentation to trigger Save Code Now from webhook.

This is the POST request I received from Bitbucket when pushing commit to a sample repository.

Oct 26 2022, 12:07 PM · Web app
swh-public-ci added a comment to D8774: nixguix: Use content-disposition from http head request if provided.

Build is green

Oct 26 2022, 12:05 PM
anlambert accepted D8774: nixguix: Use content-disposition from http head request if provided.

LGTM

Oct 26 2022, 12:02 PM
ardumont closed D8773: nixguix: Deal with edge case url with version instead of extension.
Oct 26 2022, 12:00 PM
ardumont committed rDLS026fea21da49: nixguix: Deal with edge case url with version instead of extension (authored by ardumont).
nixguix: Deal with edge case url with version instead of extension
Oct 26 2022, 12:00 PM
ardumont added inline comments to D8774: nixguix: Use content-disposition from http head request if provided.
Oct 26 2022, 12:00 PM
ardumont added inline comments to D8774: nixguix: Use content-disposition from http head request if provided.
Oct 26 2022, 11:59 AM
ardumont updated the diff for D8774: nixguix: Use content-disposition from http head request if provided.

Adapt according to last review

Oct 26 2022, 11:59 AM
vlorentz updated the task description for T4661: Some Git loader tasks for gitlab.com origins have "lister_name: github" in their arguments.
Oct 26 2022, 11:55 AM · Lister, Scheduling utilities
vlorentz renamed T4661: Some Git loader tasks for gitlab.com origins have "lister_name: github" in their arguments from InvalidOrigin: netloc should be 'github.com', not 'gitlab.com' to Some Git loader tasks for gitlab.com origins have "lister_name: github" in their arguments.
Oct 26 2022, 11:54 AM · Lister, Scheduling utilities
anlambert added a comment to T4548: Add a public API endpoint and documentation to trigger Save Code Now from webhook.

Regarding JSON payload sent by SourceForge, we cannot guess the visit type (git, svn, or hg) from it so we will have to query the project endpoint from SourceForge REST API to get it.

Oct 26 2022, 11:54 AM · Web app
vlorentz updated the task description for T4661: Some Git loader tasks for gitlab.com origins have "lister_name: github" in their arguments.
Oct 26 2022, 11:54 AM · Lister, Scheduling utilities
vlorentz placed T4661: Some Git loader tasks for gitlab.com origins have "lister_name: github" in their arguments up for grabs.
Oct 26 2022, 11:54 AM · Lister, Scheduling utilities
swh-sentry-integration assigned T4661: Some Git loader tasks for gitlab.com origins have "lister_name: github" in their arguments to vlorentz.
Oct 26 2022, 11:53 AM · Lister, Scheduling utilities
swh-public-ci added a comment to D8773: nixguix: Deal with edge case url with version instead of extension.

Build is green

Oct 26 2022, 11:49 AM
anlambert added a comment to T4548: Add a public API endpoint and documentation to trigger Save Code Now from webhook.

Based on the received HTTP POST request headers, we should be able to determine which type of forge sent us the request and then
extract repository URL and visit type from the JSON payload in order to create a Save Code Now request when new commits are pushed.

We can do that and it certainly works, but once we start allowing a single endpoint for all forges, we cannot go back.

What about a separate path for each forge type, so we have more flexibility later on? eg. to route/"cache" requests from some forges differently directly in Varnish

Oct 26 2022, 11:44 AM · Web app
ardumont updated the diff for D8773: nixguix: Deal with edge case url with version instead of extension.

Rebase (commit id mismatch locally and in the diff, don't recall what changed)
-> must be the commit message.

Oct 26 2022, 11:43 AM
olasd committed rDSNIP8fe0cbf957b5: takedowns: Transparently compute ""origin SWHIDs"" (authored by olasd).
takedowns: Transparently compute ""origin SWHIDs""
Oct 26 2022, 11:41 AM
anlambert added inline comments to D8774: nixguix: Use content-disposition from http head request if provided.
Oct 26 2022, 11:41 AM
anlambert accepted D8773: nixguix: Deal with edge case url with version instead of extension.

I am not a big fan either of adding more tarball detection heuristics but after quickly hacking on the code, it seems this is the only way to handle these edge case URLS (plus there is some cases when even analyzing HTTP headers cannot help to detect if the URL targets a tarball P1512). So let's land this and move on in deploying and testing that lister on staging.

Oct 26 2022, 11:34 AM
anlambert created P1512 (An Untitled Masterwork).
Oct 26 2022, 11:29 AM
vlorentz placed T3880: Support Git commits with no angle brackets in author name up for grabs.
Oct 26 2022, 11:22 AM · Git loader
vlorentz added a revision to T4660: Workaround / in tree entry names: D8776: converters: Replace '/' with '_' in directory entries.
Oct 26 2022, 11:21 AM · Git loader
vlorentz updated the summary of D8776: converters: Replace '/' with '_' in directory entries.
Oct 26 2022, 11:21 AM
vlorentz triaged T4660: Workaround / in tree entry names as Low priority.
Oct 26 2022, 11:21 AM · Git loader
vlorentz added a comment to T4548: Add a public API endpoint and documentation to trigger Save Code Now from webhook.

Based on the received HTTP POST request headers, we should be able to determine which type of forge sent us the request and then
extract repository URL and visit type from the JSON payload in order to create a Save Code Now request when new commits are pushed.

Oct 26 2022, 11:04 AM · Web app
vlorentz added a parent task for T4659: Fix all crashes of the git loader caused by malformed git objects: T3653: Stabilize loader git.
Oct 26 2022, 10:37 AM · meta-task, Git loader
vlorentz added a subtask for T3653: Stabilize loader git: T4659: Fix all crashes of the git loader caused by malformed git objects.
Oct 26 2022, 10:37 AM · Git loader
vlorentz added a parent task for T1339: Handle malformed author and committer dates: T4659: Fix all crashes of the git loader caused by malformed git objects.
Oct 26 2022, 10:33 AM · Storage manager, Git loader
vlorentz added a subtask for T4659: Fix all crashes of the git loader caused by malformed git objects: T1339: Handle malformed author and committer dates.
Oct 26 2022, 10:33 AM · meta-task, Git loader
vlorentz renamed T4659: Fix all crashes of the git loader caused by malformed git objects from Fix all crashes of the git loader to Fix all crashes of the git loader caused by malformed git objects.
Oct 26 2022, 10:32 AM · meta-task, Git loader
vlorentz added a subtask for T4659: Fix all crashes of the git loader caused by malformed git objects: Unknown Object (Maniphest Task).
Oct 26 2022, 10:32 AM · meta-task, Git loader
vlorentz placed T4658: ObjectFormatException: Unknown field b'>' up for grabs.
Oct 26 2022, 10:31 AM · Git loader
vlorentz added a parent task for T4658: ObjectFormatException: Unknown field b'>': T4659: Fix all crashes of the git loader caused by malformed git objects.
Oct 26 2022, 10:31 AM · Git loader
vlorentz added a parent task for T3880: Support Git commits with no angle brackets in author name: T4659: Fix all crashes of the git loader caused by malformed git objects.
Oct 26 2022, 10:31 AM · Git loader