Page MenuHomeSoftware Heritage
Feed All Stories

Jan 28 2021

vsellier added a revision to T3001: Webapp is not displaying the origin type on the search results: D4960: Correctly return origin_visit_status.type value everywhere.
Jan 28 2021, 12:12 PM · Storage manager, Web app
vsellier added projects to T3001: Webapp is not displaying the origin type on the search results: Web app, Storage manager.
Jan 28 2021, 12:11 PM · Storage manager, Web app
vsellier changed the status of T3001: Webapp is not displaying the origin type on the search results from Open to Work in Progress.
Jan 28 2021, 12:11 PM · Storage manager, Web app
ardumont updated the task description for T2998: Deploy lister next-gen in staging.
Jan 28 2021, 12:02 PM · System administration, Lister
ardumont added a comment to T2998: Deploy lister next-gen in staging.

cli run for the bitbucket lister, status ok:

Jan 28 2021, 12:02 PM · System administration, Lister
ardumont added a comment to T2998: Deploy lister next-gen in staging.

cli run for the github instance, status ok:

Jan 28 2021, 11:56 AM · System administration, Lister
ardumont updated the task description for T2998: Deploy lister next-gen in staging.
Jan 28 2021, 11:54 AM · System administration, Lister
Harbormaster failed to build B18836: rDTSCN5e01c09af4c6: retry request in case of backend failure for rDTSCN5e01c09af4c6: retry request in case of backend failure!
Jan 28 2021, 11:49 AM
Harbormaster failed to build B18835: rDTSCNebad16c02da6: algo_min: remove the current node as well for rDTSCNebad16c02da6: algo_min: remove the current node as well!
Jan 28 2021, 11:46 AM
DanSeraf committed rDTSCN5e01c09af4c6: retry request in case of backend failure (authored by DanSeraf).
retry request in case of backend failure
Jan 28 2021, 11:44 AM
DanSeraf closed D4947: scanner-benchmark: algo_min fixed, retry mechanism on request error.
Jan 28 2021, 11:44 AM
DanSeraf committed rDTSCNebad16c02da6: algo_min: remove the current node as well (authored by DanSeraf).
algo_min: remove the current node as well
Jan 28 2021, 11:44 AM
ardumont closed D4957: gitlab: Deal with missing or trailing / in url input.
Jan 28 2021, 11:21 AM
ardumont committed rDLS72be074a795e: gitlab: Deal with missing or trailing / in url input (authored by ardumont).
gitlab: Deal with missing or trailing / in url input
Jan 28 2021, 11:21 AM
swh-public-ci added a comment to D4957: gitlab: Deal with missing or trailing / in url input.

Build is green

Jan 28 2021, 11:16 AM
ardumont updated the diff for D4957: gitlab: Deal with missing or trailing / in url input.

Rebase

Jan 28 2021, 11:12 AM
ardumont closed D4959: tox.ini: Work around build failure due to upstream release.
Jan 28 2021, 11:12 AM
ardumont committed rDLS3bede83d0f51: tox.ini: Work around build failure due to upstream release (authored by ardumont).
tox.ini: Work around build failure due to upstream release
Jan 28 2021, 11:12 AM
ardumont added a comment to D4959: tox.ini: Work around build failure due to upstream release.

yuck

Jan 28 2021, 11:10 AM
olasd accepted D4959: tox.ini: Work around build failure due to upstream release.

yuck

Jan 28 2021, 11:05 AM
zack accepted D4947: scanner-benchmark: algo_min fixed, retry mechanism on request error.
Jan 28 2021, 10:58 AM
ardumont requested review of D4959: tox.ini: Work around build failure due to upstream release.
Jan 28 2021, 10:52 AM
ardumont updated the summary of D4957: gitlab: Deal with missing or trailing / in url input.
Jan 28 2021, 10:50 AM
ardumont added a comment to D4957: gitlab: Deal with missing or trailing / in url input.

Build has FAILED

Jan 28 2021, 10:39 AM
vsellier created P930 (An Untitled Masterwork).
Jan 28 2021, 10:30 AM
ardumont added a project to T3000: Investigate scheduler journal client discrepancies: Scheduling utilities.
Jan 28 2021, 10:29 AM · Scheduling utilities
ardumont updated the task description for T2993: Deploy visit-stats journal client on production.
Jan 28 2021, 10:24 AM · System administration, Scheduling utilities
ardumont triaged T3000: Investigate scheduler journal client discrepancies as Normal priority.
Jan 28 2021, 10:24 AM · Scheduling utilities
ardumont added a comment to T2993: Deploy visit-stats journal client on production.

Status update, the main part of the scheduler journal client is done [1]. 98M origins
referenced in the cache table.

Jan 28 2021, 10:21 AM · System administration, Scheduling utilities
swh-public-ci added a comment to D4957: gitlab: Deal with missing or trailing / in url input.

Build has FAILED

Jan 28 2021, 9:47 AM
Harbormaster failed remote builds in B18830: Diff 17689 for D4957: gitlab: Deal with missing or trailing / in url input!
Jan 28 2021, 9:33 AM
swh-public-ci added a comment to D4957: gitlab: Deal with missing or trailing / in url input.

Build has FAILED

Jan 28 2021, 9:33 AM
ardumont updated the diff for D4957: gitlab: Deal with missing or trailing / in url input.

Adapt according to suggestions

  • make it parametric on both case
  • fix test
Jan 28 2021, 9:31 AM
ardumont closed D4958: cli: Make cli work with new lister....
Jan 28 2021, 9:20 AM
ardumont committed rDLS17b0e7af260c: cli: Make cli work with new lister (authored by ardumont).
cli: Make cli work with new lister
Jan 28 2021, 9:20 AM
swh-public-ci added a comment to D4958: cli: Make cli work with new lister....

Build is green

Jan 28 2021, 9:16 AM
ardumont updated the diff for D4958: cli: Make cli work with new lister....

Rebase

Jan 28 2021, 9:12 AM
ardumont added inline comments to D4957: gitlab: Deal with missing or trailing / in url input.
Jan 28 2021, 9:12 AM

Jan 27 2021

rdicosmo committed rMSLDb1b1173ba3d2: Final version PIDApalooza (authored by rdicosmo).
Final version PIDApalooza
Jan 27 2021, 8:11 PM
Harbormaster failed remote builds in B18827: Diff 17686 for D4958: cli: Make cli work with new lister...!
Jan 27 2021, 7:13 PM
swh-public-ci added a comment to D4958: cli: Make cli work with new lister....

Build has FAILED

Jan 27 2021, 7:13 PM
anlambert added inline comments to D4957: gitlab: Deal with missing or trailing / in url input.
Jan 27 2021, 7:13 PM
anlambert accepted D4957: gitlab: Deal with missing or trailing / in url input.
Jan 27 2021, 7:10 PM
anlambert added inline comments to D4957: gitlab: Deal with missing or trailing / in url input.
Jan 27 2021, 7:10 PM
Harbormaster failed remote builds in B18826: Diff 17685 for D4957: gitlab: Deal with missing or trailing / in url input!
Jan 27 2021, 7:10 PM
swh-public-ci added a comment to D4957: gitlab: Deal with missing or trailing / in url input.

Build has FAILED

Jan 27 2021, 7:09 PM
ardumont updated the diff for D4958: cli: Make cli work with new lister....

(for the test and the soon to be adapted, yes, my thoughts exactly ;)

Jan 27 2021, 7:08 PM
ardumont retitled D4957: gitlab: Deal with missing or trailing / in url input from gitlab: Allow sysadm to forget the trailing / to gitlab: Deal with missing or trailing / in url input.
Jan 27 2021, 7:07 PM
ardumont updated the diff for D4957: gitlab: Deal with missing or trailing / in url input.

Use the right repository...

Jan 27 2021, 7:06 PM
ardumont updated the diff for D4957: gitlab: Deal with missing or trailing / in url input.

Thanks again!

Jan 27 2021, 7:05 PM
ardumont added inline comments to D4957: gitlab: Deal with missing or trailing / in url input.
Jan 27 2021, 7:02 PM
ardumont added a comment to T2998: Deploy lister next-gen in staging.

gitlab instance deployed, status ok:

Jan 27 2021, 7:01 PM · System administration, Lister
ardumont updated the task description for T2998: Deploy lister next-gen in staging.
Jan 27 2021, 6:49 PM · System administration, Lister
anlambert added a comment to D4958: cli: Make cli work with new lister....

Legacy option should be removed soon so no tests needed here ;-)

Jan 27 2021, 6:46 PM
anlambert accepted D4958: cli: Make cli work with new lister....
Jan 27 2021, 6:46 PM
ardumont requested review of D4958: cli: Make cli work with new lister....
Jan 27 2021, 6:44 PM
anlambert added inline comments to D4957: gitlab: Deal with missing or trailing / in url input.
Jan 27 2021, 6:42 PM
swh-public-ci added a comment to D4957: gitlab: Deal with missing or trailing / in url input.

Build is green

Jan 27 2021, 6:41 PM
ardumont updated the diff for D4957: gitlab: Deal with missing or trailing / in url input.

Adapt according to review

Jan 27 2021, 6:38 PM
anlambert requested changes to D4957: gitlab: Deal with missing or trailing / in url input.

This would be cleaner to process the url in the lister constructor instead (same as in debian one).

Jan 27 2021, 6:11 PM
vsellier added a comment to T2920: Document staging infrastructure.

This is a tryout to generate a global schema of the staging environment (P929):

Jan 27 2021, 6:09 PM · Documentation, System administration, Staging environment
vsellier created P929 Staging infrastructure.
Jan 27 2021, 6:07 PM
ardumont requested review of D4957: gitlab: Deal with missing or trailing / in url input.
Jan 27 2021, 6:05 PM
swh-public-ci added a comment to D4936: deposit.cli: Warn users when missing origin tags are detected.

Build is green

Jan 27 2021, 5:51 PM
ardumont updated the task description for T2998: Deploy lister next-gen in staging.
Jan 27 2021, 5:49 PM · System administration, Lister
ardumont updated the task description for T2998: Deploy lister next-gen in staging.
Jan 27 2021, 5:49 PM · System administration, Lister
ardumont updated the diff for D4936: deposit.cli: Warn users when missing origin tags are detected.

Adapt according to review

Jan 27 2021, 5:48 PM
ardumont added inline comments to D4936: deposit.cli: Warn users when missing origin tags are detected.
Jan 27 2021, 5:45 PM
ardumont committed rDLScbd2cce33979: test_cli: Drop launchpad lister from the test_get_lister (authored by ardumont).
test_cli: Drop launchpad lister from the test_get_lister
Jan 27 2021, 5:18 PM
ardumont closed D4956: launchpad: Actually mock the anonymous login to launchpad.
Jan 27 2021, 4:37 PM
ardumont committed rDLSb11b4d100186: launchpad: Actually mock the anonymous login to launchpad (authored by ardumont).
launchpad: Actually mock the anonymous login to launchpad
Jan 27 2021, 4:37 PM
vsellier accepted D4956: launchpad: Actually mock the anonymous login to launchpad.

It seems to be ok :)

Jan 27 2021, 4:32 PM
ardumont updated the task description for T2998: Deploy lister next-gen in staging.
Jan 27 2021, 4:27 PM · System administration, Lister
ardumont moved T2979: Port debian lister to the new Lister API from code review to done on the Sprint 2021 01 board.
Jan 27 2021, 4:25 PM · Lister, Sprint 2021 01
ardumont updated the summary of D4956: launchpad: Actually mock the anonymous login to launchpad.
Jan 27 2021, 4:22 PM
swh-public-ci added a comment to D4956: launchpad: Actually mock the anonymous login to launchpad.

Build is green

Jan 27 2021, 4:19 PM
ardumont updated the diff for D4956: launchpad: Actually mock the anonymous login to launchpad.

Drop unneeded requests_mock (at some point i wanted to use it)

Jan 27 2021, 4:16 PM
ardumont requested review of D4956: launchpad: Actually mock the anonymous login to launchpad.
Jan 27 2021, 4:15 PM
anlambert changed the status of T2992: Port launchpad lister to the new Lister API from Open to Work in Progress.
Jan 27 2021, 4:05 PM · Lister
anlambert changed the status of T2992: Port launchpad lister to the new Lister API, a subtask of T2442: Provide a unified API for listers to interact with the scheduler, from Open to Work in Progress.
Jan 27 2021, 4:05 PM · Sprint 2021 01, Scheduling utilities
anlambert added a comment to D4955: Drop no longer swh.lister.core.{indexing,page_by_page}_lister.

@anlambert, thanks, jsyk, i'm working on fixing failing test in the chroot debian in the end goal to deploy the listers in staging.

Jan 27 2021, 4:03 PM
ardumont added a comment to D4955: Drop no longer swh.lister.core.{indexing,page_by_page}_lister.

@anlambert, thanks, jsyk, i'm working on fixing failing test in the chroot debian in the end goal to deploy the listers in staging.

Jan 27 2021, 3:57 PM
ardumont closed D4955: Drop no longer swh.lister.core.{indexing,page_by_page}_lister.
Jan 27 2021, 3:56 PM
ardumont committed rDLS461bf0997317: Drop no longer swh.lister.core.{indexing,page_by_page}_lister (authored by ardumont).
Drop no longer swh.lister.core.{indexing,page_by_page}_lister
Jan 27 2021, 3:56 PM
douardda requested changes to D4931: Add mapping of definitions and harvests.

few requests, please:

  • add tests with this commit; every introduced function should have at least one test.
  • add doctrings to your new functions,
  • improve the commit message (see https://chris.beams.io/posts/git-commit/ ); with the current one, I have no idea what exactly is done in this commit, and more importantly, why this is needed for.
Jan 27 2021, 3:54 PM
anlambert accepted D4955: Drop no longer swh.lister.core.{indexing,page_by_page}_lister.
Jan 27 2021, 3:54 PM
ardumont requested review of D4955: Drop no longer swh.lister.core.{indexing,page_by_page}_lister.
Jan 27 2021, 3:50 PM
douardda accepted D4914: simulator: stop using the database as a cache for origin data.
Jan 27 2021, 3:48 PM
douardda closed T2970: Make swh-journal tests not depend on swh-model any more as Resolved.

Let's consider it as done.

Jan 27 2021, 3:47 PM · Journal
ardumont committed rDLSe09ad272d767: tests: Drop unneeded reset instruction (authored by ardumont).
tests: Drop unneeded reset instruction
Jan 27 2021, 3:45 PM
douardda closed D4951: Remove tests' journal_data.py in favor of the version in swh-model.
Jan 27 2021, 3:38 PM
douardda committed rDJNL9703864ef366: Remove tests' journal_data.py in favor of the version in swh-model (authored by douardda).
Remove tests' journal_data.py in favor of the version in swh-model
Jan 27 2021, 3:38 PM
vsellier committed rDSNIP0fe3238bdabf: counters: batch redis calls (authored by vsellier).
counters: batch redis calls
Jan 27 2021, 3:38 PM
vsellier committed rDSNIPe1076146c645: counters: add local counter to follow the message count (authored by vsellier).
counters: add local counter to follow the message count
Jan 27 2021, 3:38 PM
vsellier closed D4954: cgit: Don't stop the listing when a repository page is not available.
Jan 27 2021, 3:06 PM
vsellier committed rDLSf6f9f1ca28a9: cgit: Don't stop the listing when a repository page is not available (authored by vsellier).
cgit: Don't stop the listing when a repository page is not available
Jan 27 2021, 3:06 PM
vsellier added a comment to D4954: cgit: Don't stop the listing when a repository page is not available.

Thanks :)

Jan 27 2021, 3:05 PM
anlambert accepted D4954: cgit: Don't stop the listing when a repository page is not available.

Looks good !

Jan 27 2021, 3:03 PM
swh-public-ci added a comment to D4954: cgit: Don't stop the listing when a repository page is not available.

Build is green

Jan 27 2021, 2:57 PM
anlambert added a comment to D4954: cgit: Don't stop the listing when a repository page is not available.

Less code to modify and easier to read and understand the intended behavior from my point of view.

maybe so.

One pro I see for your implementation is if at some point the main loop (get_pages)
fails to fetch something (which we don't currently expect to see the assert added
here), we'll have more information available in the log instead of an AssertionError.
We'll indeed have an actual HttpError with the status code and everything in the
stacktrace.

All in all, i think you are right ;)

Jan 27 2021, 2:54 PM