Use an exception to validate a repo page can be accessed
- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Feed All Stories
All Stories
All Stories
Jan 27 2021
Jan 27 2021
vsellier updated the diff for D4954: cgit: Don't stop the listing when a repository page is not available.
ardumont added a comment to D4954: cgit: Don't stop the listing when a repository page is not available.
Less code to modify and easier to read and understand the intended behavior from my point of view.
anlambert added a comment to D4954: cgit: Don't stop the listing when a repository page is not available.
In D4954#124697, @ardumont wrote:You should rather catch the requests.exceptions.HTTPError originally raised by _get_and_parse method.
picture me curious, why?
(it's missing the eventual logging.warning instruction but its use is debatable as well ;)
ardumont changed the status of T2998: Deploy lister next-gen in staging from Open to Work in Progress.
ardumont added a comment to D4954: cgit: Don't stop the listing when a repository page is not available.
You should rather catch the requests.exceptions.HTTPError originally raised by _get_and_parse method.
swh-public-ci added a comment to D4954: cgit: Don't stop the listing when a repository page is not available.
Build is green
vsellier committed rDLS91fcde83410d: cgit: Add support for last_update information during listing (authored by vsellier).
cgit: Add support for last_update information during listing
anlambert requested changes to D4954: cgit: Don't stop the listing when a repository page is not available.
You should rather catch the requests.exceptions.HTTPError originally raised by _get_and_parse method.
swh-public-ci added a comment to D4953: cgit: Add support for last_update information during listing.
Build is green
vsellier updated the diff for D4954: cgit: Don't stop the listing when a repository page is not available.
rebase
Restore missing log when the date can't be parsed
swh-public-ci added a comment to D4954: cgit: Don't stop the listing when a repository page is not available.
Build is green
swh-public-ci added a comment to D4953: cgit: Add support for last_update information during listing.
Build is green
vsellier updated the diff for D4954: cgit: Don't stop the listing when a repository page is not available.
rebase
Remove useless variable
Build is green
Add mapping of definitions and harvests
anlambert triaged T2999: Optimize the number of HTTP requests sent by the cgit lister as Normal priority.
Looks good to me !
vsellier requested review of D4954: cgit: Don't stop the listing when a repository page is not available.
vsellier moved T2944: Deploy swh-search v0.4.1 from in-progress to deployed/landed/monitoring on the System administration board.
swh-public-ci added a comment to D4953: cgit: Add support for last_update information during listing.
Build is green
- Reorder methods
- Adapt date parsing according the review
status:
- backfill almost done on getty [1] [2]
Thanks for tackling this !
ardumont moved T2941: hedgedoc: Detect and allow upgrade from Weekly backlog to Backlog on the System administration board.
ardumont moved T2993: Deploy visit-stats journal client on production from in-progress to code-review/await-feedback/pause on the System administration board.
ardumont moved T2993: Deploy visit-stats journal client on production from Weekly backlog to in-progress on the System administration board.
ardumont moved T2998: Deploy lister next-gen in staging from Backlog to Weekly backlog on the System administration board.
lgtm
swh-public-ci added a comment to D4951: Remove tests' journal_data.py in favor of the version in swh-model.
Build is green
douardda updated the diff for D4951: Remove tests' journal_data.py in favor of the version in swh-model.
set the DeprecationWarning category in journal_data
swh-public-ci added a comment to D4951: Remove tests' journal_data.py in favor of the version in swh-model.
Build is green
douardda updated the diff for D4951: Remove tests' journal_data.py in favor of the version in swh-model.
remove stuff added mistakenly, and properly deprecate journal_data instead of breaking swh-storage
Looks good to me.
douardda added inline comments to D4951: Remove tests' journal_data.py in favor of the version in swh-model.
To decrease the time to recover the lag, several journal client were launched in // with :
/usr/bin/swh search --config-file /etc/softwareheritage/search/journal_client_objects.yml journal-client objects
douardda added inline comments to D4951: Remove tests' journal_data.py in favor of the version in swh-model.
douardda added inline comments to D4951: Remove tests' journal_data.py in favor of the version in swh-model.
Allocate more memory to search1
Jan 26 2021
Jan 26 2021
Everything looks fine and visit-stats is growing gently:
ardumont closed T2987: Port gitlab lister to the new `swh.lister.pattern.Lister` API, a subtask of T2442: Provide a unified API for listers to interact with the scheduler, as Resolved.
swh-public-ci added a comment to D4953: cgit: Add support for last_update information during listing.
Build is green
swh-public-ci added a comment to D4953: cgit: Add support for last_update information during listing.
Build is green
Inline unecessary indirection
Build is green
Add missing test coverage
Rename "singleton" to "record"
vsellier changed the status of T2988: Improve cgit lister to add last modification date of the repos from Open to Work in Progress.
ardumont requested changes to D4951: Remove tests' journal_data.py in favor of the version in swh-model.
for the sake of my questions
anlambert committed rDWAPPSab5401d73db1: tests/resources: Fix mypy 0.800 errors (authored by anlambert).
tests/resources: Fix mypy 0.800 errors
(i don't know why or if it's sensible that the decorator got dropped though ;)
douardda requested review of D4951: Remove tests' journal_data.py in favor of the version in swh-model.
Build is green
anlambert closed T2979: Port debian lister to the new Lister API, a subtask of T2442: Provide a unified API for listers to interact with the scheduler, as Resolved.
anlambert closed T2979: Port debian lister to the new Lister API as Resolved by committing rDLSbb0184c004d2: debian: Reimplement lister using new Lister API.
anlambert committed rDLSbb0184c004d2: debian: Reimplement lister using new Lister API (authored by anlambert).
debian: Reimplement lister using new Lister API
Build is green
Rebase
douardda committed rDMODcad940dc8c07: Add swh-journal's model-related test data set in swh-model (authored by douardda).
Add swh-journal's model-related test data set in swh-model
Build is green
tests: Remove no longer used conftest files
Rebase and remove conftest.py file
sgtm
Build is green
ardumont committed rDLS97254a19f226: gitlab: Implement keyset-based pagination listing (authored by ardumont).
gitlab: Implement keyset-based pagination listing
Reword commit
Build is green
Rebase
Looks good but the commit message could be more detailed and rephrased to something like:
Back on this, the plan is now to make swh-journal not depend on the actual model definition, which is currently mostly due to the presence of the journal_data.py in swh-journal. So the plan is to move this file in swh-model so it's kept up to date with swh-model, even if it's mostly used for testing other packages (like swh-journal).
Build is green