Page MenuHomeSoftware Heritage
Feed All Stories

Nov 30 2022

KShivendu requested review of D8907: feat: Add Hex.pm lister.
Nov 30 2022, 8:22 PM
KShivendu added a revision to T4687: Hex.pm lister (Erlang package manager): D8907: feat: Add Hex.pm lister.
Nov 30 2022, 8:17 PM · Lister
anlambert closed D8902: save_code_now: Indicate if request was created from webhook in Web UI.
Nov 30 2022, 4:22 PM
anlambert committed rDWAPPSdd3936616b66: package.json: Upgrade dependencies (authored by anlambert).
package.json: Upgrade dependencies
Nov 30 2022, 4:22 PM
anlambert committed rDWAPPSf392e87cf449: save_code_now: Indicate if request was created from webhook in Web UI (authored by anlambert).
save_code_now: Indicate if request was created from webhook in Web UI
Nov 30 2022, 4:22 PM
anlambert closed D8890: metrics: Add gauge counting save origin requests created from webhooks.
Nov 30 2022, 4:22 PM
anlambert committed rDWAPPSce8da30b9842: metrics/tests/test_app: Fix test name (authored by anlambert).
metrics/tests/test_app: Fix test name
Nov 30 2022, 4:22 PM
anlambert committed rDWAPPSb4fc49b3fe92: metrics: Add gauge counting save origin requests created from webhooks (authored by anlambert).
metrics: Add gauge counting save origin requests created from webhooks
Nov 30 2022, 4:22 PM
anlambert closed D8889: save_code_now: Add from_webhook field to SaveOriginRequest model.
Nov 30 2022, 4:22 PM
anlambert committed rDWAPPS12a291957fa0: save_code_now: Add webhook info fields to SaveOriginRequest model (authored by anlambert).
save_code_now: Add webhook info fields to SaveOriginRequest model
Nov 30 2022, 4:22 PM
swh-public-ci added a comment to D8902: save_code_now: Indicate if request was created from webhook in Web UI.

Build is green

Nov 30 2022, 3:55 PM
swh-public-ci added a comment to D8890: metrics: Add gauge counting save origin requests created from webhooks.

Build is green

Nov 30 2022, 3:55 PM
swh-public-ci added a comment to D8889: save_code_now: Add from_webhook field to SaveOriginRequest model.

Build is green

Nov 30 2022, 3:55 PM
anlambert updated the diff for D8902: save_code_now: Indicate if request was created from webhook in Web UI.

Rebase

Nov 30 2022, 3:40 PM
anlambert updated the diff for D8890: metrics: Add gauge counting save origin requests created from webhooks.

Rebase

Nov 30 2022, 3:40 PM
anlambert updated the diff for D8889: save_code_now: Add from_webhook field to SaveOriginRequest model.

Fix docstring and doc build.

Nov 30 2022, 3:40 PM
anlambert committed rDDOC8f0abba63b56: Makefile.sphinx: Restore custom sphinx options coming from tox files (authored by anlambert).
Makefile.sphinx: Restore custom sphinx options coming from tox files
Nov 30 2022, 3:37 PM
swh-public-ci added a comment to D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.

Build is green

Nov 30 2022, 3:18 PM
jayeshv retitled D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks from Make numeric cursors a bit more opaque to avoid cursor hacks to [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.
Nov 30 2022, 3:17 PM
jayeshv updated the diff for D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.

Fix for python 3.7

Nov 30 2022, 3:16 PM
olasd added a comment to P1534 Command-Line Input.
\copy (select name, instance_name, visit_type, origins_known, origins_enabled, origins_never_visited, origins_known - origins_never_visited as origins_visited, origins_with_pending_changes from scheduler_metrics inner join listers on id=lister_id order by 1,2) to stdout with (format csv, header);
Nov 30 2022, 3:06 PM
olasd edited P1534 Command-Line Input.
Nov 30 2022, 3:05 PM
zack renamed T4711: publications page: add 2022 scientific papers from publications page: add recent scientific papers to publications page: add 2022 scientific papers.
Nov 30 2022, 3:04 PM · Website
zack triaged T4711: publications page: add 2022 scientific papers as Normal priority.
Nov 30 2022, 3:04 PM · Website
olasd created P1534 Command-Line Input.
Nov 30 2022, 3:03 PM
olasd created P1533 Command-Line Input.
Nov 30 2022, 3:01 PM
jayeshv updated subscribers of D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.

Thanks, just fixed it :)

You can't use .removeprefix and .removesuffix because we still need to support Python 3.7. Also, the fix is not enough, because you need to change the encoding scheme, as it is still not injective:

>>> get_decoded_cursor(get_encoded_cursor("foo"))
'foo'
>>> get_decoded_cursor(get_encoded_cursor("TRKJJeGRfooJJEHDNOHT"))
'foo'

This fix looks somewhat like a hack to me. Do you think we should use some other method (maybe encrypting the cursor) to fix this issue?

Is it really needed, though? If yes, encrypting looks overkill. If you just want to make it hard to work with, append the (b64-encoded) output of zlib.crc32() or something

Nov 30 2022, 2:55 PM
vlorentz added a comment to D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.

Thanks, just fixed it :)

Nov 30 2022, 2:53 PM
Harbormaster failed remote builds in B33051: Diff 32099 for D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks!
Nov 30 2022, 2:45 PM
swh-public-ci added a comment to D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.

Build has FAILED

Nov 30 2022, 2:45 PM
jayeshv added a comment to D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.

.lstrip() and .rstrip() are not .removeprefix() and .removesuffix():

>>> get_decoded_cursor(get_encoded_cursor("FOO"))
'F'
Nov 30 2022, 2:45 PM
jayeshv updated the diff for D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.

bug fix

Nov 30 2022, 2:43 PM
vlorentz requested changes to D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.

.lstrip() and .rstrip() are not .removeprefix() and .removesuffix():

Nov 30 2022, 2:27 PM
vlorentz created P1532 .csv.zst to .csv.zst .
Nov 30 2022, 1:39 PM
jayeshv retitled D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks from [WIP] Make numeric cursors a bit more opaque to avoid cursor hacks to Make numeric cursors a bit more opaque to avoid cursor hacks.
Nov 30 2022, 12:26 PM
swh-public-ci added a comment to D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.

Build is green

Nov 30 2022, 12:25 PM
jayeshv updated the diff for D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.

typo fix

Nov 30 2022, 12:23 PM
swh-public-ci added a comment to D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.

Build is green

Nov 30 2022, 12:19 PM
jayeshv updated the diff for D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.

remove unused code

Nov 30 2022, 12:17 PM
jayeshv requested review of D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.
Nov 30 2022, 12:15 PM
jayeshv added a revision to T4647: GraphQL: Make cursors truly opaque : D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.
Nov 30 2022, 12:13 PM · GraphQL API
vlorentz closed D8904: metadata_dictionary: Fix 'Invalid IPv6 URL' crash.
Nov 30 2022, 11:46 AM
vlorentz closed D8905: Fix crash when indexing two REMD objects from the same deposit.
Nov 30 2022, 11:46 AM
vlorentz committed rDCIDXf74b47bcf45d: Fix crash when indexing two REMD objects from the same deposit (authored by vlorentz).
Fix crash when indexing two REMD objects from the same deposit
Nov 30 2022, 11:46 AM
vlorentz committed rDCIDXb2d8afff6fa5: metadata_dictionary: Fix 'Invalid IPv6 URL' crash (authored by vlorentz).
metadata_dictionary: Fix 'Invalid IPv6 URL' crash
Nov 30 2022, 11:46 AM
olasd accepted D8904: metadata_dictionary: Fix 'Invalid IPv6 URL' crash.

...

Nov 30 2022, 10:39 AM
olasd accepted D8905: Fix crash when indexing two REMD objects from the same deposit.

Thanks!

Nov 30 2022, 10:38 AM
vlorentz requested review of D8905: Fix crash when indexing two REMD objects from the same deposit.
Nov 30 2022, 10:17 AM
vlorentz added a revision to T4710: DuplicateId errors when indexing from deposits: D8905: Fix crash when indexing two REMD objects from the same deposit.
Nov 30 2022, 10:13 AM · Indexer
vlorentz requested review of D8904: metadata_dictionary: Fix 'Invalid IPv6 URL' crash.
Nov 30 2022, 9:49 AM
vlorentz triaged T4710: DuplicateId errors when indexing from deposits as High priority.
Nov 30 2022, 9:46 AM · Indexer
swh-sentry-integration assigned T4710: DuplicateId errors when indexing from deposits to vlorentz.
Nov 30 2022, 9:45 AM · Indexer
vlorentz added a revision to T4709: npm mapping: ValueError: Invalid IPv6 URL: D8904: metadata_dictionary: Fix 'Invalid IPv6 URL' crash.
Nov 30 2022, 9:45 AM · Indexer
vlorentz triaged T4709: npm mapping: ValueError: Invalid IPv6 URL as Normal priority.
Nov 30 2022, 9:44 AM · Indexer
swh-sentry-integration assigned T4709: npm mapping: ValueError: Invalid IPv6 URL to vlorentz.
Nov 30 2022, 9:44 AM · Indexer
ardumont accepted D8902: save_code_now: Indicate if request was created from webhook in Web UI.
Nov 30 2022, 9:44 AM
ardumont accepted D8889: save_code_now: Add from_webhook field to SaveOriginRequest model.
Nov 30 2022, 9:41 AM

Nov 29 2022

swh-public-ci added a comment to D8883: Add a script to generate a topological sort.

Build is green

Nov 29 2022, 6:32 PM
swh-public-ci added a comment to D8883: Add a script to generate a topological sort.

Build is green

Nov 29 2022, 6:31 PM
swh-public-ci added a comment to D8903: luigi: Add tasks UploadGraphToS3 and DownloadGraphFromS3.

Build is green

Nov 29 2022, 6:29 PM
vlorentz updated the test plan for D8883: Add a script to generate a topological sort.
Nov 29 2022, 6:25 PM
vlorentz updated the test plan for D8883: Add a script to generate a topological sort.
Nov 29 2022, 6:25 PM
vlorentz updated the summary of D8883: Add a script to generate a topological sort.
Nov 29 2022, 6:24 PM
vlorentz updated the diff for D8883: Add a script to generate a topological sort.

fix rebase

Nov 29 2022, 6:23 PM
Harbormaster failed remote builds in B33042: Diff 32090 for D8883: Add a script to generate a topological sort!
Nov 29 2022, 6:22 PM
vlorentz published D8883: Add a script to generate a topological sort for review.
Nov 29 2022, 6:22 PM
vlorentz updated the diff for D8903: luigi: Add tasks UploadGraphToS3 and DownloadGraphFromS3.

fix incorrect decompressed path

Nov 29 2022, 6:21 PM
swh-public-ci added a comment to D8899: README: Update list of indexers.

Build is green

Nov 29 2022, 5:59 PM
vlorentz closed D8899: README: Update list of indexers.
Nov 29 2022, 5:55 PM
vlorentz committed rDCIDXf44e14b11f10: README: Update list of indexers (authored by vlorentz).
README: Update list of indexers
Nov 29 2022, 5:55 PM
vlorentz updated the diff for D8899: README: Update list of indexers.

rebase

Nov 29 2022, 5:55 PM
anlambert requested review of D8902: save_code_now: Indicate if request was created from webhook in Web UI.
Nov 29 2022, 5:43 PM
vlorentz requested review of D8903: luigi: Add tasks UploadGraphToS3 and DownloadGraphFromS3.
Nov 29 2022, 5:15 PM
olasd committed rDCIDXd4d4c59cc3bb: Fix ordering and idempotence in the 136 -> 137 upgrade script (authored by olasd).
Fix ordering and idempotence in the 136 -> 137 upgrade script
Nov 29 2022, 5:11 PM
Harbormaster failed to build B33030: rDGRPHd5122a9b97b3: luigi: Send progress reports to the scheduler for rDGRPHd5122a9b97b3: luigi: Send progress reports to the scheduler!
Nov 29 2022, 5:10 PM
Harbormaster failed to build B33031: rDGRPHec7f568b1305: luigi: Clean up local files before compressing for rDGRPHec7f568b1305: luigi: Clean up local files before compressing!
Nov 29 2022, 5:10 PM
swh-public-ci added a comment to D8890: metrics: Add gauge counting save origin requests created from webhooks.

Build is green

Nov 29 2022, 5:08 PM
anlambert added a revision to T4548: Add a public API endpoint and documentation to trigger Save Code Now from webhook: D8902: save_code_now: Indicate if request was created from webhook in Web UI.
Nov 29 2022, 5:00 PM · Web app
vlorentz closed D8901: luigi: Send progress reports to the scheduler.
Nov 29 2022, 4:56 PM
vlorentz committed rDDATASETa1cf9b871c5f: luigi: Send progress reports to the scheduler (authored by vlorentz).
luigi: Send progress reports to the scheduler
Nov 29 2022, 4:56 PM
vlorentz committed rDGRPHec7f568b1305: luigi: Clean up local files before compressing (authored by vlorentz).
luigi: Clean up local files before compressing
Nov 29 2022, 4:56 PM
vlorentz closed D8900: luigi: Send progress reports to the scheduler.
Nov 29 2022, 4:56 PM
vlorentz committed rDGRPHd5122a9b97b3: luigi: Send progress reports to the scheduler (authored by vlorentz).
luigi: Send progress reports to the scheduler
Nov 29 2022, 4:56 PM
ardumont accepted D8901: luigi: Send progress reports to the scheduler.
Nov 29 2022, 4:53 PM
ardumont accepted D8900: luigi: Send progress reports to the scheduler.
Nov 29 2022, 4:51 PM
ardumont accepted D8899: README: Update list of indexers.
Nov 29 2022, 4:49 PM
vlorentz requested review of D8901: luigi: Send progress reports to the scheduler.
Nov 29 2022, 4:47 PM
swh-public-ci added a comment to D8889: save_code_now: Add from_webhook field to SaveOriginRequest model.

Build is green

Nov 29 2022, 4:46 PM
Harbormaster failed remote builds in B33028: Diff 32078 for D8890: metrics: Add gauge counting save origin requests created from webhooks!
Nov 29 2022, 4:45 PM
swh-public-ci added a comment to D8890: metrics: Add gauge counting save origin requests created from webhooks.

Build has FAILED

Nov 29 2022, 4:45 PM
anlambert updated the diff for D8890: metrics: Add gauge counting save origin requests created from webhooks.

Rebase

Nov 29 2022, 4:31 PM
anlambert updated the diff for D8889: save_code_now: Add from_webhook field to SaveOriginRequest model.

Update apidoc for /origin/save/ endpoint with new webhook related fields

Nov 29 2022, 4:31 PM
olasd committed rDPROVc626cc21b3ea: Add an indexed `revision_date` column on the `content_in_revision` relation (authored by olasd).
Add an indexed `revision_date` column on the `content_in_revision` relation
Nov 29 2022, 4:28 PM
vlorentz requested review of D8900: luigi: Send progress reports to the scheduler.
Nov 29 2022, 4:27 PM
vlorentz requested review of D8899: README: Update list of indexers.
Nov 29 2022, 3:56 PM
swh-public-ci added a comment to D8890: metrics: Add gauge counting save origin requests created from webhooks.

Build is green

Nov 29 2022, 3:55 PM
swh-public-ci added a comment to D8889: save_code_now: Add from_webhook field to SaveOriginRequest model.

Build is green

Nov 29 2022, 3:53 PM
vlorentz closed D8898: Add Luigi to requirements of swh-dataset and swh-graph.
Nov 29 2022, 3:42 PM
vlorentz committed rDDOC144d5c2702b4: Add Luigi to requirements of swh-dataset and swh-graph (authored by vlorentz).
Add Luigi to requirements of swh-dataset and swh-graph
Nov 29 2022, 3:42 PM
ardumont accepted D8898: Add Luigi to requirements of swh-dataset and swh-graph.
Nov 29 2022, 3:39 PM