Page MenuHomeSoftware Heritage
Feed Advanced Search

Jun 17 2021

anlambert added inline comments to D5875: Add an endpoint to list and access raw extrinsic metadata..
Jun 17 2021, 11:18 AM

Jun 16 2021

anlambert closed D5882: docker/keycloak: Enable users registration.
Jun 16 2021, 5:52 PM
anlambert committed rDENVbda11aee61b1: docker/keycloak: Enable users registration (authored by anlambert).
docker/keycloak: Enable users registration
Jun 16 2021, 5:52 PM
anlambert requested review of D5882: docker/keycloak: Enable users registration.
Jun 16 2021, 5:47 PM
anlambert accepted D5867: Fix boolean mapping in metadata document.
Jun 16 2021, 2:53 PM · Archive search
anlambert closed D5879: identify: Fix exclude_patterns parameter type for identify_object.
Jun 16 2021, 12:47 PM
anlambert committed rDLDHGf3232bfd6786: identify: Fix exclude_patterns parameter type for identify_object (authored by anlambert).
identify: Fix exclude_patterns parameter type for identify_object
Jun 16 2021, 12:47 PM
anlambert updated the diff for D5879: identify: Fix exclude_patterns parameter type for identify_object.

Bump swh-model requirement

Jun 16 2021, 12:28 PM
anlambert added a comment to D5879: identify: Fix exclude_patterns parameter type for identify_object.
In D5879#150207, @zack wrote:

I think this also needs bumping the versioned dependency on swh-model (and a release of that).

already done ;-)

Jun 16 2021, 12:27 PM
anlambert added a comment to D5879: identify: Fix exclude_patterns parameter type for identify_object.
In D5879#150207, @zack wrote:

I think this also needs bumping the versioned dependency on swh-model (and a release of that).

Jun 16 2021, 12:26 PM
anlambert requested review of D5879: identify: Fix exclude_patterns parameter type for identify_object.
Jun 16 2021, 12:17 PM
anlambert closed D5877: assets/save: Ensure to use canonical github repo URL as origin URL.
Jun 16 2021, 10:52 AM
anlambert committed rDWAPPS1bd55d031ec8: assets/save: Ensure to use canonical github repo URL as origin URL (authored by anlambert).
assets/save: Ensure to use canonical github repo URL as origin URL
Jun 16 2021, 10:52 AM

Jun 15 2021

anlambert added a revision to T1110: document GitHub caseness caveats: D5877: assets/save: Ensure to use canonical github repo URL as origin URL.
Jun 15 2021, 5:58 PM · GitHub lister, Documentation
anlambert added a revision to T2187: Origin URL duplicates due to caps and .git URL: D5877: assets/save: Ensure to use canonical github repo URL as origin URL.
Jun 15 2021, 5:58 PM · Data Model
anlambert accepted D5874: web: Configure instance_name configuration key per webapp instance.
Jun 15 2021, 5:21 PM
anlambert accepted D5873: web: Add the instance_name configuration key.
Jun 15 2021, 4:03 PM
anlambert accepted D5858: Schedule save code now as recurring origins to ingest when successful.

Looks good to me !

Jun 15 2021, 3:42 PM
anlambert added inline comments to D5867: Fix boolean mapping in metadata document.
Jun 15 2021, 2:52 PM · Archive search
anlambert closed D5866: assets: Simplify promises handling with js async/await syntax.
Jun 15 2021, 2:49 PM
anlambert committed rDWAPPS8b4e77cae679: package.json: Upgrade dependencies (authored by anlambert).
package.json: Upgrade dependencies
Jun 15 2021, 2:49 PM
anlambert committed rDWAPPS2ba3891ea7e0: assets: Simplify promises handling with js async/await syntax (authored by anlambert).
assets: Simplify promises handling with js async/await syntax
Jun 15 2021, 2:49 PM
anlambert added inline comments to D5858: Schedule save code now as recurring origins to ingest when successful.
Jun 15 2021, 2:29 PM
anlambert added inline comments to D5858: Schedule save code now as recurring origins to ingest when successful.
Jun 15 2021, 2:26 PM
anlambert updated the diff for D5866: assets: Simplify promises handling with js async/await syntax.

Rebase

Jun 15 2021, 2:22 PM
anlambert requested changes to D5858: Schedule save code now as recurring origins to ingest when successful.

Possible duplicated origins should be handled but looks good otherwise.

Jun 15 2021, 2:06 PM
anlambert accepted D5828: Redirect to the browse origin page when the user searchs for an exact, already archived URL.

Looks good to me ! You can now merge your feature branch in the master one and push it to remote, phabricator will automatically close the diff afterwards.

Jun 15 2021, 12:24 PM
anlambert closed D5869: docker/conf/swh-web: Ensure to use a new empty database in dev mode.
Jun 15 2021, 12:14 PM
anlambert committed rDENV9de769abbf4b: docker/conf/swh-web: Ensure to use a new empty database in dev mode (authored by anlambert).
docker/conf/swh-web: Ensure to use a new empty database in dev mode
Jun 15 2021, 12:14 PM
anlambert added a comment to D5869: docker/conf/swh-web: Ensure to use a new empty database in dev mode.

An alternative would be to use a postgres db in docker instead.

But fine, let's see how it goes.

Jun 15 2021, 12:11 PM
anlambert requested review of D5869: docker/conf/swh-web: Ensure to use a new empty database in dev mode.
Jun 15 2021, 11:47 AM
anlambert requested changes to D5828: Redirect to the browse origin page when the user searchs for an exact, already archived URL.

Almost there, more code can be simplified and commit message should be improved.

Jun 15 2021, 11:10 AM
anlambert updated the summary of D5828: Redirect to the browse origin page when the user searchs for an exact, already archived URL.
Jun 15 2021, 11:07 AM

Jun 14 2021

anlambert requested review of D5866: assets: Simplify promises handling with js async/await syntax.
Jun 14 2021, 6:10 PM
anlambert added inline comments to D5828: Redirect to the browse origin page when the user searchs for an exact, already archived URL.
Jun 14 2021, 6:03 PM
anlambert requested changes to D5828: Redirect to the browse origin page when the user searchs for an exact, already archived URL.

Looks good but can be improved a little.

Jun 14 2021, 5:18 PM
anlambert closed T3365: save code now: Failure to ingest new 'archives' type when head response is incomplete as Resolved.

Issue has been resolved and tarballs hosted on the Internet Archive can now be properly loaded in production (see example).

Jun 14 2021, 3:28 PM · Save Code Now
anlambert added a comment to T2187: Origin URL duplicates due to caps and .git URL.

This should happen both client and server side (as fallback).

Jun 14 2021, 1:45 PM · Data Model
anlambert accepted D5860: test_origin_intrinsic_metadata_long_description: Re-increase description size.

Ack, I should have ping on you on that change. Plus now that painless script errors are caught by an exception handler, long pytest report printing the bulk error exceptions should not happen so often.

Jun 14 2021, 1:42 PM

Jun 11 2021

anlambert closed D5859: common/origin_save/origin_exists: Handle Internet Archive artifact URLs.
Jun 11 2021, 5:23 PM
anlambert committed rDWAPPSa0db251b3280: common/origin_save/origin_exists: Handle Internet Archive artifact URLs (authored by anlambert).
common/origin_save/origin_exists: Handle Internet Archive artifact URLs
Jun 11 2021, 5:23 PM
anlambert added a comment to T3365: save code now: Failure to ingest new 'archives' type when head response is incomplete.

Fix implemented in D5859 works \o/

Jun 11 2021, 5:11 PM · Save Code Now
anlambert requested review of D5859: common/origin_save/origin_exists: Handle Internet Archive artifact URLs.
Jun 11 2021, 4:51 PM
anlambert added a revision to T3365: save code now: Failure to ingest new 'archives' type when head response is incomplete: D5859: common/origin_save/origin_exists: Handle Internet Archive artifact URLs.
Jun 11 2021, 4:43 PM · Save Code Now
anlambert closed D5857: tarball/uncompress: Ensure destination folder exists before unpacking.
Jun 11 2021, 3:42 PM
anlambert committed rDCORE02bf96407ff4: tarball/uncompress: Ensure destination folder exists before unpacking (authored by anlambert).
tarball/uncompress: Ensure destination folder exists before unpacking
Jun 11 2021, 3:42 PM
anlambert closed D5856: tests/tarball: Use uncompress function instead of private ones.
Jun 11 2021, 3:42 PM
anlambert committed rDCORE14f8c4296819: tests/tarball: Use uncompress function instead of private ones (authored by anlambert).
tests/tarball: Use uncompress function instead of private ones
Jun 11 2021, 3:42 PM
anlambert requested review of D5856: tests/tarball: Use uncompress function instead of private ones.
Jun 11 2021, 3:29 PM
anlambert added a comment to T3365: save code now: Failure to ingest new 'archives' type when head response is incomplete.

I just figured out that data we are missing (Content-Length, Last-Modified) from tarballs archived by the Internet archive are in fact available in x-archive-orig-* HTTP response headers

Jun 11 2021, 2:59 PM · Save Code Now
anlambert created P1068 (An Untitled Masterwork).
Jun 11 2021, 2:41 PM
anlambert closed D5855: tests/test_search: Use a reasonably long description value.
Jun 11 2021, 1:53 PM
anlambert committed rDSEA35142063f335: tests/test_search: Use a reasonably long description value (authored by anlambert).
tests/test_search: Use a reasonably long description value
Jun 11 2021, 1:53 PM
anlambert requested review of D5855: tests/test_search: Use a reasonably long description value.
Jun 11 2021, 12:54 PM
anlambert closed D5853: tests/elasticsearch: Catch painless script errors and pretty print them.
Jun 11 2021, 12:41 PM
anlambert committed rDSEA870aecd97dc6: tests/elasticsearch: Catch painless script errors and pretty print them (authored by anlambert).
tests/elasticsearch: Catch painless script errors and pretty print them
Jun 11 2021, 12:41 PM
anlambert updated the diff for D5853: tests/elasticsearch: Catch painless script errors and pretty print them.

Forgot to remove bug in painless code

Jun 11 2021, 12:36 PM
anlambert updated the diff for D5853: tests/elasticsearch: Catch painless script errors and pretty print them.

Use multiline string and use dict.get to prevent errors if JSON schema changes.

Jun 11 2021, 12:26 PM
anlambert added inline comments to D5853: tests/elasticsearch: Catch painless script errors and pretty print them.
Jun 11 2021, 11:37 AM
anlambert added a comment to T3365: save code now: Failure to ingest new 'archives' type when head response is incomplete.

@ardumont, @rdicosmo, I just figured out that data we are missing (Content-Length, Last-Modified) from tarballs archived by the Internet archive are in fact available in x-archive-orig-* HTTP response headers, see below:

Jun 11 2021, 11:29 AM · Save Code Now

Jun 10 2021

anlambert requested review of D5853: tests/elasticsearch: Catch painless script errors and pretty print them.
Jun 10 2021, 7:53 PM
anlambert committed rCJSWH14c02b443eb0: jobs/swh-environment: Double timeout value (authored by anlambert).
jobs/swh-environment: Double timeout value
Jun 10 2021, 4:37 PM
anlambert closed D5852: package.loader: Handle tarball download erroneously marked as gzipped.
Jun 10 2021, 4:13 PM
anlambert committed rDLDBASEad79654a5316: package/loader: Handle tarball download erroneously marked as gzipped (authored by anlambert).
package/loader: Handle tarball download erroneously marked as gzipped
Jun 10 2021, 4:13 PM
anlambert updated the diff for D5852: package.loader: Handle tarball download erroneously marked as gzipped.

Update commit message

Jun 10 2021, 4:10 PM
anlambert closed T3369: swh.core.tarballl.uncompress: Some zip files fail to be uncompressed as Resolved by committing rDCOREcbf07d953e4a: tarball: Add support to unpack ZIP archive with implode compression.
Jun 10 2021, 4:09 PM · Core & foundations
anlambert closed D5850: tarball: Add support to unpack ZIP archive with implode compression.
Jun 10 2021, 4:09 PM
anlambert committed rDCOREcbf07d953e4a: tarball: Add support to unpack ZIP archive with implode compression (authored by anlambert).
tarball: Add support to unpack ZIP archive with implode compression
Jun 10 2021, 4:09 PM
anlambert added inline comments to D5852: package.loader: Handle tarball download erroneously marked as gzipped.
Jun 10 2021, 4:05 PM
anlambert updated the diff for D5852: package.loader: Handle tarball download erroneously marked as gzipped.

Fix typo

Jun 10 2021, 4:05 PM
anlambert updated the summary of D5852: package.loader: Handle tarball download erroneously marked as gzipped.
Jun 10 2021, 3:26 PM
anlambert requested review of D5852: package.loader: Handle tarball download erroneously marked as gzipped.
Jun 10 2021, 3:25 PM
anlambert updated the diff for D5850: tarball: Add support to unpack ZIP archive with implode compression.

Apply @vlorentz comment

Jun 10 2021, 3:25 PM
anlambert added inline comments to D5850: tarball: Add support to unpack ZIP archive with implode compression.
Jun 10 2021, 1:40 PM
anlambert requested review of D5850: tarball: Add support to unpack ZIP archive with implode compression.
Jun 10 2021, 1:40 PM
anlambert added a revision to T3369: swh.core.tarballl.uncompress: Some zip files fail to be uncompressed: D5850: tarball: Add support to unpack ZIP archive with implode compression.
Jun 10 2021, 1:38 PM · Core & foundations
anlambert added a comment to T3157: Add a search feature to snapshot branches view.

This will simplify branches view implementation. You should create a diff to handle it.

Yes. Let me check it's side effects as well. The 'archive.lookup_snapshot' is called from multiple places.

Jun 10 2021, 12:15 PM · Easy hack, Web app
anlambert added a comment to T3157: Add a search feature to snapshot branches view.

@anlambert Thank you. I will try to do some refactoring here.

Jun 10 2021, 11:58 AM · Easy hack, Web app

Jun 9 2021

anlambert added a comment to T3157: Add a search feature to snapshot branches view.

There is no need to use JavaScript here from my point of view.

Jun 9 2021, 6:35 PM · Easy hack, Web app
anlambert added a comment to T3157: Add a search feature to snapshot branches view.

The ideal fix would be to refactor the code to move the entire presentation logic to the template engine. (We already have a check in the template to show "The list of branches is empty" message in case the branch list is empty. But, i think this case will never occur with the current server code)

Jun 9 2021, 6:24 PM · Easy hack, Web app
anlambert added a comment to T3157: Add a search feature to snapshot branches view.

More answers after further reading and analysis.

Jun 9 2021, 6:18 PM · Easy hack, Web app
anlambert added a comment to T3157: Add a search feature to snapshot branches view.

This will eventually cause a 404 error when the user search for a non existing branch name in the UI. I don't think this is the intended behavior.

Jun 9 2021, 6:08 PM · Easy hack, Web app
anlambert committed rDMFCD3b1742d6d4c9: conftest: Fix postgresql_fact parameter name (authored by anlambert).
conftest: Fix postgresql_fact parameter name
Jun 9 2021, 5:00 PM
anlambert committed rDDATASET58a6a6e46768: mypy: Fix errors with release >= v0.900 (authored by anlambert).
mypy: Fix errors with release >= v0.900
Jun 9 2021, 4:57 PM
anlambert committed rDDATASETc8c1b90d9874: requirements-swh: Add missing swh-storage dependency (authored by anlambert).
requirements-swh: Add missing swh-storage dependency
Jun 9 2021, 4:57 PM
anlambert committed rDDATASET3522facf2713: Add pytest.ini and tox.ini files (authored by anlambert).
Add pytest.ini and tox.ini files
Jun 9 2021, 4:57 PM
anlambert accepted D5840: Add typing stubs dependencies for mypy>0.900.
Jun 9 2021, 4:43 PM
anlambert added inline comments to D5840: Add typing stubs dependencies for mypy>0.900.
Jun 9 2021, 4:39 PM
anlambert committed rDJNL8358298c4e31: mypy: Fix errors with release >= v0.900 (authored by anlambert).
mypy: Fix errors with release >= v0.900
Jun 9 2021, 4:30 PM
anlambert committed rDLDBASE644134a86d3a: mypy: Fix errors with release >= v0.900 (authored by anlambert).
mypy: Fix errors with release >= v0.900
Jun 9 2021, 4:18 PM
anlambert added a comment to D5839: Add typing stubs dependencies for mypy>0.900.

not for review, just to make sure it's enough to fix the build [1]

Jun 9 2021, 4:15 PM
anlambert committed rDLDG317d5c208896: mypy: Fix errors with release >= v0.900 (authored by anlambert).
mypy: Fix errors with release >= v0.900
Jun 9 2021, 4:09 PM
anlambert committed rDLDHG2b3dbb0ab563: mypy: Fix errors with release >= v0.900 (authored by anlambert).
mypy: Fix errors with release >= v0.900
Jun 9 2021, 3:56 PM
anlambert committed rDLDSVNb01318f80a81: mypy: Fix errors with release >= v0.900 (authored by anlambert).
mypy: Fix errors with release >= v0.900
Jun 9 2021, 3:45 PM
anlambert committed rDMOD8540c6700ccd: mypy: Fix errors with release >= v0.900 (authored by anlambert).
mypy: Fix errors with release >= v0.900
Jun 9 2021, 3:37 PM
anlambert committed rDOBJS37dcacf162d2: mypy: Fix errors with release >= v0.900 (authored by anlambert).
mypy: Fix errors with release >= v0.900
Jun 9 2021, 3:33 PM
anlambert committed rDOBJSRPL720b7af2bff7: mypy: Fix errors with release >= v0.900 (authored by anlambert).
mypy: Fix errors with release >= v0.900
Jun 9 2021, 3:26 PM
anlambert committed rDTPLe0b767a08207: mypy: Fix errors with release >= v0.900 (authored by anlambert).
mypy: Fix errors with release >= v0.900
Jun 9 2021, 3:22 PM
anlambert committed rDTSCNf9f73cd27ae2: mypy: Fix errors with release >= v0.900 (authored by anlambert).
mypy: Fix errors with release >= v0.900
Jun 9 2021, 3:15 PM
anlambert committed rDSEA3e129a3f48ee: mypy: Fix errors with release >= v0.900 (authored by anlambert).
mypy: Fix errors with release >= v0.900
Jun 9 2021, 3:09 PM