Page MenuHomeSoftware Heritage
Feed Advanced Search

Mar 12 2020

DanSeraf committed rDTSCN8448ae0a840e: get_subpaths test adapted to the new temp_folder structure (authored by DanSeraf).
get_subpaths test adapted to the new temp_folder structure
Mar 12 2020, 6:42 PM
DanSeraf committed rDTSCN3de533865fef: changed folder structure in temp_folder fixture (authored by DanSeraf).
changed folder structure in temp_folder fixture
Mar 12 2020, 6:42 PM
DanSeraf closed D2811: scanner: added test for the model.
Mar 12 2020, 6:42 PM
DanSeraf updated the diff for D2811: scanner: added test for the model.

changes

Mar 12 2020, 6:02 PM

Mar 11 2020

DanSeraf created D2811: scanner: added test for the model.
Mar 11 2020, 2:57 PM
DanSeraf closed T2300: swh-scanner: print a nicer error message when rate limit is hit as Resolved by committing rDTSCNcb5faf6e866a: exceptions: APIError response.
Mar 11 2020, 9:51 AM · Easy hack, Code scanner
DanSeraf committed rDTSCNcb5faf6e866a: exceptions: APIError response (authored by DanSeraf).
exceptions: APIError response
Mar 11 2020, 9:51 AM
DanSeraf committed rDTSCN1857499820d0: let live_server start directly (authored by DanSeraf).
let live_server start directly
Mar 11 2020, 9:51 AM
DanSeraf closed D2794: exceptions: custom error response.
Mar 11 2020, 9:51 AM
DanSeraf updated the diff for D2794: exceptions: custom error response.

removed --no-start-live-server option in pytest.ini

Mar 11 2020, 9:34 AM
DanSeraf added inline comments to D2794: exceptions: custom error response.
Mar 11 2020, 9:29 AM

Mar 10 2020

DanSeraf updated the diff for D2794: exceptions: custom error response.

rebase

Mar 10 2020, 6:02 PM
DanSeraf updated the diff for D2794: exceptions: custom error response.
flake8
Mar 10 2020, 3:14 PM
DanSeraf updated the diff for D2794: exceptions: custom error response.
assert APIError raises when the payload is too large
live_server override, only one per session
Mar 10 2020, 3:10 PM
DanSeraf added a comment to D2794: exceptions: custom error response.

Now that swh-scanner has tests, it would be nice if every new feature and bugfix added (or changed) one or more tests.

Mar 10 2020, 1:40 PM
DanSeraf created D2794: exceptions: custom error response.
Mar 10 2020, 12:11 PM
DanSeraf closed T2299: scanner: add integration tests as Resolved by committing rDTSCNa5555dd12464: basic test for the scanner.
Mar 10 2020, 10:46 AM · Code scanner
DanSeraf committed rDTSCNa5555dd12464: basic test for the scanner (authored by DanSeraf).
basic test for the scanner
Mar 10 2020, 10:46 AM
DanSeraf closed D2791: swh-scanner: integration tests.
Mar 10 2020, 10:46 AM

Mar 9 2020

DanSeraf updated the diff for D2791: swh-scanner: integration tests.

make tox pass

Mar 9 2020, 5:50 PM
DanSeraf updated the diff for D2791: swh-scanner: integration tests.

rebase

Mar 9 2020, 3:19 PM
DanSeraf closed D2785: scanner: check that pid exists before yield the sub path.
Mar 9 2020, 12:36 PM
DanSeraf committed rDTSCNb984695a79a8: scanner: behavior when a symlink is encountered (authored by DanSeraf).
scanner: behavior when a symlink is encountered
Mar 9 2020, 12:36 PM
DanSeraf updated the diff for D2785: scanner: check that pid exists before yield the sub path.

if a symbolic link is encountered, the persistent identifier is parsed by pid_of_file (as in swh.model)

Mar 9 2020, 12:29 PM

Mar 6 2020

DanSeraf updated the diff for D2791: swh-scanner: integration tests.

rebase

Mar 6 2020, 5:08 PM
DanSeraf updated the diff for D2785: scanner: check that pid exists before yield the sub path.

comments

Mar 6 2020, 5:03 PM
DanSeraf added a comment to D2785: scanner: check that pid exists before yield the sub path.

When does this happen? Could you add a test for this behavior?

When, for example, there is a broken symlink. It can't read if it's a file or a directory.

Could you add this as a comment?

Mar 6 2020, 4:56 PM
DanSeraf added a comment to D2791: swh-scanner: integration tests.

I'm restarting the tests

Mar 6 2020, 4:48 PM
DanSeraf updated the diff for D2791: swh-scanner: integration tests.

clean flask_api.py

Mar 6 2020, 4:47 PM
DanSeraf updated the diff for D2791: swh-scanner: integration tests.

make flake8 pass

Mar 6 2020, 4:44 PM
DanSeraf created D2791: swh-scanner: integration tests.
Mar 6 2020, 4:40 PM
DanSeraf committed rDTSCNe21189377049: model: getJsonTree, wrong name on recursive call (authored by DanSeraf).
model: getJsonTree, wrong name on recursive call
Mar 6 2020, 4:04 PM
DanSeraf closed D2784: model: getJsonTree, wrong name on recursive call.
Mar 6 2020, 4:04 PM
DanSeraf added a comment to D2785: scanner: check that pid exists before yield the sub path.

When does this happen? Could you add a test for this behavior?

Mar 6 2020, 4:02 PM
DanSeraf created D2785: scanner: check that pid exists before yield the sub path.
Mar 6 2020, 2:37 PM
DanSeraf created D2784: model: getJsonTree, wrong name on recursive call.
Mar 6 2020, 2:28 PM
DanSeraf added a watcher for Code scanner: DanSeraf.
Mar 6 2020, 10:46 AM

Mar 3 2020

DanSeraf committed rDTSCN9c3954efa07a: model: using f-strings in printNode function (authored by DanSeraf).
model: using f-strings in printNode function
Mar 3 2020, 4:44 PM
DanSeraf closed D2758: model: using f-strings in printNode function.
Mar 3 2020, 4:43 PM
DanSeraf created D2758: model: using f-strings in printNode function.
Mar 3 2020, 3:00 PM
DanSeraf committed rDTSCNcedf6fc9b068: model: output result to json file (authored by DanSeraf).
model: output result to json file
Mar 3 2020, 2:46 PM
DanSeraf closed T2298: scanner: support alternative output formats as Resolved by committing rDTSCNcedf6fc9b068: model: output result to json file.
Mar 3 2020, 2:46 PM · Code scanner
DanSeraf closed D2748: model: output result to json file.
Mar 3 2020, 2:46 PM
DanSeraf updated the diff for D2748: model: output result to json file.

cli: --format option, text and json
model:

  • changed function name from getJsonChild to getJsonTree
  • show function: output based on format
Mar 3 2020, 1:36 PM

Mar 2 2020

DanSeraf updated the diff for D2748: model: output result to json file.

cli: flake8

Mar 2 2020, 7:22 PM
DanSeraf created D2748: model: output result to json file.
Mar 2 2020, 7:17 PM

Feb 28 2020

DanSeraf committed rDTSCN08059a547774: wrong url parsing (authored by DanSeraf).
wrong url parsing
Feb 28 2020, 4:57 PM
DanSeraf triaged T2298: scanner: support alternative output formats as Low priority.
Feb 28 2020, 3:36 PM · Code scanner
DanSeraf committed rDTSCNdd9004026c04: color only when stdout is connected to terminal, removed logger (authored by DanSeraf).
color only when stdout is connected to terminal, removed logger
Feb 28 2020, 3:12 PM
DanSeraf committed rDTSCN1541e9f44a9e: default url (authored by DanSeraf).
default url
Feb 28 2020, 3:12 PM
DanSeraf committed rDTSCNc0dbe0b1907b: cli: scanner group with scan option, updated description (authored by DanSeraf).
cli: scanner group with scan option, updated description
Feb 28 2020, 3:12 PM
DanSeraf committed rDTSCNed5a359e2ef4: added: model, logger and exceptions (authored by DanSeraf).
added: model, logger and exceptions
Feb 28 2020, 3:12 PM
DanSeraf committed rDTSCN40442b4d6bf8: scanner: asynchronous operation, type annotation (authored by DanSeraf).
scanner: asynchronous operation, type annotation
Feb 28 2020, 3:12 PM
DanSeraf committed rDTSCNf39b3d2c3374: test requirements (authored by DanSeraf).
test requirements
Feb 28 2020, 3:12 PM
DanSeraf committed rDTSCNb09cf5b29168: code scanner prototype (authored by DanSeraf).
code scanner prototype
Feb 28 2020, 3:12 PM
DanSeraf closed D2657: code scanner prototype.
Feb 28 2020, 3:12 PM
DanSeraf updated the summary of D2657: code scanner prototype.
Feb 28 2020, 2:53 PM
DanSeraf updated the diff for D2657: code scanner prototype.
default url
Feb 28 2020, 1:29 PM
DanSeraf updated the diff for D2657: code scanner prototype.
  • cli: scanner group with scan option, updated description
  • color only when stdout is connected to terminal
Feb 28 2020, 1:24 PM

Feb 27 2020

DanSeraf added a comment to D2657: code scanner prototype.

It's still missing a description of what it does (ie. check that files exist in swh), in the CLI help and/or README.

and I still think you should use a subcommand, in case you want to add new features in the future without breaking scripts that depend on this one.

Feb 27 2020, 4:12 PM
DanSeraf added a comment to D2657: code scanner prototype.
Feb 27 2020, 3:56 PM

Feb 26 2020

DanSeraf updated the diff for D2657: code scanner prototype.
The cli supports only one operation: swh scanner [options] path, where path is the root path we want to scan.
The output now is a tree like structure with a green/blue text if the path is found, red otherwise.
Feb 26 2020, 4:38 PM

Feb 17 2020

DanSeraf committed rDWAPPS792f7263ffc6: input size limit for /known/ endpoint (authored by DanSeraf).
input size limit for /known/ endpoint
Feb 17 2020, 11:54 AM
DanSeraf closed D2661: Web API: /known/ input size limit.
Feb 17 2020, 11:54 AM
DanSeraf updated the diff for D2661: Web API: /known/ input size limit.

rebase

Feb 17 2020, 11:54 AM

Feb 14 2020

DanSeraf updated the diff for D2661: Web API: /known/ input size limit.

limit is set inside the endpoint

Feb 14 2020, 11:12 AM

Feb 12 2020

DanSeraf updated the diff for D2661: Web API: /known/ input size limit.

rebase

Feb 12 2020, 4:36 PM
DanSeraf added inline comments to D2661: Web API: /known/ input size limit.
Feb 12 2020, 4:09 PM
DanSeraf added inline comments to D2661: Web API: /known/ input size limit.
Feb 12 2020, 3:38 PM
DanSeraf updated the summary of D2661: Web API: /known/ input size limit.
Feb 12 2020, 3:18 PM
DanSeraf created D2661: Web API: /known/ input size limit.
Feb 12 2020, 3:17 PM
DanSeraf created D2657: code scanner prototype.
Feb 12 2020, 12:13 PM
DanSeraf committed rDTSCN57a7a3d6e770: init swh-py-template (authored by DanSeraf).
init swh-py-template
Feb 12 2020, 11:00 AM

Jan 28 2020

DanSeraf created D2595: @danseraf: change public key.
Jan 28 2020, 3:16 PM
DanSeraf committed rDWAPPS773ee1a21b9c: /known/ api endpoint (authored by DanSeraf).
/known/ api endpoint
Jan 28 2020, 1:52 PM
DanSeraf committed rDWAPPS42f4b086a393: lookup missing hashes in the storage (authored by DanSeraf).
lookup missing hashes in the storage
Jan 28 2020, 1:52 PM
DanSeraf committed rDWAPPS7765946353da: group persistent identifiers by their type (authored by DanSeraf).
group persistent identifiers by their type
Jan 28 2020, 1:52 PM
DanSeraf closed D2582: Web API endpoint /known/.
Jan 28 2020, 1:52 PM
DanSeraf updated the diff for D2582: Web API endpoint /known/.
rebase
Jan 28 2020, 1:17 PM

Jan 27 2020

DanSeraf added a comment to D2582: Web API endpoint /known/.
Jan 27 2020, 8:34 PM
DanSeraf updated the diff for D2582: Web API endpoint /known/.
utils.py: function to get a parsed persistent identifier
Jan 27 2020, 4:58 PM
DanSeraf added a comment to D2582: Web API endpoint /known/.
  • returning 400 instead of 500 for invalid pid, type annotation on lookup_missing_hashes

This part is ok now, but cf my earlier comment: "You should add a small wrapper around parse_persistent_identifier to handle the error, that can be shared with resolve_swh_persistent_id."

Jan 27 2020, 4:00 PM
DanSeraf updated the diff for D2582: Web API endpoint /known/.
fix /known/ test expected values
Jan 27 2020, 3:15 PM
DanSeraf updated the diff for D2582: Web API endpoint /known/.
minor changes, rebase
Jan 27 2020, 3:07 PM
DanSeraf updated the diff for D2582: Web API endpoint /known/.
  • returning 400 instead of 500 for invalid pid, type annotation on lookup_missing_hashes
  • rebase
Jan 27 2020, 2:32 PM
DanSeraf updated the diff for D2582: Web API endpoint /known/.
  • /known/ endpoint tests
  • test_service: testing missing hashes lookup with present pids
Jan 27 2020, 11:51 AM

Jan 24 2020

DanSeraf updated the diff for D2582: Web API endpoint /known/.
  • known moved to identifiers view, parse pids before grouping them
  • test present pids
Jan 24 2020, 6:49 PM
DanSeraf added inline comments to D2582: Web API endpoint /known/.
Jan 24 2020, 5:28 PM
DanSeraf updated the summary of D2582: Web API endpoint /known/.
Jan 24 2020, 3:38 PM
DanSeraf created D2582: Web API endpoint /known/.
Jan 24 2020, 3:37 PM

Jan 22 2020

DanSeraf committed rDSTOcc2581008a7d: assert list doesn't have too many values (authored by DanSeraf).
assert list doesn't have too many values
Jan 22 2020, 3:55 PM
DanSeraf committed rDSTO2ebce62ef4ed: test endpoint: content_missing (sha1_git), snapshot_missing (authored by DanSeraf).
test endpoint: content_missing (sha1_git), snapshot_missing
Jan 22 2020, 3:55 PM
DanSeraf committed rDSTOc40d327e44df: in memory changes (authored by DanSeraf).
in memory changes
Jan 22 2020, 3:55 PM
DanSeraf committed rDSTO55ebd23f5176: storage: Add endpoint to get missing content (by sha1_git) and missing snapshot (authored by DanSeraf).
storage: Add endpoint to get missing content (by sha1_git) and missing snapshot
Jan 22 2020, 3:55 PM
DanSeraf closed D2570: storage: Add endpoint to get missing content (by sha1_git) and missing snapshot.
Jan 22 2020, 3:55 PM
DanSeraf updated the diff for D2570: storage: Add endpoint to get missing content (by sha1_git) and missing snapshot.
  • storage: Add endpoint to get missing content (by sha1_git) and missing snapshot
  • in memory changes
  • test endpoint: content_missing (sha1_git), snapshot_missing
  • assert list doesn't have too many values
Jan 22 2020, 3:55 PM
DanSeraf updated the diff for D2570: storage: Add endpoint to get missing content (by sha1_git) and missing snapshot.
  • assert list doesn't have too many values
Jan 22 2020, 3:04 PM
DanSeraf added inline comments to D2570: storage: Add endpoint to get missing content (by sha1_git) and missing snapshot.
Jan 22 2020, 3:01 PM
DanSeraf added inline comments to D2570: storage: Add endpoint to get missing content (by sha1_git) and missing snapshot.
Jan 22 2020, 2:53 PM
DanSeraf updated the diff for D2570: storage: Add endpoint to get missing content (by sha1_git) and missing snapshot.
  • in memory changes
  • test endpoint: content_missing (sha1_git), snapshot_missing
Jan 22 2020, 2:30 PM