In D2960#71669, @ardumont wrote:

I can define a dir_filter function but how could i get the values i need, like the full path generated inside from_disk, without touching the implementation?

I'm not sure i understand the question.
from_disk has a save_path parameter. If set to True, the paths are stored alongside the output result.
Isn't that enough?

In D2960#71656, @ardumont wrote:

I think in the end, this should be implemented with the dir_filter parameter instead.

Please, check that you can do this. If you can, this will, there is no need to change the current implementation.
And instead, either:

• not touch swh.model at all, then define your ignore_path function in swh-scanner and use it when calling from_disk.
• if you think, this can be a shared behavior (i guess it can), define a function callable here in swh-model (like accept_all_directories and the other ones next to it).

In both cases, this won't touch Directory.from_disk though.

rebase

In D2863#71069, @vlorentz wrote:

Could you add tests that check this bug doesn't happen again?

ids of sunburst must be unique

requested changes
resolved bug due to wrong percentage calculation

In D2863#70007, @vlorentz wrote:

Sorry I didn't do it earlier, but I finally got around to actually running this code, and I found a bug: if two directories have the same name (or if there's one named "total"), then the swh-scanner process exits early, and nothing is shown in the browser.

requested changes

• test dataframe generation
• requested changes

The problem here is that actually swh-model il calculating the persistent identifier of the input path, so it also has to exclude the specified paths.

rebase

better json output

tests

In D2855#68683, @ardumont wrote:

/with jedi mind trick voice

You want to add types
You want to update the tests

;)

changes

removed --no-start-live-server option in pytest.ini

rebase

In D2794#66864, @vlorentz wrote:

Now that swh-scanner has tests, it would be nice if every new feature and bugfix added (or changed) one or more tests.

make tox pass

rebase

if a symbolic link is encountered, the persistent identifier is parsed by pid_of_file (as in swh.model)

rebase

comments

In D2785#66652, @vlorentz wrote:

In D2785#66602, @DanSeraf wrote:

In D2785#66553, @vlorentz wrote:

When does this happen? Could you add a test for this behavior?

When, for example, there is a broken symlink. It can't read if it's a file or a directory.

Could you add this as a comment?

In D2791#66642, @vlorentz wrote:

I'm restarting the tests

clean flask_api.py

make flake8 pass

In D2785#66553, @vlorentz wrote:

When does this happen? Could you add a test for this behavior?

cli: --format option, text and json
model:

• changed function name from getJsonChild to getJsonTree
• show function: output based on format

cli: flake8

• cli: scanner group with scan option, updated description
• color only when stdout is connected to terminal

In D2657#65016, @vlorentz wrote:

It's still missing a description of what it does (ie. check that files exist in swh), in the CLI help and/or README.

and I still think you should use a subcommand, in case you want to add new features in the future without breaking scripts that depend on this one.

In D2657#64937, @ardumont wrote:

rebase

limit is set inside the endpoint

rebase