• wrong check when using the source path

wrong directory check

rebase

missing import in mypy.ini

rebase

requested changes

update

update

rebase

parent

child diff

squash

child diff

In D3069#74580, @vlorentz wrote:

Could you split this into two diffs?

The new json output will be like the following:

requested changes

• requirements-swh: swh.model version >= 0.0.65
• match pattern with fnmatch
• tests adapted to accept exclude patterns

In D2989#72502, @ardumont wrote:

(maybe not immediately though, it's currently building ;).

it's built so it's fine now ;)

In D2989#72439, @vlorentz wrote:

Instead of building the set of all paths to exclude (possibly very large), I think you should pass patterns to directory_filter and use [[ https://docs.python.org/3/library/fnmatch.html | fnmatch ]] there

In D2960#71967, @ardumont wrote:

Looks good to me.

Don't you want to implement your callable here (the one which actually exclude stuff)?

Because now, the diff title is no longer in sync with the code ;)

Or do you plan to add it in swh-scanner?

requested changes

requested changes

In D2960#71669, @ardumont wrote:

I can define a dir_filter function but how could i get the values i need, like the full path generated inside from_disk, without touching the implementation?

I'm not sure i understand the question.
from_disk has a save_path parameter. If set to True, the paths are stored alongside the output result.
Isn't that enough?

In D2960#71656, @ardumont wrote:

I think in the end, this should be implemented with the dir_filter parameter instead.

Please, check that you can do this. If you can, this will, there is no need to change the current implementation.
And instead, either:

• not touch swh.model at all, then define your ignore_path function in swh-scanner and use it when calling from_disk.
• if you think, this can be a shared behavior (i guess it can), define a function callable here in swh-model (like accept_all_directories and the other ones next to it).

In both cases, this won't touch Directory.from_disk though.

rebase

In D2863#71069, @vlorentz wrote:

Could you add tests that check this bug doesn't happen again?

ids of sunburst must be unique

requested changes
resolved bug due to wrong percentage calculation

In D2863#70007, @vlorentz wrote:

Sorry I didn't do it earlier, but I finally got around to actually running this code, and I found a bug: if two directories have the same name (or if there's one named "total"), then the swh-scanner process exits early, and nothing is shown in the browser.

requested changes

• test dataframe generation
• requested changes

The problem here is that actually swh-model il calculating the persistent identifier of the input path, so it also has to exclude the specified paths.

rebase

better json output

tests

In D2855#68683, @ardumont wrote:

/with jedi mind trick voice

You want to add types
You want to update the tests

;)

changes

removed --no-start-live-server option in pytest.ini

rebase