Page MenuHomeSoftware Heritage

DanSeraf (Daniele Serafini)
User

Projects

User Details

User Since
Sep 23 2019, 3:10 PM (40 w, 5 d)

Recent Activity

Fri, Jul 3

DanSeraf created P712 (An Untitled Masterwork).
Fri, Jul 3, 12:43 PM

Mon, Jun 22

DanSeraf committed rDTSCN0f10ec6ae8fe: dashboard: file visualization per directory path (authored by DanSeraf).
dashboard: file visualization per directory path
Mon, Jun 22, 7:39 PM
DanSeraf closed T2364: scanner: file browser in the sunburst/dashboard output as Resolved by committing rDTSCN0f10ec6ae8fe: dashboard: file visualization per directory path.
Mon, Jun 22, 7:39 PM · Code scanner
DanSeraf closed D3293: scanner: dashboard file visualization per directory path.
Mon, Jun 22, 7:39 PM
DanSeraf closed T2336: scanner: add support for an exclusion list as Resolved.
Mon, Jun 22, 2:57 PM · Code scanner
DanSeraf updated the diff for D3293: scanner: dashboard file visualization per directory path.
  • non-minified css in assets
Mon, Jun 22, 2:50 PM

Sat, Jun 20

DanSeraf updated the diff for D3293: scanner: dashboard file visualization per directory path.
  • workaround to check the table body
  • bootstrap css update
Sat, Jun 20, 4:37 PM
DanSeraf added a comment to D3293: scanner: dashboard file visualization per directory path.

Unfortunately the expected values can't be tested.

Why not?

Because the dash_html_components checks for object identity only. I wrote a comment inside the test.

Sat, Jun 20, 4:37 PM

Fri, Jun 19

DanSeraf added a comment to D3293: scanner: dashboard file visualization per directory path.
  • It's missing tests. I understand it's not easy to do for a GUI, but could you see what you can do about it?
Fri, Jun 19, 4:54 PM
DanSeraf updated the diff for D3293: scanner: dashboard file visualization per directory path.
  • init.py in dashboard directory
Fri, Jun 19, 4:51 PM
DanSeraf updated the diff for D3293: scanner: dashboard file visualization per directory path.
  • css as static asset
Fri, Jun 19, 4:41 PM

Tue, Jun 16

DanSeraf updated the summary of D3293: scanner: dashboard file visualization per directory path.
Tue, Jun 16, 6:45 PM
DanSeraf created D3293: scanner: dashboard file visualization per directory path.
Tue, Jun 16, 6:31 PM
DanSeraf committed rDTSCNbf9e586436d8: model: get file attributes from a specific directory (authored by DanSeraf).
model: get file attributes from a specific directory
Tue, Jun 16, 5:54 PM
DanSeraf closed D3284: scanner: retrieve file attributes.
Tue, Jun 16, 5:54 PM
DanSeraf updated the diff for D3284: scanner: retrieve file attributes.
  • wrong check when using the source path
Tue, Jun 16, 1:10 PM
DanSeraf updated the diff for D3284: scanner: retrieve file attributes.

wrong directory check

Tue, Jun 16, 12:15 PM
DanSeraf added inline comments to D3284: scanner: retrieve file attributes.
Tue, Jun 16, 12:15 PM

Mon, Jun 15

DanSeraf created D3284: scanner: retrieve file attributes.
Mon, Jun 15, 6:35 PM

Fri, Jun 5

DanSeraf committed rDTSCN13ac68a9471e: model: iterate only the child nodes instead of the child nodes attributes (authored by DanSeraf).
model: iterate only the child nodes instead of the child nodes attributes
Fri, Jun 5, 4:57 PM
DanSeraf closed D3230: scanner: nodes iteration function in model.
Fri, Jun 5, 4:57 PM
DanSeraf created D3230: scanner: nodes iteration function in model.
Fri, Jun 5, 12:00 PM

Jun 4 2020

DanSeraf committed rDTSCNe126d2ac98c9: interactive dashboard (authored by DanSeraf).
interactive dashboard
Jun 4 2020, 11:46 AM
DanSeraf closed D3216: scanner: Interactive dashboard.
Jun 4 2020, 11:46 AM

Jun 3 2020

DanSeraf created D3216: scanner: Interactive dashboard.
Jun 3 2020, 4:29 PM

Apr 30 2020

DanSeraf closed T2365: scanner: add color legend for sunburst output as Resolved by committing rDTSCNfb8ae03e494c: plot: color legend.
Apr 30 2020, 12:41 PM · Code scanner
DanSeraf committed rDTSCNfb8ae03e494c: plot: color legend (authored by DanSeraf).
plot: color legend
Apr 30 2020, 12:41 PM
DanSeraf closed D3099: plot: color legend.
Apr 30 2020, 12:41 PM
DanSeraf created D3099: plot: color legend.
Apr 30 2020, 12:02 PM

Apr 29 2020

DanSeraf committed rDTSCN623a9dbe6157: ndjson output format (authored by DanSeraf).
ndjson output format
Apr 29 2020, 4:40 PM
DanSeraf closed T2363: scanner: json output should return both known and unknown files/dirs as Resolved by committing rDTSCN623a9dbe6157: ndjson output format.
Apr 29 2020, 4:40 PM · Code scanner
DanSeraf closed D3085: scanner: ndjson output format.
Apr 29 2020, 4:40 PM
DanSeraf updated the diff for D3085: scanner: ndjson output format.

rebase

Apr 29 2020, 4:40 PM
DanSeraf updated the diff for D3085: scanner: ndjson output format.

missing import in mypy.ini

Apr 29 2020, 2:28 PM
DanSeraf updated the summary of D3085: scanner: ndjson output format.
Apr 29 2020, 1:17 PM
DanSeraf created D3085: scanner: ndjson output format.
Apr 29 2020, 1:16 PM
DanSeraf committed rDTSCN3f00bb004b4d: flat json output with known and swhid values (authored by DanSeraf).
flat json output with known and swhid values
Apr 29 2020, 12:57 PM
DanSeraf closed D3069: scanner: json output format.
Apr 29 2020, 12:57 PM
DanSeraf updated the diff for D3069: scanner: json output format.

rebase

Apr 29 2020, 12:56 PM
DanSeraf committed rDTSCNbbf296f7523a: model: known attribute in Tree structure (authored by DanSeraf).
model: known attribute in Tree structure
Apr 29 2020, 12:36 PM
DanSeraf closed D3070: model: known attribute in Tree structure.
Apr 29 2020, 12:36 PM

Apr 27 2020

DanSeraf updated the diff for D3070: model: known attribute in Tree structure.

requested changes

Apr 27 2020, 7:26 PM
DanSeraf updated the diff for D3069: scanner: json output format.

update

Apr 27 2020, 5:18 PM
DanSeraf updated the diff for D3069: scanner: json output format.

update

Apr 27 2020, 4:54 PM
DanSeraf updated the diff for D3070: model: known attribute in Tree structure.

rebase

Apr 27 2020, 4:52 PM
DanSeraf updated the diff for D3069: scanner: json output format.

parent

Apr 27 2020, 4:46 PM
DanSeraf updated the diff for D3070: model: known attribute in Tree structure.

child diff

Apr 27 2020, 4:46 PM
DanSeraf updated the diff for D3070: model: known attribute in Tree structure.

squash

Apr 27 2020, 4:42 PM
DanSeraf updated the diff for D3070: model: known attribute in Tree structure.

child diff

Apr 27 2020, 4:38 PM
DanSeraf added a comment to D3069: scanner: json output format.

Could you split this into two diffs?

Apr 27 2020, 4:17 PM
DanSeraf updated the summary of D3069: scanner: json output format.
Apr 27 2020, 4:09 PM
DanSeraf created D3070: model: known attribute in Tree structure.
Apr 27 2020, 4:05 PM
DanSeraf created D3069: scanner: json output format.
Apr 27 2020, 11:25 AM

Apr 22 2020

DanSeraf added a comment to T2363: scanner: json output should return both known and unknown files/dirs.

The new json output will be like the following:

Apr 22 2020, 6:13 PM · Code scanner

Apr 17 2020

DanSeraf committed rDTSCNbe3d3c124ac7: requirements-swh: swh.model version >= 0.0.64 (authored by DanSeraf).
requirements-swh: swh.model version >= 0.0.64
Apr 17 2020, 12:32 PM
DanSeraf committed rDTSCNd52afd0eaedd: scanner: option to exclude specific patterns (authored by DanSeraf).
scanner: option to exclude specific patterns
Apr 17 2020, 12:32 PM
DanSeraf closed D2989: scanner: exclusion list through glob patterns.
Apr 17 2020, 12:32 PM

Apr 16 2020

DanSeraf updated the diff for D2989: scanner: exclusion list through glob patterns.

requested changes

Apr 16 2020, 5:16 PM

Apr 15 2020

DanSeraf closed T2362: scanner: aiohttp.client_exceptions.ServerDisconnectedError: None as Invalid.
Apr 15 2020, 5:58 PM · Code scanner

Apr 14 2020

DanSeraf added inline comments to D2989: scanner: exclusion list through glob patterns.
Apr 14 2020, 4:36 PM

Apr 10 2020

DanSeraf updated the diff for D2989: scanner: exclusion list through glob patterns.
  • requirements-swh: swh.model version >= 0.0.65
  • match pattern with fnmatch
  • tests adapted to accept exclude patterns
Apr 10 2020, 3:51 PM

Apr 9 2020

DanSeraf added a comment to D2989: scanner: exclusion list through glob patterns.

(maybe not immediately though, it's currently building ;).

it's built so it's fine now ;)

Apr 9 2020, 5:07 PM
DanSeraf added a comment to D2989: scanner: exclusion list through glob patterns.

Instead of building the set of all paths to exclude (possibly very large), I think you should pass patterns to directory_filter and use [[ https://docs.python.org/3/library/fnmatch.html | fnmatch ]] there

Apr 9 2020, 4:23 PM
DanSeraf added inline comments to D2989: scanner: exclusion list through glob patterns.
Apr 9 2020, 4:18 PM
DanSeraf created D2989: scanner: exclusion list through glob patterns.
Apr 9 2020, 1:05 PM

Apr 8 2020

DanSeraf committed rDMOD5d6883b26594: from_disk: path parameter to dir_filter functions (authored by DanSeraf).
from_disk: path parameter to dir_filter functions
Apr 8 2020, 11:58 AM
DanSeraf closed D2960: swh-model: exclude path support for the function from_disk in Directory.
Apr 8 2020, 11:58 AM
DanSeraf added a comment to D2960: swh-model: exclude path support for the function from_disk in Directory.

Looks good to me.

Don't you want to implement your callable here (the one which actually exclude stuff)?

Because now, the diff title is no longer in sync with the code ;)

Or do you plan to add it in swh-scanner?

Apr 8 2020, 11:58 AM
DanSeraf updated the diff for D2960: swh-model: exclude path support for the function from_disk in Directory.

requested changes

Apr 8 2020, 11:33 AM
DanSeraf updated the diff for D2960: swh-model: exclude path support for the function from_disk in Directory.

requested changes

Apr 8 2020, 9:47 AM

Apr 6 2020

DanSeraf added a comment to D2960: swh-model: exclude path support for the function from_disk in Directory.

I can define a dir_filter function but how could i get the values i need, like the full path generated inside from_disk, without touching the implementation?

I'm not sure i understand the question.
from_disk has a save_path parameter. If set to True, the paths are stored alongside the output result.
Isn't that enough?

Apr 6 2020, 8:27 PM
DanSeraf added a comment to D2960: swh-model: exclude path support for the function from_disk in Directory.

I think in the end, this should be implemented with the dir_filter parameter instead.

Please, check that you can do this. If you can, this will, there is no need to change the current implementation.
And instead, either:

  • not touch swh.model at all, then define your ignore_path function in swh-scanner and use it when calling from_disk.
  • if you think, this can be a shared behavior (i guess it can), define a function callable here in swh-model (like accept_all_directories and the other ones next to it).

In both cases, this won't touch Directory.from_disk though.

Apr 6 2020, 6:10 PM
DanSeraf updated the summary of D2960: swh-model: exclude path support for the function from_disk in Directory.
Apr 6 2020, 4:10 PM
DanSeraf created D2960: swh-model: exclude path support for the function from_disk in Directory.
Apr 6 2020, 4:10 PM

Apr 2 2020

DanSeraf committed rDTSCNa618bf17c049: new visualization format: sunburst chart (authored by DanSeraf).
new visualization format: sunburst chart
Apr 2 2020, 2:08 PM
DanSeraf committed rDTSCNe4bb51b7d894: changed structure of temp_folder fixture (authored by DanSeraf).
changed structure of temp_folder fixture
Apr 2 2020, 2:08 PM
DanSeraf committed rDTSCNa49c4e5ec053: model: get information about directories (authored by DanSeraf).
model: get information about directories
Apr 2 2020, 2:08 PM
DanSeraf closed D2863: scanner: show result with a sunburst chart.
Apr 2 2020, 2:08 PM
DanSeraf updated the diff for D2863: scanner: show result with a sunburst chart.

rebase

Apr 2 2020, 2:07 PM

Apr 1 2020

DanSeraf added a comment to D2863: scanner: show result with a sunburst chart.

Could you add tests that check this bug doesn't happen again?

Apr 1 2020, 5:09 PM
DanSeraf updated the diff for D2863: scanner: show result with a sunburst chart.

ids of sunburst must be unique

Apr 1 2020, 1:42 PM

Mar 27 2020

DanSeraf updated the diff for D2863: scanner: show result with a sunburst chart.

requested changes
resolved bug due to wrong percentage calculation

Mar 27 2020, 1:20 PM
DanSeraf added a comment to D2863: scanner: show result with a sunburst chart.

Sorry I didn't do it earlier, but I finally got around to actually running this code, and I found a bug: if two directories have the same name (or if there's one named "total"), then the swh-scanner process exits early, and nothing is shown in the browser.

Mar 27 2020, 10:58 AM

Mar 26 2020

DanSeraf updated the diff for D2863: scanner: show result with a sunburst chart.

requested changes

Mar 26 2020, 7:35 PM
DanSeraf created P632 (An Untitled Masterwork).
Mar 26 2020, 2:32 PM

Mar 25 2020

DanSeraf updated the diff for D2863: scanner: show result with a sunburst chart.
  • test dataframe generation
  • requested changes
Mar 25 2020, 7:14 PM
DanSeraf added a comment to T2336: scanner: add support for an exclusion list.

The problem here is that actually swh-model il calculating the persistent identifier of the input path, so it also has to exclude the specified paths.

Mar 25 2020, 3:28 PM · Code scanner

Mar 23 2020

DanSeraf added inline comments to D2863: scanner: show result with a sunburst chart.
Mar 23 2020, 9:25 PM

Mar 20 2020

DanSeraf updated the summary of D2863: scanner: show result with a sunburst chart.
Mar 20 2020, 7:08 PM
DanSeraf created D2863: scanner: show result with a sunburst chart.
Mar 20 2020, 7:02 PM
DanSeraf committed rDTSCN5c62848288b0: dump json instead of dict (authored by DanSeraf).
dump json instead of dict
Mar 20 2020, 12:17 PM
DanSeraf closed D2855: scanner: dump json instead of dict.
Mar 20 2020, 12:17 PM
DanSeraf updated the diff for D2855: scanner: dump json instead of dict.

rebase

Mar 20 2020, 12:17 PM
DanSeraf updated the diff for D2855: scanner: dump json instead of dict.

better json output

Mar 20 2020, 11:54 AM

Mar 19 2020

DanSeraf updated the diff for D2855: scanner: dump json instead of dict.

tests

Mar 19 2020, 7:44 PM
DanSeraf added a comment to D2855: scanner: dump json instead of dict.

/with jedi mind trick voice

You want to add types
You want to update the tests

;)

Mar 19 2020, 7:33 PM
DanSeraf reclaimed D2855: scanner: dump json instead of dict.
Mar 19 2020, 4:18 PM
DanSeraf abandoned D2855: scanner: dump json instead of dict.
Mar 19 2020, 2:12 PM
DanSeraf created D2855: scanner: dump json instead of dict.
Mar 19 2020, 12:42 PM

Mar 12 2020

DanSeraf committed rDTSCN3d462c8364ed: added test for the model (authored by DanSeraf).
added test for the model
Mar 12 2020, 6:42 PM