User Details
User Details
- User Since
- Oct 1 2018, 11:23 AM (218 w, 4 d)
Thu, Dec 8
Thu, Dec 8
vlorentz requested review of D8947: Document statsd metrics and link to dashboards.
Add dataset name to the export id
vlorentz closed T4354: Contribute terms to ForgeFed, a subtask of T4249: Choose/define an ontology to use for indexed extrinsic origin metadata, as Resolved.
vlorentz updated the task description for T4354: Contribute terms to ForgeFed.
vlorentz added a comment to D8944: replay: Copy dir states and external paths in copy_from operations.
Are you sure the path argument to add_directory cannot start with a / or contain ..?
vlorentz added a comment to D8939: Rework the replaying exception handling.
Could you use a logger instance, and add if logger.isEnabledFor(logging.DEBUG): before logger.debug statements that use hash_to_hex?
vlorentz requested changes to D8943: svn: Use urllib.parse.quote to percent encode svn URLs.
Wed, Dec 7
Wed, Dec 7
vlorentz committed rDGRPHcd69e48b5acc: Replace RunAll with RunExportCompressUpload (authored by vlorentz).
Replace RunAll with RunExportCompressUpload
vlorentz committed rDGRPH233b0508395a: Prevent incorrect warning from being printed to output files (authored by vlorentz).
Prevent incorrect warning from being printed to output files
vlorentz committed rDGRPH042af3adf5b6: Fix crash when the sensitive dataset directory does not exist (authored by vlorentz).
Fix crash when the sensitive dataset directory does not exist
vlorentz requested review of D8935: Add dataset name to the export id.
Remove tool ids from Kafka messages
vlorentz updated the diff for D8932: Replace RunAll with RunExportCompressUpload.
rebase
vlorentz updated the diff for D8931: Prevent incorrect warning from being printed to output files.
I'm tired
vlorentz updated the diff for D8932: Replace RunAll with RunExportCompressUpload.
rebase
vlorentz updated the diff for D8931: Prevent incorrect warning from being printed to output files.
remove useless function
vlorentz updated the diff for D8932: Replace RunAll with RunExportCompressUpload.
rebase
vlorentz updated the diff for D8931: Prevent incorrect warning from being printed to output files.
less awful fix
vlorentz planned changes to D8931: Prevent incorrect warning from being printed to output files.
vlorentz added a comment to D8931: Prevent incorrect warning from being printed to output files.
vlorentz requested review of D8934: Remove tool ids from Kafka messages.
vlorentz triaged T4719: indexer storage crashes on kafka errors, because of integers in the key as Normal priority.
vlorentz requested review of D8932: Replace RunAll with RunExportCompressUpload.
vlorentz requested review of D8931: Prevent incorrect warning from being printed to output files.
vlorentz committed rDGRPH66253a872d6b: Add missing dependency on pytest-mock (authored by vlorentz).
Add missing dependency on pytest-mock
vlorentz committed rDGRPHb8ddd6ceadbd: origin_contributors: Fix typo and improve readability (authored by vlorentz).
origin_contributors: Fix typo and improve readability
vlorentz committed rDGRPHe65858a73918: Split swh/graph/luigi.py into modules (authored by vlorentz).
Split swh/graph/luigi.py into modules
vlorentz committed rDGRPHb76801259953: Add CLI script to generate Luigi config and call it (authored by vlorentz).
Add CLI script to generate Luigi config and call it
vlorentz committed rDGRPHdfd4c1dc3b22: ListOriginContributors: Ignore null author/committer in revisions/releases (authored by vlorentz).
ListOriginContributors: Ignore null author/committer in revisions/releases
vlorentz committed rDGRPHab2703efcb9a: Add Luigi task TopoSort and add a simple test (authored by vlorentz).
Add Luigi task TopoSort and add a simple test
Add ListOriginContributors
vlorentz committed rDGRPH559d4068bfe1: Regenerate the test dataset to include a release with no author (authored by vlorentz).
Regenerate the test dataset to include a release with no author
vlorentz committed rDGRPH7bee5d47a6eb: revert multithreading, it's actually twice as slow as singlethread (authored by vlorentz).
revert multithreading, it's actually twice as slow as singlethread
Improve comments
vlorentz committed rDGRPH922894410b6e: Add a sample of two ancestor with each node (authored by vlorentz).
Add a sample of two ancestor with each node
tentative multithread DFS
Implement a naive topological sort
vlorentz committed rDGRPHb8dc411ccd30: luigi: Add tasks UploadGraphToS3 and DownloadGraphFromS3 (authored by vlorentz).
luigi: Add tasks UploadGraphToS3 and DownloadGraphFromS3
vlorentz updated the diff for D8919: Add CLI script to generate Luigi config and call it.
rebase + fix typos + improve readability
vlorentz updated the diff for D8917: Split swh/graph/luigi.py into modules.
rebase
vlorentz updated the diff for D8912: ListOriginContributors: Ignore null author/committer in revisions/releases.
rebase
vlorentz updated the diff for D8910: Regenerate the test dataset to include a release with no author.
rebase
vlorentz updated the diff for D8883: Add a script to generate a topological sort.
rebase
vlorentz updated the diff for D8903: luigi: Add tasks UploadGraphToS3 and DownloadGraphFromS3.
rebase
vlorentz committed rDDATASETeceaf73f0fba: luigi.CreateAthena: Fix validation of DB name (authored by vlorentz).
luigi.CreateAthena: Fix validation of DB name
vlorentz committed rDDATASETc717f60fe08e: luigi.RunExportAll: Default to exporting all formats (authored by vlorentz).
luigi.RunExportAll: Default to exporting all formats
vlorentz committed rDDATASET22f7ed11f688: exporters/orc: Fix crash on visit status with no type (authored by vlorentz).
exporters/orc: Fix crash on visit status with no type
vlorentz added inline comments to D8908: Add ListOriginContributors.
vlorentz closed T1345: Update metadata docs about using CodeMeta vocabulary, a subtask of T1649: Update documentation with compliance scenario changes, as Resolved.
vlorentz added a comment to T1345: Update metadata docs about using CodeMeta vocabulary.
yes
Tue, Dec 6
Tue, Dec 6
vlorentz added a comment to D8907: feat: Add Hex.pm lister.
order sounds best. Do you want to do it?
vlorentz added a comment to T4394: Add support for running metadata fetchers without a VCS/package loaders.
We decided to add recurring fetches, so it will take care both of backfilling now, and visiting from time to time in the future. We're going to assume 3 months for now, as it seems reasonable to not exhaust rate limits.
vlorentz added a revision to T2220: swh-graph in production: D8919: Add CLI script to generate Luigi config and call it.
vlorentz added revisions to T4676: Add Luigi workflow in swh-dataset: D8919: Add CLI script to generate Luigi config and call it, D8924: exporters/orc: Fix crash on visit status with no type, D8925: luigi.CreateAthena: Fix validation of DB name, D8926: luigi.RunExportAll: Default to exporting all formats.
vlorentz requested review of D8926: luigi.RunExportAll: Default to exporting all formats.
vlorentz requested review of D8925: luigi.CreateAthena: Fix validation of DB name.
vlorentz requested review of D8924: exporters/orc: Fix crash on visit status with no type.
nice
ah, so it doesn't matter for other loaders. Phew!
vlorentz added a comment to D8920: from_disk.Content: Add missing path info for symlink.
Does it mean we were silently dropping paths until this? Which loaders use this?
Mon, Dec 5
Mon, Dec 5
vlorentz added a comment to D8918: gitlab: allow ignoring projects with certain path prefixes.
Could you add this check?
vlorentz requested review of D8919: Add CLI script to generate Luigi config and call it.
vlorentz requested review of D8917: Split swh/graph/luigi.py into modules.
vlorentz requested review of D8877: Fix incorrect error messages when failing to connect.
vlorentz updated the task description for T4712: Write Luigi tasks to regenerate the license dataset.