Page MenuHomeSoftware Heritage

TG1999 (Tushar Goel)
User

Projects

User does not belong to any projects.

User Details

User Since
May 8 2019, 6:11 AM (134 w, 7 h)

Recent Activity

Mar 8 2021

TG1999 closed D5212: mapping: make the regexp check the length of the hash.
Mar 8 2021, 11:03 AM
TG1999 committed rDMFCDc299d0db5fd5: mapping: make the regexp check the length of the hash (authored by TG1999).
mapping: make the regexp check the length of the hash
Mar 8 2021, 11:03 AM
TG1999 updated the diff for D5212: mapping: make the regexp check the length of the hash.

Change commit message

Mar 8 2021, 9:57 AM

Mar 7 2021

TG1999 updated subscribers of D5212: mapping: make the regexp check the length of the hash.

Ping @vlorentz

Mar 7 2021, 1:46 PM
TG1999 requested review of D5212: mapping: make the regexp check the length of the hash.
Mar 7 2021, 1:46 PM

Mar 5 2021

TG1999 accepted D5200: mapping: pass ExtendedSWHID instead of str.
Mar 5 2021, 10:26 AM
TG1999 accepted D5199: mapping: Update swh-model version to 1.0.0.
Mar 5 2021, 10:02 AM

Mar 2 2021

TG1999 closed D5165: change mapping status from boolean to enum.
Mar 2 2021, 3:05 PM
TG1999 committed rDMFCDb8711b141781: Change Mapping Status From Boolean To Enum (authored by TG1999).
Change Mapping Status From Boolean To Enum
Mar 2 2021, 3:05 PM
TG1999 updated the diff for D5165: change mapping status from boolean to enum.

change function name from is_hex to is_sha1

Mar 2 2021, 3:03 PM
TG1999 updated the diff for D5165: change mapping status from boolean to enum.

Add regexp

Mar 2 2021, 10:37 AM

Mar 1 2021

TG1999 added inline comments to D5165: change mapping status from boolean to enum.
Mar 1 2021, 5:10 PM
TG1999 added inline comments to D5165: change mapping status from boolean to enum.
Mar 1 2021, 2:33 PM
TG1999 updated the diff for D5165: change mapping status from boolean to enum.

Add case for partial mapping

Mar 1 2021, 11:42 AM
TG1999 updated subscribers of D5165: change mapping status from boolean to enum.

Hey @vlorentz please have a look !!

Mar 1 2021, 11:11 AM
TG1999 requested review of D5165: change mapping status from boolean to enum.
Mar 1 2021, 11:10 AM

Feb 23 2021

TG1999 closed D5036: Add Orchestrator.
Feb 23 2021, 5:43 PM
TG1999 committed rDMFCD89092343ee46: Add Orchestrator (authored by TG1999).
Add Orchestrator
Feb 23 2021, 5:43 PM
TG1999 updated the diff for D5036: Add Orchestrator.

Add timezone in mock data

Feb 23 2021, 4:49 PM
TG1999 updated the diff for D5036: Add Orchestrator.

Remove replace timezone from date

Feb 23 2021, 2:53 PM
TG1999 updated the diff for D5036: Add Orchestrator.

Use dateutil to parse date

Feb 23 2021, 12:12 PM
TG1999 updated the diff for D5036: Add Orchestrator.

Resolve datetime error

Feb 23 2021, 9:50 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Change date from string to datetime

Feb 23 2021, 8:23 AM

Feb 22 2021

TG1999 added inline comments to D5036: Add Orchestrator.
Feb 22 2021, 1:51 PM
TG1999 updated the diff for D5036: Add Orchestrator.

Remove Redundancy from Readme
Undo change in mapping_utils

Feb 22 2021, 11:16 AM
TG1999 added inline comments to D5036: Add Orchestrator.
Feb 22 2021, 11:11 AM
TG1999 added inline comments to D5036: Add Orchestrator.
Feb 22 2021, 10:43 AM
TG1999 added inline comments to D5036: Add Orchestrator.
Feb 22 2021, 10:41 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Add class for row

Feb 22 2021, 9:23 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Change SQL structure
Add readme in tests

Feb 22 2021, 9:16 AM

Feb 19 2021

TG1999 added inline comments to D5036: Add Orchestrator.
Feb 19 2021, 7:45 PM
TG1999 added inline comments to D5036: Add Orchestrator.
Feb 19 2021, 4:16 PM
TG1999 added inline comments to D5036: Add Orchestrator.
Feb 19 2021, 2:35 PM
TG1999 updated the diff for D5036: Add Orchestrator.

Add DBversion

Feb 19 2021, 1:46 PM
TG1999 accepted D5114: mapping_utils: define constants in uppercase and at the beginning of the file.
Feb 19 2021, 12:58 PM
TG1999 accepted D5115: mapping_utils: Define AUTHORITY and FETCHER properly instead of evolving them in every single use..
Feb 19 2021, 11:51 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Add tests for CLI

Feb 19 2021, 9:06 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Fix timezone errors

Feb 19 2021, 8:49 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Removed last_run_date table and add clearcode_env

Feb 19 2021, 8:46 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Remove mypy errors

Feb 19 2021, 7:42 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Remove black errors

Feb 19 2021, 7:39 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Remove Flake8 errors

Feb 19 2021, 7:38 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Shift mapping code in mapping_utils
Add type annotations
Correct map_previously_data
Improve select statements
Use hstore instead of a single row table

Feb 19 2021, 7:33 AM

Feb 18 2021

TG1999 added inline comments to D5036: Add Orchestrator.
Feb 18 2021, 8:12 PM

Feb 16 2021

TG1999 accepted D5079: Deduplicate 'mapping_status' and mapping logic from functions called by map_harvest.
Feb 16 2021, 10:08 AM
TG1999 added a comment to D5079: Deduplicate 'mapping_status' and mapping logic from functions called by map_harvest.

Thanks for this

Feb 16 2021, 10:07 AM
TG1999 closed D5061: map_row will return list of RawExtrensicMetadata.
Feb 16 2021, 9:41 AM
TG1999 committed rDMFCDf2c1b0bea04a: Return list of RawExtrensicMetadata instead of tuple (authored by TG1999).
Return list of RawExtrensicMetadata instead of tuple
Feb 16 2021, 9:41 AM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Reword commit message

Feb 16 2021, 9:38 AM
TG1999 added a comment to D5061: map_row will return list of RawExtrensicMetadata.

Done, made the change

Feb 16 2021, 9:23 AM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Reword commit message

Feb 16 2021, 9:22 AM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Reword commit message

Feb 16 2021, 7:58 AM

Feb 15 2021

TG1999 retitled D5061: map_row will return list of RawExtrensicMetadata from Change return type of map_row to map_row will return list of RawExtrensicMetadata.
Feb 15 2021, 3:17 PM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Add type of List[RawExtrensicMetadata] and change format of RawExtrensicMetadata

Feb 15 2021, 2:00 PM
TG1999 added inline comments to D5061: map_row will return list of RawExtrensicMetadata.
Feb 15 2021, 11:02 AM

Feb 14 2021

TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change type of MetadataAuthority

Feb 14 2021, 11:18 AM

Feb 13 2021

TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Resolve Flake 8 error

Feb 13 2021, 1:41 PM
TG1999 added inline comments to D5061: map_row will return list of RawExtrensicMetadata.
Feb 13 2021, 1:39 PM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change input type of map_row_data_with_metadata

Feb 13 2021, 1:39 PM

Feb 12 2021

TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change return type of mapping row with metadata

Feb 12 2021, 8:36 AM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change return type of mapping row with metadata

Feb 12 2021, 8:32 AM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change return type of mapping row with metadata

Feb 12 2021, 8:28 AM

Feb 11 2021

TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change return type of map_row

Feb 11 2021, 2:42 PM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change return type of map_row

Feb 11 2021, 1:01 PM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change return type of map_row

Feb 11 2021, 12:42 PM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change return type of map_row

Feb 11 2021, 12:32 PM

Feb 10 2021

TG1999 requested review of D5061: map_row will return list of RawExtrensicMetadata.
Feb 10 2021, 7:24 PM

Feb 8 2021

TG1999 updated the diff for D5036: Add Orchestrator.

Add Orchestrator

Feb 8 2021, 9:36 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Add Orchestrator

Feb 8 2021, 9:34 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Add Orchestrator

Feb 8 2021, 7:04 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Add Orchestrator

Feb 8 2021, 6:55 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Add Orchestrator

Feb 8 2021, 6:06 AM

Feb 7 2021

TG1999 requested review of D5036: Add Orchestrator.
Feb 7 2021, 6:58 PM

Feb 2 2021

TG1999 closed D4931: Add mapping of definitions and harvests.
Feb 2 2021, 10:47 AM
TG1999 committed rDMFCD9940287285ba: Map a row from clearcode toolkit with software heritage archive (authored by TG1999).
Map a row from clearcode toolkit with software heritage archive
Feb 2 2021, 10:47 AM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Map a row from clearcode toolkit with software heritage archive

Feb 2 2021, 10:42 AM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add feature to map a row from clearcode toolkit database with content and revision table from software heritage archive

Feb 2 2021, 10:37 AM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add feature to map a row from clearcode toolkit database with content and revision table from software heritage archive
This is to build a mechanism to map a row [path(Primary Key), content(binary data), last_modified_date(timestamp with timezone), map_error(error message while mapping), uuid] from clearcode toolkit database, with software heritage archive using content table for sha1 and revision table for sha1_git and extract required information from that row. Then return list of data that has been mapped and mapping status(if able to map every hash of that row, will return True, else return False) so the row that is not being able to map for now can be stored in a state, and can be mapped in future.
Add various exception classes in error.py that can be raised while mapping a row. Check if that row is a definition or harvest and also check if that row does not has invalid path, raise exception if path is invalid. If row is a definiton then map the data using map_definition and if it is a harvest then map it using map_harvest. Use storage instead of sql queries while mapping with the data inside archive. Add tests to cover all the cases and add docstrings to explain how every function works.

Feb 2 2021, 10:04 AM

Feb 1 2021

TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Use swh storage instead of sql queries to map sha1 with SWHIDs
Add different type of exceptions in error.py
Add function map_row, and try to check if the ID of the row is valid ID or not, if it is not a valid ID, then raise different exceptions defined in error.py. And if the ID is valid then check if the ID is a definition or harvest.
If the given ID is a definition then map the hashes with the archive (sha1 with content and sha1git with revision) and if able to map the defintion then return True,[(SWHID, TargetType(revision/content), Origin)], otherwise return None.
If the given ID is a harvest, then check the tool of the harvest (licensee, scancode, clearlydefined) and map accordig to the tool (map_licensee, map_scancode, map_clearlydefined), then return mapping status of that harvest (True if every sha1 of that harvest is mapped and False if failed to do it) and a list of data([SWHID,TargetType,None]).
Add tests to cover all the cases

Feb 1 2021, 6:32 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Use swh storage instead of sql queries to map sha1 with SWHIDs
Add different type of errors in error.py
Add function map_row, and try to check if the ID of the row is valid ID or not, if it is not a valid ID, then raise different exceptions defined in error.py. And if the ID is valid then check if the ID is a definition or harvest.
If the given ID is a definition then map the hashes with the archive (sha1 with content and sha1git with revision) and if able to map the defintion then return True,[(SWHID, TargetType(revision/content), Origin)], otherwise return None.
If the given ID is a harvest, then check the tool of the harvest (licensee, scancode, clearlydefined) and map accordig to the tool (map_licensee, map_scancode, map_clearlydefined), then return mapping status of that harvest (True if every sha1 of that harvest is mapped and False if failed to do it) and a list of data([SWHID,TargetType,None]).
Add tests to cover all the cases

Feb 1 2021, 5:54 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Use swh storage instead of sql queries to map sha1 with SWHIDs
Add different type of error classes in error.py
Add function map_row, and try to check if the ID of the row is valid ID or not, if it is not a valid ID, then raise different exceptions defined in error.py. And if the ID is valid then check if the ID is a definition or harvest.
If the given ID is a definition then map the hashes with the archive (sha1 with content and sha1git with revision) and return a valid SWHID, TargetType(revision/content), Origin
If the given ID is a harvest, then check the tool of the harvest (licensee, scancode, clearlydefined) and map accordig to the tool (map_licensee, map_scancode, map_clearlydefined), then return mapping status of that harvest (True if every sha1 of that harvest is mapped and False if failed to do it) and a list of data([SWHID,TargetType,None]).
Add tests to cover all the cases"

Feb 1 2021, 5:48 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not
Add error classes for all conditions where metadata can be wrong or for an invalid ID
Add docstring for error class

Feb 1 2021, 2:44 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not
Add error classes for all conditions where metadata can be wrong or for an invalid ID

Feb 1 2021, 11:04 AM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Feb 1 2021, 10:53 AM

Jan 29 2021

TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Jan 29 2021, 5:45 PM
TG1999 added inline comments to D4931: Add mapping of definitions and harvests.
Jan 29 2021, 4:48 PM
TG1999 added inline comments to D4931: Add mapping of definitions and harvests.
Jan 29 2021, 4:28 PM
TG1999 added inline comments to D4931: Add mapping of definitions and harvests.
Jan 29 2021, 4:16 PM
TG1999 added a comment to D4931: Add mapping of definitions and harvests.

@vlorentz made the suggested changes, thanks for a detailed review :)

Jan 29 2021, 2:40 PM
TG1999 added inline comments to D4931: Add mapping of definitions and harvests.
Jan 29 2021, 2:39 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Jan 29 2021, 2:36 PM
TG1999 added inline comments to D4931: Add mapping of definitions and harvests.
Jan 29 2021, 1:14 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Jan 29 2021, 7:42 AM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Jan 29 2021, 7:39 AM

Jan 28 2021

TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Jan 28 2021, 8:26 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Jan 28 2021, 6:15 PM
TG1999 updated subscribers of D4931: Add mapping of definitions and harvests.

Hi @douardda @vlorentz @zack please have a look :)

Jan 28 2021, 4:15 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Jan 28 2021, 4:12 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping of definitions and harvests

Jan 28 2021, 3:57 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping of definitions and harvests

Jan 28 2021, 3:37 PM