Page MenuHomeSoftware Heritage

TG1999 (Tushar Goel)
User

Projects

User Details

User Since
May 8 2019, 6:11 AM (94 w, 1 d)

Recent Activity

Tue, Feb 23

TG1999 closed D5036: Add Orchestrator.
Tue, Feb 23, 5:43 PM
TG1999 committed rDMFCD89092343ee46: Add Orchestrator (authored by TG1999).
Add Orchestrator
Tue, Feb 23, 5:43 PM
TG1999 updated the diff for D5036: Add Orchestrator.

Add timezone in mock data

Tue, Feb 23, 4:49 PM
TG1999 updated the diff for D5036: Add Orchestrator.

Remove replace timezone from date

Tue, Feb 23, 2:53 PM
TG1999 updated the diff for D5036: Add Orchestrator.

Use dateutil to parse date

Tue, Feb 23, 12:12 PM
TG1999 updated the diff for D5036: Add Orchestrator.

Resolve datetime error

Tue, Feb 23, 9:50 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Change date from string to datetime

Tue, Feb 23, 8:23 AM

Mon, Feb 22

TG1999 added inline comments to D5036: Add Orchestrator.
Mon, Feb 22, 1:51 PM
TG1999 updated the diff for D5036: Add Orchestrator.

Remove Redundancy from Readme
Undo change in mapping_utils

Mon, Feb 22, 11:16 AM
TG1999 added inline comments to D5036: Add Orchestrator.
Mon, Feb 22, 11:11 AM
TG1999 added inline comments to D5036: Add Orchestrator.
Mon, Feb 22, 10:43 AM
TG1999 added inline comments to D5036: Add Orchestrator.
Mon, Feb 22, 10:41 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Add class for row

Mon, Feb 22, 9:23 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Change SQL structure
Add readme in tests

Mon, Feb 22, 9:16 AM

Fri, Feb 19

TG1999 added inline comments to D5036: Add Orchestrator.
Fri, Feb 19, 7:45 PM
TG1999 added inline comments to D5036: Add Orchestrator.
Fri, Feb 19, 4:16 PM
TG1999 added inline comments to D5036: Add Orchestrator.
Fri, Feb 19, 2:35 PM
TG1999 updated the diff for D5036: Add Orchestrator.

Add DBversion

Fri, Feb 19, 1:46 PM
TG1999 accepted D5114: mapping_utils: define constants in uppercase and at the beginning of the file.
Fri, Feb 19, 12:58 PM
TG1999 accepted D5115: mapping_utils: Define AUTHORITY and FETCHER properly instead of evolving them in every single use..
Fri, Feb 19, 11:51 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Add tests for CLI

Fri, Feb 19, 9:06 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Fix timezone errors

Fri, Feb 19, 8:49 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Removed last_run_date table and add clearcode_env

Fri, Feb 19, 8:46 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Remove mypy errors

Fri, Feb 19, 7:42 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Remove black errors

Fri, Feb 19, 7:39 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Remove Flake8 errors

Fri, Feb 19, 7:38 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Shift mapping code in mapping_utils
Add type annotations
Correct map_previously_data
Improve select statements
Use hstore instead of a single row table

Fri, Feb 19, 7:33 AM

Thu, Feb 18

TG1999 added inline comments to D5036: Add Orchestrator.
Thu, Feb 18, 8:12 PM

Tue, Feb 16

TG1999 accepted D5079: Deduplicate 'mapping_status' and mapping logic from functions called by map_harvest.
Tue, Feb 16, 10:08 AM
TG1999 added a comment to D5079: Deduplicate 'mapping_status' and mapping logic from functions called by map_harvest.

Thanks for this

Tue, Feb 16, 10:07 AM
TG1999 closed D5061: map_row will return list of RawExtrensicMetadata.
Tue, Feb 16, 9:41 AM
TG1999 committed rDMFCDf2c1b0bea04a: Return list of RawExtrensicMetadata instead of tuple (authored by TG1999).
Return list of RawExtrensicMetadata instead of tuple
Tue, Feb 16, 9:41 AM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Reword commit message

Tue, Feb 16, 9:38 AM
TG1999 added a comment to D5061: map_row will return list of RawExtrensicMetadata.

Done, made the change

Tue, Feb 16, 9:23 AM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Reword commit message

Tue, Feb 16, 9:22 AM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Reword commit message

Tue, Feb 16, 7:58 AM

Mon, Feb 15

TG1999 retitled D5061: map_row will return list of RawExtrensicMetadata from Change return type of map_row to map_row will return list of RawExtrensicMetadata.
Mon, Feb 15, 3:17 PM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Add type of List[RawExtrensicMetadata] and change format of RawExtrensicMetadata

Mon, Feb 15, 2:00 PM
TG1999 added inline comments to D5061: map_row will return list of RawExtrensicMetadata.
Mon, Feb 15, 11:02 AM

Sun, Feb 14

TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change type of MetadataAuthority

Sun, Feb 14, 11:18 AM

Sat, Feb 13

TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Resolve Flake 8 error

Sat, Feb 13, 1:41 PM
TG1999 added inline comments to D5061: map_row will return list of RawExtrensicMetadata.
Sat, Feb 13, 1:39 PM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change input type of map_row_data_with_metadata

Sat, Feb 13, 1:39 PM

Fri, Feb 12

TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change return type of mapping row with metadata

Fri, Feb 12, 8:36 AM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change return type of mapping row with metadata

Fri, Feb 12, 8:32 AM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change return type of mapping row with metadata

Fri, Feb 12, 8:28 AM

Thu, Feb 11

TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change return type of map_row

Thu, Feb 11, 2:42 PM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change return type of map_row

Thu, Feb 11, 1:01 PM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change return type of map_row

Thu, Feb 11, 12:42 PM
TG1999 updated the diff for D5061: map_row will return list of RawExtrensicMetadata.

Change return type of map_row

Thu, Feb 11, 12:32 PM

Wed, Feb 10

TG1999 requested review of D5061: map_row will return list of RawExtrensicMetadata.
Wed, Feb 10, 7:24 PM

Mon, Feb 8

TG1999 updated the diff for D5036: Add Orchestrator.

Add Orchestrator

Mon, Feb 8, 9:36 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Add Orchestrator

Mon, Feb 8, 9:34 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Add Orchestrator

Mon, Feb 8, 7:04 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Add Orchestrator

Mon, Feb 8, 6:55 AM
TG1999 updated the diff for D5036: Add Orchestrator.

Add Orchestrator

Mon, Feb 8, 6:06 AM

Sun, Feb 7

TG1999 requested review of D5036: Add Orchestrator.
Sun, Feb 7, 6:58 PM

Tue, Feb 2

TG1999 closed D4931: Add mapping of definitions and harvests.
Tue, Feb 2, 10:47 AM
TG1999 committed rDMFCD9940287285ba: Map a row from clearcode toolkit with software heritage archive (authored by TG1999).
Map a row from clearcode toolkit with software heritage archive
Tue, Feb 2, 10:47 AM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Map a row from clearcode toolkit with software heritage archive

Tue, Feb 2, 10:42 AM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add feature to map a row from clearcode toolkit database with content and revision table from software heritage archive

Tue, Feb 2, 10:37 AM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add feature to map a row from clearcode toolkit database with content and revision table from software heritage archive
This is to build a mechanism to map a row [path(Primary Key), content(binary data), last_modified_date(timestamp with timezone), map_error(error message while mapping), uuid] from clearcode toolkit database, with software heritage archive using content table for sha1 and revision table for sha1_git and extract required information from that row. Then return list of data that has been mapped and mapping status(if able to map every hash of that row, will return True, else return False) so the row that is not being able to map for now can be stored in a state, and can be mapped in future.
Add various exception classes in error.py that can be raised while mapping a row. Check if that row is a definition or harvest and also check if that row does not has invalid path, raise exception if path is invalid. If row is a definiton then map the data using map_definition and if it is a harvest then map it using map_harvest. Use storage instead of sql queries while mapping with the data inside archive. Add tests to cover all the cases and add docstrings to explain how every function works.

Tue, Feb 2, 10:04 AM

Mon, Feb 1

TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Use swh storage instead of sql queries to map sha1 with SWHIDs
Add different type of exceptions in error.py
Add function map_row, and try to check if the ID of the row is valid ID or not, if it is not a valid ID, then raise different exceptions defined in error.py. And if the ID is valid then check if the ID is a definition or harvest.
If the given ID is a definition then map the hashes with the archive (sha1 with content and sha1git with revision) and if able to map the defintion then return True,[(SWHID, TargetType(revision/content), Origin)], otherwise return None.
If the given ID is a harvest, then check the tool of the harvest (licensee, scancode, clearlydefined) and map accordig to the tool (map_licensee, map_scancode, map_clearlydefined), then return mapping status of that harvest (True if every sha1 of that harvest is mapped and False if failed to do it) and a list of data([SWHID,TargetType,None]).
Add tests to cover all the cases

Mon, Feb 1, 6:32 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Use swh storage instead of sql queries to map sha1 with SWHIDs
Add different type of errors in error.py
Add function map_row, and try to check if the ID of the row is valid ID or not, if it is not a valid ID, then raise different exceptions defined in error.py. And if the ID is valid then check if the ID is a definition or harvest.
If the given ID is a definition then map the hashes with the archive (sha1 with content and sha1git with revision) and if able to map the defintion then return True,[(SWHID, TargetType(revision/content), Origin)], otherwise return None.
If the given ID is a harvest, then check the tool of the harvest (licensee, scancode, clearlydefined) and map accordig to the tool (map_licensee, map_scancode, map_clearlydefined), then return mapping status of that harvest (True if every sha1 of that harvest is mapped and False if failed to do it) and a list of data([SWHID,TargetType,None]).
Add tests to cover all the cases

Mon, Feb 1, 5:54 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Use swh storage instead of sql queries to map sha1 with SWHIDs
Add different type of error classes in error.py
Add function map_row, and try to check if the ID of the row is valid ID or not, if it is not a valid ID, then raise different exceptions defined in error.py. And if the ID is valid then check if the ID is a definition or harvest.
If the given ID is a definition then map the hashes with the archive (sha1 with content and sha1git with revision) and return a valid SWHID, TargetType(revision/content), Origin
If the given ID is a harvest, then check the tool of the harvest (licensee, scancode, clearlydefined) and map accordig to the tool (map_licensee, map_scancode, map_clearlydefined), then return mapping status of that harvest (True if every sha1 of that harvest is mapped and False if failed to do it) and a list of data([SWHID,TargetType,None]).
Add tests to cover all the cases"

Mon, Feb 1, 5:48 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not
Add error classes for all conditions where metadata can be wrong or for an invalid ID
Add docstring for error class

Mon, Feb 1, 2:44 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not
Add error classes for all conditions where metadata can be wrong or for an invalid ID

Mon, Feb 1, 11:04 AM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Mon, Feb 1, 10:53 AM

Fri, Jan 29

TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Fri, Jan 29, 5:45 PM
TG1999 added inline comments to D4931: Add mapping of definitions and harvests.
Fri, Jan 29, 4:48 PM
TG1999 added inline comments to D4931: Add mapping of definitions and harvests.
Fri, Jan 29, 4:28 PM
TG1999 added inline comments to D4931: Add mapping of definitions and harvests.
Fri, Jan 29, 4:16 PM
TG1999 added a comment to D4931: Add mapping of definitions and harvests.

@vlorentz made the suggested changes, thanks for a detailed review :)

Fri, Jan 29, 2:40 PM
TG1999 added inline comments to D4931: Add mapping of definitions and harvests.
Fri, Jan 29, 2:39 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Fri, Jan 29, 2:36 PM
TG1999 added inline comments to D4931: Add mapping of definitions and harvests.
Fri, Jan 29, 1:14 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Fri, Jan 29, 7:42 AM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Fri, Jan 29, 7:39 AM

Thu, Jan 28

TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Thu, Jan 28, 8:26 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Thu, Jan 28, 6:15 PM
TG1999 updated subscribers of D4931: Add mapping of definitions and harvests.

Hi @douardda @vlorentz @zack please have a look :)

Thu, Jan 28, 4:15 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping for definitions and harvests
Add functions map_row, map_definition, map_harvest to check whether swh archive is able to map clearlydefined object or not

Thu, Jan 28, 4:12 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping of definitions and harvests

Thu, Jan 28, 3:57 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping of definitions and harvests

Thu, Jan 28, 3:37 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping of definitions and harvests

Thu, Jan 28, 3:01 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping of definitions and harvests

Thu, Jan 28, 1:29 PM
TG1999 retitled D4931: Add mapping of definitions and harvests from Add Orchestrator: Add mapping of definitions and harvests to Add mapping of definitions and harvests.
Thu, Jan 28, 12:39 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping of definitions and harvests

Thu, Jan 28, 12:34 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping of definitions and harvest

Thu, Jan 28, 12:17 PM
TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping of definitions and harvest

Thu, Jan 28, 12:16 PM

Wed, Jan 27

TG1999 updated the diff for D4931: Add mapping of definitions and harvests.

Add mapping of definitions and harvests

Wed, Jan 27, 1:56 PM
TG1999 requested review of D4931: Add mapping of definitions and harvests.
Wed, Jan 27, 1:56 PM

Jan 20 2021

TG1999 closed D4829: Add mapping of sha1 with swh ID.
Jan 20 2021, 10:21 AM
TG1999 committed rDMFCD6d70494b713e: Add mapping of sha1 with SWH ID (authored by TG1999).
Add mapping of sha1 with SWH ID
Jan 20 2021, 10:21 AM
TG1999 updated the diff for D4829: Add mapping of sha1 with swh ID.

Add mapping of sha1 with swh ID

Jan 20 2021, 10:12 AM
TG1999 updated the diff for D4829: Add mapping of sha1 with swh ID.

Add mapping of sha1 with swh ID

Jan 20 2021, 10:03 AM
TG1999 updated the diff for D4829: Add mapping of sha1 with swh ID.

Add mapping of sha1 with swh ID

Jan 20 2021, 7:10 AM

Jan 19 2021

TG1999 updated the diff for D4829: Add mapping of sha1 with swh ID.

Add mapping of sha1 with swh ID

Jan 19 2021, 12:05 PM
TG1999 updated the diff for D4829: Add mapping of sha1 with swh ID.

Add mapping of sha1 with swh ID

Jan 19 2021, 12:04 PM

Jan 18 2021

TG1999 updated the diff for D4829: Add mapping of sha1 with swh ID.

Add mapping of sha1 with swh ID

Jan 18 2021, 7:06 AM