Page MenuHomeSoftware Heritage

Add the possibility to search using a SWHID
ClosedPublic

Authored by jayeshv on Aug 10 2022, 2:50 PM.

Details

Summary

Added a new entrypoint called searchSwhid. Search using a valid
SWHID will return a searchresult connection. The list will have
a single object in most of the cases, but could return multiple items.
Result object will be available in the target field.

Related to T4263

Diff Detail

Repository
rDGQL GraphQL API
Branch
swhid-resolver
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 30743
Build 48067: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 48066: arc lint + arc unit

Event Timeline

jayeshv retitled this revision from WIP Add the possibility to search using a SWHID to WIP: Add the possibility to search using a SWHID.Aug 10 2022, 2:50 PM
jayeshv edited the summary of this revision. (Show Details)

Build is green

Patch application report for D8231 (id=29690)

Rebasing onto 3a1be9a212...

First, rewinding head to replay your work on top of it...
Applying: Add the possibility to search using a SWHID
Changes applied before test
commit b55841856b9063f67060b9e279ab4ee1902e5ff1
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Aug 10 10:01:29 2022 +0200

    Add the possibility to search using a SWHID
    
    Added a new entrypoint called searchSwhid. Search using a valid
    SWHID will return a searchresult connection. The list will have
    a single object in most of the cases, but could return multiple items.
    Result object will be available in the target field.
    
    Related to T4263

See https://jenkins.softwareheritage.org/job/DGQL/job/tests-on-diff/88/ for more details.

Build is green

Patch application report for D8231 (id=29694)

Rebasing onto 3a1be9a212...

First, rewinding head to replay your work on top of it...
Applying: Add the possibility to search using a SWHID
Changes applied before test
commit d042d3e7d7edb0ec945381d626aefc9221462b44
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Aug 10 10:01:29 2022 +0200

    Add the possibility to search using a SWHID
    
    Added a new entrypoint called searchSwhid. Search using a valid
    SWHID will return a searchresult connection. The list will have
    a single object in most of the cases, but could return multiple items.
    Result object will be available in the target field.
    
    Related to T4263

See https://jenkins.softwareheritage.org/job/DGQL/job/tests-on-diff/89/ for more details.

Build is green

Patch application report for D8231 (id=29697)

Rebasing onto 3a1be9a212...

First, rewinding head to replay your work on top of it...
Applying: Add the possibility to search using a SWHID
Changes applied before test
commit 9067f36d97e4a33681e2ff87cceeeb554adb70c4
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Aug 10 10:01:29 2022 +0200

    Add the possibility to search using a SWHID
    
    Added a new entrypoint called searchSwhid. Search using a valid
    SWHID will return a searchresult connection. The list will have
    a single object in most of the cases, but could return multiple items.
    Result object will be available in the target field.
    
    Related to T4263

See https://jenkins.softwareheritage.org/job/DGQL/job/tests-on-diff/90/ for more details.

jayeshv retitled this revision from WIP: Add the possibility to search using a SWHID to Add the possibility to search using a SWHID.Aug 10 2022, 4:35 PM
jayeshv edited the summary of this revision. (Show Details)

Build is green

Patch application report for D8231 (id=29702)

Rebasing onto 3a1be9a212...

First, rewinding head to replay your work on top of it...
Applying: Add the possibility to search using a SWHID
Changes applied before test
commit d80001cae06218c90443c37d2f188e6f4bb13331
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Aug 10 10:01:29 2022 +0200

    Add the possibility to search using a SWHID
    
    Added a new entrypoint called searchSwhid. Search using a valid
    SWHID will return a searchresult connection. The list will have
    a single object in most of the cases, but could return multiple items.
    Result object will be available in the target field.
    
    Related to T4263

See https://jenkins.softwareheritage.org/job/DGQL/job/tests-on-diff/91/ for more details.

Build is green

Patch application report for D8231 (id=29706)

Rebasing onto 3a1be9a212...

First, rewinding head to replay your work on top of it...
Applying: Add the possibility to search using a SWHID
Changes applied before test
commit 17badb128b6e632be9320436a654be982bf743f8
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Aug 10 10:01:29 2022 +0200

    Add the possibility to search using a SWHID
    
    Added a new entrypoint called searchSwhid. Search using a valid
    SWHID will return a searchresult connection. The list will have
    a single object in most of the cases, but could return multiple items.
    Result object will be available in the target field.
    
    Related to T4263

See https://jenkins.softwareheritage.org/job/DGQL/job/tests-on-diff/92/ for more details.

Extra check for object type while returing directory and snapshot

Build is green

Patch application report for D8231 (id=29707)

Rebasing onto 3a1be9a212...

First, rewinding head to replay your work on top of it...
Applying: Add the possibility to search using a SWHID
Changes applied before test
commit 2d9599db194038cc16fce04ba009943933ddea3f
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Aug 10 10:01:29 2022 +0200

    Add the possibility to search using a SWHID
    
    Added a new entrypoint called searchSwhid. Search using a valid
    SWHID will return a searchresult connection. The list will have
    a single object in most of the cases, but could return multiple items.
    Result object will be available in the target field.
    
    Related to T4263

See https://jenkins.softwareheritage.org/job/DGQL/job/tests-on-diff/93/ for more details.

anlambert added inline comments.
swh/graphql/schema/schema.graphql
875

Do we really need that match field ? How will you compute its value for origin search and when using query language ?

swh/graphql/schema/schema.graphql
875

This can be 'null' where we can't compute a value.
I hope it will be useful with search indexes and partial match. In the search (not with swhid, but with a query string), the idea is to sort by match by default.

swh/graphql/schema/schema.graphql
875

I do not think it is needed as we use elasticsearch as a search backend (see swh-search)
and results are already ordered by match.

By the way, I saw that you are using the PostgreSQL search backend in current implementation.
This is deprecated so you should plug swh-search to search for origins and use query language.

swh/graphql/schema/schema.graphql
875

Thanks, I have the swh-search backend implementation in another diff.

Origin search will be possible using the search entrypoint (not implemented yet) or by adding filters to the origins query (this is using Postgres backend at the moment).

Looks good to me but still not really convinced by the match field in search results, we should only rely on the result ordering from the search backend imho.

This revision is now accepted and ready to land.Aug 16 2022, 11:29 AM

Looks good to me but still not really convinced by the match field in search results, we should only rely on the result ordering from the search backend imho.

Thanks. I will remove the field now. Later, we can add an object to return all the metadata from the search. I think the match percentage could be one among them.
From the server, we will always be sorting the data we return with the match order, but it is for the client to use or show these metadata the way they want.

I will introduce this metadata object along with the search entrypoint.
Btw, I will rename the searchSwhid to resolveSwhid as suggested by @vlorentz .

Remove match field
Rename searchSwhid to resolveSwhid

Build is green

Patch application report for D8231 (id=29731)

Rebasing onto 3a1be9a212...

First, rewinding head to replay your work on top of it...
Applying: Add the possibility to search using a SWHID
Changes applied before test
commit 4a12d434ff8a2463c6d5f05e70956e7f31a72d09
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Aug 10 10:01:29 2022 +0200

    Add the possibility to search using a SWHID
    
    Added a new entrypoint called searchSwhid. Search using a valid
    SWHID will return a searchresult connection. The list will have
    a single object in most of the cases, but could return multiple items.
    Result object will be available in the target field.
    
    Related to T4263

See https://jenkins.softwareheritage.org/job/DGQL/job/tests-on-diff/97/ for more details.