Details

Reviewers

Group Reviewers

Maniphest Tasks

T4263: Add a search entrypoint to the GraphQL schema

Summary

Added a new entrypoint called searchSwhid. Search using a valid
SWHID will return a searchresult connection. The list will have
a single object in most of the cases, but could return multiple items.
Result object will be available in the target field.

Related to T4263

Diff Detail

Repository

rDGQL GraphQL API

Branch

swhid-resolver

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 30782
Build 48137: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 48136: arc lint + arc unit

Event Timeline

jayeshv created this revision.Aug 10 2022, 2:50 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptAug 10 2022, 2:50 PM

jayeshv retitled this revision from WIP Add the possibility to search using a SWHID to WIP: Add the possibility to search using a SWHID.Aug 10 2022, 2:50 PM

jayeshv edited the summary of this revision. (Show Details)

Build is green

Patch application report for D8231 (id=29690)

Rebasing onto 3a1be9a212...

First, rewinding head to replay your work on top of it...
Applying: Add the possibility to search using a SWHID

Changes applied before test

commit b55841856b9063f67060b9e279ab4ee1902e5ff1
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Aug 10 10:01:29 2022 +0200

    Add the possibility to search using a SWHID
    
    Added a new entrypoint called searchSwhid. Search using a valid
    SWHID will return a searchresult connection. The list will have
    a single object in most of the cases, but could return multiple items.
    Result object will be available in the target field.
    
    Related to T4263

See https://jenkins.softwareheritage.org/job/DGQL/job/tests-on-diff/88/ for more details.

Harbormaster completed remote builds in B30743: Diff 29690.Aug 10 2022, 2:52 PM

jayeshv requested review of this revision.Aug 10 2022, 2:52 PM

Add functional tests

Build is green

Patch application report for D8231 (id=29694)

Rebasing onto 3a1be9a212...

First, rewinding head to replay your work on top of it...
Applying: Add the possibility to search using a SWHID

Changes applied before test

commit d042d3e7d7edb0ec945381d626aefc9221462b44
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Aug 10 10:01:29 2022 +0200

    Add the possibility to search using a SWHID
    
    Added a new entrypoint called searchSwhid. Search using a valid
    SWHID will return a searchresult connection. The list will have
    a single object in most of the cases, but could return multiple items.
    Result object will be available in the target field.
    
    Related to T4263

See https://jenkins.softwareheritage.org/job/DGQL/job/tests-on-diff/89/ for more details.

Harbormaster completed remote builds in B30747: Diff 29694.Aug 10 2022, 3:50 PM

schema updates

Build is green

Patch application report for D8231 (id=29697)

Rebasing onto 3a1be9a212...

First, rewinding head to replay your work on top of it...
Applying: Add the possibility to search using a SWHID

Changes applied before test

commit 9067f36d97e4a33681e2ff87cceeeb554adb70c4
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Aug 10 10:01:29 2022 +0200

    Add the possibility to search using a SWHID
    
    Added a new entrypoint called searchSwhid. Search using a valid
    SWHID will return a searchresult connection. The list will have
    a single object in most of the cases, but could return multiple items.
    Result object will be available in the target field.
    
    Related to T4263

See https://jenkins.softwareheritage.org/job/DGQL/job/tests-on-diff/90/ for more details.

Harbormaster completed remote builds in B30751: Diff 29697.Aug 10 2022, 4:35 PM

jayeshv retitled this revision from WIP: Add the possibility to search using a SWHID to Add the possibility to search using a SWHID.Aug 10 2022, 4:35 PM

jayeshv edited the summary of this revision. (Show Details)

doc string update

Build is green

Patch application report for D8231 (id=29702)

Rebasing onto 3a1be9a212...

First, rewinding head to replay your work on top of it...
Applying: Add the possibility to search using a SWHID

Changes applied before test

commit d80001cae06218c90443c37d2f188e6f4bb13331
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Aug 10 10:01:29 2022 +0200

    Add the possibility to search using a SWHID
    
    Added a new entrypoint called searchSwhid. Search using a valid
    SWHID will return a searchresult connection. The list will have
    a single object in most of the cases, but could return multiple items.
    Result object will be available in the target field.
    
    Related to T4263

See https://jenkins.softwareheritage.org/job/DGQL/job/tests-on-diff/91/ for more details.

Harbormaster completed remote builds in B30756: Diff 29702.Aug 10 2022, 6:48 PM

bug fix

Build is green

Patch application report for D8231 (id=29706)

Rebasing onto 3a1be9a212...

First, rewinding head to replay your work on top of it...
Applying: Add the possibility to search using a SWHID

Changes applied before test

commit 17badb128b6e632be9320436a654be982bf743f8
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Aug 10 10:01:29 2022 +0200

    Add the possibility to search using a SWHID
    
    Added a new entrypoint called searchSwhid. Search using a valid
    SWHID will return a searchresult connection. The list will have
    a single object in most of the cases, but could return multiple items.
    Result object will be available in the target field.
    
    Related to T4263

See https://jenkins.softwareheritage.org/job/DGQL/job/tests-on-diff/92/ for more details.

Harbormaster completed remote builds in B30760: Diff 29706.Aug 11 2022, 9:23 AM

Extra check for object type while returing directory and snapshot

Build is green

Patch application report for D8231 (id=29707)

Rebasing onto 3a1be9a212...

First, rewinding head to replay your work on top of it...
Applying: Add the possibility to search using a SWHID

Changes applied before test

commit 2d9599db194038cc16fce04ba009943933ddea3f
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Aug 10 10:01:29 2022 +0200

    Add the possibility to search using a SWHID
    
    Added a new entrypoint called searchSwhid. Search using a valid
    SWHID will return a searchresult connection. The list will have
    a single object in most of the cases, but could return multiple items.
    Result object will be available in the target field.
    
    Related to T4263

See https://jenkins.softwareheritage.org/job/DGQL/job/tests-on-diff/93/ for more details.

Harbormaster completed remote builds in B30761: Diff 29707.Aug 11 2022, 10:51 AM

anlambert added a subscriber: anlambert.Aug 11 2022, 11:25 AM

anlambert added inline comments.

swh/graphql/schema/schema.graphql
875	Do we really need that match field ? How will you compute its value for origin search and when using query language ?

jayeshv added inline comments.Aug 11 2022, 11:30 AM

swh/graphql/schema/schema.graphql
875	This can be 'null' where we can't compute a value. I hope it will be useful with search indexes and partial match. In the search (not with swhid, but with a query string), the idea is to sort by match by default.

anlambert added inline comments.Aug 11 2022, 11:37 AM

swh/graphql/schema/schema.graphql
875	I do not think it is needed as we use elasticsearch as a search backend (see swh-search) and results are already ordered by match. By the way, I saw that you are using the PostgreSQL search backend in current implementation. This is deprecated so you should plug `swh-search` to search for origins and use query language.

jayeshv added inline comments.Aug 11 2022, 11:45 AM

swh/graphql/schema/schema.graphql
875	Thanks, I have the swh-search backend implementation in another diff. Origin search will be possible using the search entrypoint (not implemented yet) or by adding filters to the origins query (this is using Postgres backend at the moment).

Looks good to me but still not really convinced by the match field in search results, we should only rely on the result ordering from the search backend imho.

This revision is now accepted and ready to land.Aug 16 2022, 11:29 AM

In D8231#214340, @anlambert wrote:

Looks good to me but still not really convinced by the match field in search results, we should only rely on the result ordering from the search backend imho.

Thanks. I will remove the field now. Later, we can add an object to return all the metadata from the search. I think the match percentage could be one among them.
From the server, we will always be sorting the data we return with the match order, but it is for the client to use or show these metadata the way they want.

I will introduce this metadata object along with the search entrypoint.
Btw, I will rename the searchSwhid to resolveSwhid as suggested by @vlorentz .

Remove match field
Rename searchSwhid to resolveSwhid

Build is green

Patch application report for D8231 (id=29731)

Rebasing onto 3a1be9a212...

First, rewinding head to replay your work on top of it...
Applying: Add the possibility to search using a SWHID

Changes applied before test

commit 4a12d434ff8a2463c6d5f05e70956e7f31a72d09
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Aug 10 10:01:29 2022 +0200

    Add the possibility to search using a SWHID
    
    Added a new entrypoint called searchSwhid. Search using a valid
    SWHID will return a searchresult connection. The list will have
    a single object in most of the cases, but could return multiple items.
    Result object will be available in the target field.
    
    Related to T4263

See https://jenkins.softwareheritage.org/job/DGQL/job/tests-on-diff/97/ for more details.

Harbormaster completed remote builds in B30782: Diff 29731.Aug 16 2022, 11:54 AM

merged

Add the possibility to search using a SWHID
ClosedPublic
Actions

Details

Diff Detail

Event Timeline

Patch application report for D8231 (id=29690)

Changes applied before test

Patch application report for D8231 (id=29694)

Changes applied before test

Patch application report for D8231 (id=29697)

Changes applied before test

Patch application report for D8231 (id=29702)

Changes applied before test

Patch application report for D8231 (id=29706)

Changes applied before test

Patch application report for D8231 (id=29707)

Changes applied before test

Patch application report for D8231 (id=29731)

Changes applied before test

Revision Contents
Changeset List

Diff 29731

swh/graphql/app.py

swh/graphql/backends/archive.py

swh/graphql/resolvers/directory.py

swh/graphql/resolvers/resolver_factory.py

swh/graphql/resolvers/resolvers.py

swh/graphql/resolvers/search.py

swh/graphql/resolvers/snapshot.py

swh/graphql/schema/schema.graphql

swh/graphql/tests/data.py

swh/graphql/tests/functional/test_swhid_resolve.py

swh/graphql/tests/unit/resolvers/test_resolver_factory.py

Add the possibility to search using a SWHIDClosedPublicActions

Details

Diff Detail

Event Timeline

Patch application report for D8231 (id=29690)

Changes applied before test

Patch application report for D8231 (id=29694)

Changes applied before test

Patch application report for D8231 (id=29697)

Changes applied before test

Patch application report for D8231 (id=29702)

Changes applied before test

Patch application report for D8231 (id=29706)

Changes applied before test

Patch application report for D8231 (id=29707)

Changes applied before test

Patch application report for D8231 (id=29731)

Changes applied before test

Revision ContentsChangeset List

Diff 29731

swh/graphql/app.py

swh/graphql/backends/archive.py

swh/graphql/resolvers/directory.py

swh/graphql/resolvers/resolver_factory.py

swh/graphql/resolvers/resolvers.py

swh/graphql/resolvers/search.py

swh/graphql/resolvers/snapshot.py

swh/graphql/schema/schema.graphql

swh/graphql/tests/data.py

swh/graphql/tests/functional/test_swhid_resolve.py

swh/graphql/tests/unit/resolvers/test_resolver_factory.py

Add the possibility to search using a SWHID
ClosedPublic
Actions

Revision Contents
Changeset List