Ensure the responses that are streamed by the graph service are also streamed in
the Web API endpoint proxying it.
Closes T2900
Details
Details
- Reviewers
vlorentz - Group Reviewers
Reviewers - Maniphest Tasks
- T2900: Public graph/ API does not handle streaming results from endpoints
- Commits
- rDWAPPSe605c3fa701a: api/graph: Stream responses as in the proxied graph service
Diff Detail
Diff Detail
- Repository
- rDWAPPS Web applications
- Lint
Automatic diff as part of commit; lint not applicable. - Unit
Automatic diff as part of commit; unit tests not applicable.
Event Timeline
Comment Actions
Build is green
Patch application report for D4824 (id=17078)
Rebasing onto 2ed38516a4...
Current branch diff-target is up to date.
Changes applied before test
commit c834a81ee608a9a1fb258de2d4a991526c197033
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date: Thu Jan 7 18:57:15 2021 +0100
api/graph: Stream responses as in the proxied graph service
Ensure the responses that are streamed by the graph service are also streamed in
the Web API endpoint proxying it.
Closes T2900See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/547/ for more details.
Comment Actions
Requests execution time seems similar using the streaming approach (see below, first curl command receives a streamed response, I used the costly request from T2900).
(swh) ✔ ~/swh/swh-environment/swh-web [api-graph-streaming-response L|✚ 1⚑ 28]
12:56 $ time curl http://localhost:5004/api/1/graph/visit/edges/swh:1:rev:f39d7d78b70e0f39facb1e4fab77ad3df5c52a35/?edges=rev:rev >/dev/null
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 76.0M 0 76.0M 0 0 4226k 0 --:--:-- 0:00:18 --:--:-- 4223k
real 0m18,438s
user 0m0,965s
sys 0m4,006s
(swh) ✔ ~/swh/swh-environment/swh-web [api-graph-streaming-response L|✚ 1⚑ 28]
12:57 $ time curl -H "Authorization: Bearer ${TOKEN}" https://archive.softwareheritage.org/api/1/graph/visit/edges/swh:1:rev:f39d7d78b70e0f39facb1e4fab77ad3df5c52a35/?edges=rev:rev >/dev/null
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 76.0M 100 76.0M 0 0 3473k 0 0:00:22 0:00:22 --:--:-- 18.1M
real 0m22,432s
user 0m0,210s
sys 0m0,131sBased on my understanding, that change is needed to improve swh-fuse.
Comment Actions
Build is green
Patch application report for D4824 (id=17089)
Rebasing onto 6f8b844070...
Current branch diff-target is up to date.
Changes applied before test
commit e605c3fa701ac3df3bddba0c9324444084db462d
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date: Thu Jan 7 18:57:15 2021 +0100
api/graph: Stream responses as in the proxied graph service
Ensure the responses that are streamed by the graph service are also streamed in
the Web API endpoint proxying it.
Closes T2900See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/548/ for more details.