Page MenuHomeSoftware Heritage

WIP: FUSE: use graph/ stream API to retrieve history
Changes PlannedPublicDraft

Authored by haltode on Jan 10 2021, 12:20 PM.

Details

Summary

WIP, closes T2724.

This diff was my starting work on solving the task, since quite a lot of changes
are required to solve it. What remains to be done:

  • Add a buffer for data received by the iter_content() requests method to ensure the data flow is only cut after a complete line (and not randomly)
  • Configure the mocking offline tests to use a stream API (similar to what is done in swh-web tests)
  • Add a history/.status file similar to the existing history/by-date/.status
  • Make sure by-date/ by-hash/ by-page/ do not rely on having complete history data

Diff Detail

Repository
rDFUSE FUSE virtual file system
Branch
feature/stream-history-api
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 18263
Build 28196: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 28195: arc lint + arc unit

Event Timeline

Build has FAILED

Patch application report for D4833 (id=17120)

Rebasing onto a75335972e...

Current branch diff-target is up to date.
Changes applied before test
commit 4fe50a2915b20c3b8e6b2a0c9fa6169021728b3e
Author: Thibault Allançon <haltode@gmail.com>
Date:   Wed Jan 6 16:27:25 2021 +0100

    WIP

Link to build: https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/264/
See console output for more information: https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/264/console

Build has FAILED

Patch application report for D4833 (id=17121)

Rebasing onto a75335972e...

Current branch diff-target is up to date.
Changes applied before test
commit 922a110a59bbb590e16bc4ff855e170a3011e7f3
Author: Thibault Allançon <haltode@gmail.com>
Date:   Wed Jan 6 16:27:25 2021 +0100

    WIP

Link to build: https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/265/
See console output for more information: https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/265/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 10 2021, 12:26 PM
Harbormaster failed remote builds in B18263: Diff 17121!