Adds an API route for fetching objects as Git objects
This route is needed for the implementation of the SWH Archive-IPFS
bridge. Specifically, we need a way to, given a SWHID, fetch the exact
data that was used to hash an object.
Differential D7192
Route for fetching Git-encoded objects Ericson2314 on Feb 16 2022, 5:50 PM. Authored by
Details
Adds an API route for fetching objects as Git objects This route is needed for the implementation of the SWH Archive-IPFS
Diff Detail
Event TimelineThere are a very large number of changes, so older changes are hidden. Show Older Changes Comment Actions Build is green Patch application report for D7192 (id=27870)Rebasing onto 25bf6ca843... Current branch diff-target is up to date. Changes applied before testcommit db6856571264cdc14ced37ad895643b1d8435ec5 Author: Amélia Liao <me@amelia.how> Date: Mon Apr 25 19:56:26 2022 -0400 Adds an API route for fetching objects as Git objects This route is needed for the implementation of the SWH Archive-IPFS bridge. Specifically, we need a way to, given a SWHID, fetch the exact data that was used to hash an object. The design follows the plan @vlorentz put forth in https://sympa.inria.fr/sympa/arc/swh-devel/2022-04/msg00042.html Differential Revision: https://forge.softwareheritage.org/D7192 See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1755/ for more details. Comment Actions Build is green Patch application report for D7192 (id=27871)Rebasing onto 25bf6ca843... Current branch diff-target is up to date. Changes applied before testcommit 71c3e064a272a6f1e5a4a1f0668930e26684216f Author: Amélia Liao <me@amelia.how> Date: Mon Apr 25 19:56:26 2022 -0400 Adds an API route for fetching objects as Git objects This route is needed for the implementation of the SWH Archive-IPFS bridge. Specifically, we need a way to, given a SWHID, fetch the exact data that was used to hash an object. The design follows the plan @vlorentz put forth in https://sympa.inria.fr/sympa/arc/swh-devel/2022-04/msg00042.html commit 241f90ba38c26924aef64b62e2d318a51d4df274 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:48:35 2022 -0400 Extend `swh.web.tests.data` with new functions - `random_sha1_bytes` - `random_object_type` - `random_swhid` The first two are used to make the third, but useful in their own right. The third will be used in the following commit. commit b5f20ca9ddfcc2745a76a715e4c892c19c0d0942 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:46:37 2022 -0400 Deduplicate `SWHID_RE` Put it in `swh.web.common.utils` and import from there. See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1756/ for more details. Comment Actions Looks good to me, thanks.
Comment Actions Build has FAILED Patch application report for D7192 (id=27890)Rebasing onto 25bf6ca843... Current branch diff-target is up to date. Changes applied before testcommit e87641d4ab986ef89c646cbc00756aea0f9bd3da Author: Amélia Liao <me@amelia.how> Date: Mon Apr 25 19:56:26 2022 -0400 Adds an API route for fetching objects as Git objects This route is needed for the implementation of the SWH Archive-IPFS bridge. Specifically, we need a way to, given a SWHID, fetch the exact data that was used to hash an object. The design follows the plan @vlorentz put forth in https://sympa.inria.fr/sympa/arc/swh-devel/2022-04/msg00042.html commit 40b0fa4dfa7b571b3374a344273d7b9db83a0042 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:48:35 2022 -0400 Extend test utits with new functions In `swh.web.tests.data`: - `random_sha1_bytes - `random_object_type` - `random_swhid` In `swh.web.tests.conftest`: - `unknown_core_swhid`, a fixture commit b5f20ca9ddfcc2745a76a715e4c892c19c0d0942 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:46:37 2022 -0400 Deduplicate `SWHID_RE` Put it in `swh.web.common.utils` and import from there. Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1758/ Comment Actions Build has FAILED Patch application report for D7192 (id=27891)Rebasing onto 25bf6ca843... Current branch diff-target is up to date. Changes applied before testcommit 291e668fb9bb863936ec2e8d85e12ab6a86f25f9 Author: Amélia Liao <me@amelia.how> Date: Mon Apr 25 19:56:26 2022 -0400 Adds an API route for fetching objects as Git objects This route is needed for the implementation of the SWH Archive-IPFS bridge. Specifically, we need a way to, given a SWHID, fetch the exact data that was used to hash an object. The design follows the plan @vlorentz put forth in https://sympa.inria.fr/sympa/arc/swh-devel/2022-04/msg00042.html commit 40b0fa4dfa7b571b3374a344273d7b9db83a0042 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:48:35 2022 -0400 Extend test utits with new functions In `swh.web.tests.data`: - `random_sha1_bytes - `random_object_type` - `random_swhid` In `swh.web.tests.conftest`: - `unknown_core_swhid`, a fixture commit b5f20ca9ddfcc2745a76a715e4c892c19c0d0942 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:46:37 2022 -0400 Deduplicate `SWHID_RE` Put it in `swh.web.common.utils` and import from there. Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1759/ Comment Actions Build is green Patch application report for D7192 (id=27894)Rebasing onto 25bf6ca843... Current branch diff-target is up to date. Changes applied before testcommit b70379b50ba8ed6a76dc62dd45c6194f1398f142 Author: Amélia Liao <me@amelia.how> Date: Mon Apr 25 19:56:26 2022 -0400 Adds an API route for fetching objects as Git objects This route is needed for the implementation of the SWH Archive-IPFS bridge. Specifically, we need a way to, given a SWHID, fetch the exact data that was used to hash an object. The design follows the plan @vlorentz put forth in https://sympa.inria.fr/sympa/arc/swh-devel/2022-04/msg00042.html commit 725942c027d8010e587a7e15908002c88fcc225e Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:48:35 2022 -0400 Extend test utits with new functions In `swh.web.tests.data`: - `random_sha1_bytes - `random_object_type` - `random_swhid` In `swh.web.tests.conftest`: - `unknown_core_swhid`, a fixture commit b5f20ca9ddfcc2745a76a715e4c892c19c0d0942 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:46:37 2022 -0400 Deduplicate `SWHID_RE` Put it in `swh.web.common.utils` and import from there. See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1760/ for more details.
Comment Actions LGTM. @anlambert Good for you too?
Comment Actions Build was aborted Patch application report for D7192 (id=27895)Rebasing onto 25bf6ca843... Current branch diff-target is up to date. Changes applied before testcommit b5c42d617dafc6562a7e31f416fcfe3f8506e35d Author: Amélia Liao <me@amelia.how> Date: Mon Apr 25 19:56:26 2022 -0400 Adds an API route for fetching objects as Git objects This route is needed for the implementation of the SWH Archive-IPFS bridge. Specifically, we need a way to, given a SWHID, fetch the exact data that was used to hash an object. The design follows the plan @vlorentz put forth in https://sympa.inria.fr/sympa/arc/swh-devel/2022-04/msg00042.html commit e48569769b9468dcdd54ba5eb73045a642c5291f Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:48:35 2022 -0400 Extend test utits with new functions In `swh.web.tests.data`: - `random_sha1_bytes In `swh.web.tests.conftest`: - `unknown_core_swhid`, a pytest "fixture" commit b5f20ca9ddfcc2745a76a715e4c892c19c0d0942 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:46:37 2022 -0400 Deduplicate `SWHID_RE` Put it in `swh.web.common.utils` and import from there. Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1761/ Comment Actions I'll wait for now pending @anlambert review, but let me know if I should push again to to trigger a new build after the last one seems to have died spuriously (if I am not misunderstanding). Comment Actions Push same stuff again to restart job Figure we're about to be asleep in both our timezones, so might as well :). Comment Actions Build was aborted Patch application report for D7192 (id=27896)Rebasing onto 25bf6ca843... Current branch diff-target is up to date. Changes applied before testcommit 51e99a5707e01f2d6b800a04ac2f85e858199dbd Author: Amélia Liao <me@amelia.how> Date: Mon Apr 25 19:56:26 2022 -0400 Adds an API route for fetching objects as Git objects This route is needed for the implementation of the SWH Archive-IPFS bridge. Specifically, we need a way to, given a SWHID, fetch the exact data that was used to hash an object. The design follows the plan @vlorentz put forth in https://sympa.inria.fr/sympa/arc/swh-devel/2022-04/msg00042.html commit e48569769b9468dcdd54ba5eb73045a642c5291f Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:48:35 2022 -0400 Extend test utits with new functions In `swh.web.tests.data`: - `random_sha1_bytes In `swh.web.tests.conftest`: - `unknown_core_swhid`, a pytest "fixture" commit b5f20ca9ddfcc2745a76a715e4c892c19c0d0942 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:46:37 2022 -0400 Deduplicate `SWHID_RE` Put it in `swh.web.common.utils` and import from there. Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1762/ Comment Actions OK thanks, well, hopefully we can sort it out and get the second review French business hours next week :) I've made drafts of a) the IPFS plugin using this new route b) snapshot encoding and decoding on the IPFS side, so once I can test those this we should be close to finished! Comment Actions Thanks. Two things I forgot below:
Comment Actions Build was aborted Patch application report for D7192 (id=27919)Rebasing onto 25bf6ca843... Current branch diff-target is up to date. Changes applied before testcommit 51e99a5707e01f2d6b800a04ac2f85e858199dbd Author: Amélia Liao <me@amelia.how> Date: Mon Apr 25 19:56:26 2022 -0400 Adds an API route for fetching objects as Git objects This route is needed for the implementation of the SWH Archive-IPFS bridge. Specifically, we need a way to, given a SWHID, fetch the exact data that was used to hash an object. The design follows the plan @vlorentz put forth in https://sympa.inria.fr/sympa/arc/swh-devel/2022-04/msg00042.html commit e48569769b9468dcdd54ba5eb73045a642c5291f Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:48:35 2022 -0400 Extend test utits with new functions In `swh.web.tests.data`: - `random_sha1_bytes In `swh.web.tests.conftest`: - `unknown_core_swhid`, a pytest "fixture" commit b5f20ca9ddfcc2745a76a715e4c892c19c0d0942 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:46:37 2022 -0400 Deduplicate `SWHID_RE` Put it in `swh.web.common.utils` and import from there. Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1763/
Comment Actions Build was aborted Patch application report for D7192 (id=27920)Rebasing onto 25bf6ca843... Current branch diff-target is up to date. Changes applied before testcommit fe12bd286e8ce563205d41a0d3b154a6e5d766c8 Author: Amélia Liao <me@amelia.how> Date: Mon Apr 25 19:56:26 2022 -0400 Adds an API route for fetching objects as Git objects This route is needed for the implementation of the SWH Archive-IPFS bridge. Specifically, we need a way to, given a SWHID, fetch the exact data that was used to hash an object. The design follows the plan @vlorentz put forth in https://sympa.inria.fr/sympa/arc/swh-devel/2022-04/msg00042.html commit 7277f4d8f46296003a779245448fb48fb89a4b2a Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:48:35 2022 -0400 Extend test utits with new functions In `swh.web.tests.data`: - `random_sha1_bytes In `swh.web.tests.conftest`: - `unknown_core_swhid`, a pytest "fixture" commit b5f20ca9ddfcc2745a76a715e4c892c19c0d0942 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:46:37 2022 -0400 Deduplicate `SWHID_RE` Put it in `swh.web.common.utils` and import from there. Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1764/ Comment Actions Build has FAILED Patch application report for D7192 (id=27921)Rebasing onto 25bf6ca843... Current branch diff-target is up to date. Changes applied before testcommit b2f55a240c44b40768de0cffa759ce40f019a0c7 Author: Amélia Liao <me@amelia.how> Date: Mon Apr 25 19:56:26 2022 -0400 Adds an API route for fetching objects as Git objects This route is needed for the implementation of the SWH Archive-IPFS bridge. Specifically, we need a way to, given a SWHID, fetch the exact data that was used to hash an object. The design follows the plan @vlorentz put forth in https://sympa.inria.fr/sympa/arc/swh-devel/2022-04/msg00042.html commit 7277f4d8f46296003a779245448fb48fb89a4b2a Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:48:35 2022 -0400 Extend test utits with new functions In `swh.web.tests.data`: - `random_sha1_bytes In `swh.web.tests.conftest`: - `unknown_core_swhid`, a pytest "fixture" commit b5f20ca9ddfcc2745a76a715e4c892c19c0d0942 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:46:37 2022 -0400 Deduplicate `SWHID_RE` Put it in `swh.web.common.utils` and import from there. Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1765/
Comment Actions D7722 landed and released, you can now use that new storage.algos.directory.directory_get() function. Comment Actions Use new swh.storage.algos.directory.directory_get Rebase on lastest master, where cypress CI issue is hopefully fixed! Comment Actions Build has FAILED Patch application report for D7192 (id=27953)Rebasing onto 4a6bc69b2f... Current branch diff-target is up to date. Changes applied before testcommit 6de4c34b57751fa0284644dbae940051fce2a7bf Author: Amélia Liao <me@amelia.how> Date: Mon Apr 25 19:56:26 2022 -0400 Adds an API route for fetching objects as Git objects This route is needed for the implementation of the SWH Archive-IPFS bridge. Specifically, we need a way to, given a SWHID, fetch the exact data that was used to hash an object. The design follows the plan @vlorentz put forth in https://sympa.inria.fr/sympa/arc/swh-devel/2022-04/msg00042.html commit 055741dd3b7d340833e862951f26407fbe3d2497 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:48:35 2022 -0400 Extend test utits with new functions In `swh.web.tests.data`: - `random_sha1_bytes In `swh.web.tests.conftest`: - `unknown_core_swhid`, a pytest "fixture" commit a4d06a29323a1d26061e23dfa126d983b2cdc955 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:46:37 2022 -0400 Deduplicate `SWHID_RE` Put it in `swh.web.common.utils` and import from there. Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1773/ Comment Actions Build is green Patch application report for D7192 (id=27953)Rebasing onto 4a6bc69b2f... Current branch diff-target is up to date. Changes applied before testcommit 6de4c34b57751fa0284644dbae940051fce2a7bf Author: Amélia Liao <me@amelia.how> Date: Mon Apr 25 19:56:26 2022 -0400 Adds an API route for fetching objects as Git objects This route is needed for the implementation of the SWH Archive-IPFS bridge. Specifically, we need a way to, given a SWHID, fetch the exact data that was used to hash an object. The design follows the plan @vlorentz put forth in https://sympa.inria.fr/sympa/arc/swh-devel/2022-04/msg00042.html commit 055741dd3b7d340833e862951f26407fbe3d2497 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:48:35 2022 -0400 Extend test utits with new functions In `swh.web.tests.data`: - `random_sha1_bytes In `swh.web.tests.conftest`: - `unknown_core_swhid`, a pytest "fixture" commit a4d06a29323a1d26061e23dfa126d983b2cdc955 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:46:37 2022 -0400 Deduplicate `SWHID_RE` Put it in `swh.web.common.utils` and import from there. See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1775/ for more details. Comment Actions @vlorentz is anyone else (@anlambert?) going to review, or should I push this once it passes again? Comment Actions Build is green Patch application report for D7192 (id=27962)Rebasing onto d9fb8522ed... Current branch diff-target is up to date. Changes applied before testcommit 25a999821b3fe680f7a9c7138c058a102b807c3a Author: Amélia Liao <me@amelia.how> Date: Mon Apr 25 19:56:26 2022 -0400 Adds an API route for fetching objects as Git objects This route is needed for the implementation of the SWH Archive-IPFS bridge. Specifically, we need a way to, given a SWHID, fetch the exact data that was used to hash an object. The design follows the plan @vlorentz put forth in https://sympa.inria.fr/sympa/arc/swh-devel/2022-04/msg00042.html commit 9cd95df4775c9ba0de0e0465654c6489b96f9c77 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:48:35 2022 -0400 Extend test utits with new functions In `swh.web.tests.data`: - `random_sha1_bytes In `swh.web.tests.conftest`: - `unknown_core_swhid`, a pytest "fixture" commit 43b197d8357ffbdcd21afe393bb4c2d9fcd94a16 Author: John Ericson <John.Ericson@Obsidian.Systems> Date: Wed Apr 27 18:46:37 2022 -0400 Deduplicate `SWHID_RE` Put it in `swh.web.common.utils` and import from there. See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1778/ for more details. Comment Actions Eh, I'll just assume, given the timezone latency, that going down to one review was intentional. (With the directory algorithm factored out into swh-storage, this patch is now simpler and more "just do the obvious thing" too.) Comment Actions (Post merge, the diff that shows up is just the first commit, but they all are on master.) Comment Actions Ah. and I guess there needs to be another round of tagging and whatnot to trigger a website redeploy? |