The previous push didn't work correctly and the commit history were erased.
Details
Details
- Reviewers
zack ardumont - Group Reviewers
Reviewers - Maniphest Tasks
- T460: Extract the objstorage component from storage module to an independent module
- Commits
- rDOBJS3662fc28456c: Revert "Client to backup contents to an objstorage server instance"
rDOBJS84bc789758a6: Revert "Client to backup contents to an objstorage server instance"
rDOBJS3226739f3e9e: exc: Create a new exception type specific to the object storage. Also, add…
rDOBJS3ed1894784b1: Move all files to the correct repository swh/objstorage
rDOBJS8e79ecc58f5f: Import scripts from swh.storage
rDOBJS24993f350253: Move tests to the correct location
rDOBJS89bf2a98d033: imports: correction due to the module extraction
rDOBJS601d984e0568: Move the swh.storage.checker module into swh.objstorage.checker as this…
rDOBJS3effd252a53d: api: Move the http api-related code from storage to objstorage to keep the…
rDOBJS3b808772841b: objstorage.multiplexer: add some use case example to the…
rDOBJS78e4f662baf5: Update debian packaging config
rDOBJS38eac8bb167c: objstorage.multiplexer: add tests for the multiplexer object storage
rDOBJSa386b9d90ab8: objstorage: move the constant ID_HASH_ALGO to ObjStorage
rDOBJS8501d47df55a: Open (and fix) swh.storage.storage.revision_log_by
rDOBJS5a145eeccda0: objstorage.multiplexer.filter: Implement the multiplexer filter mecanism
rDOBJSf99f8197a018: objstorage.multiplexer: Implement the object storage multiplexer
rDOBJSaf20300a3870: objstorage.multiplexer.filter: add tests for the filters implementations
rDOBJSfce4177991af: objstorage.multiplexer.filter: add a way to easily create filter chaining
rDOBJS662fef2e7175: test_archiver: remove debug prints
rDOBJS946073fc88e2: Create a base API that define the objstorage behavior
rDOBJSa157a6d47970: Update the content integrity checker in order to follow objstorage changes
rDOBJS3a05041faa2f: Add an implementation of the object storage api and a test class
rDOBJSafc53da694eb: archiver: change objstorage initialization into the archiver
rDOBJSf14c963076b2: Fix an object id encoding into a test of objstorage api
rDOBJS9deabcbdd67b: Create a base API that define the objstorage behavior
rDOBJSc3f12b0de6c3: Improve content integrity checker's test coverage
rDOBJSb86631ce7af0: Add an implementation of the object storage api and a test class
rDOBJS4f8327f43ccf: objstorage: PathslicingObjStorage now allow arbitrary slicing
rDOBJS38f3b7eec82f: objstorage.api: Change instantiation of the object storage
rDOBJS8aa04719df7c: objstorage: PathslicingObjStorage now allow arbitrary slicing
rDOBJS43c5cdb25afe: Change the objstorage http server in order to follow the changes of the local…
rDOBJS639096443f98: checker: change the objstorage initialization of the checker
rDOBJS802832c36dba: converters: update to reflect decoding logic moved to swh.core.utils
rDOBJS89a115c74098: The checker now have the possibility to have multiple backup servers
rDOBJS5f9bc2686a0a: Change the objstorage http server in order to follow the changes of the local…
rDOBJS46f6641ed963: objstorage.api: Change instantiation of the object storage
rDOBJSe15367114f58: Add run_from_webserver function for objstorage api server
rDOBJS79ca04e1e05b: Also, add a get_random_contents access to the remote API
rDOBJS900e5d166ae3: Add some methods to the object storage in order to allow a content integrity…
rDOBJS4156533a6193: Add some methods to the object storage in order to allow a content integrity…
rDOBJS1ec84256ea10: Add some tests for the content integrity checker
rDOBJS689dcf66197a: Command line launch improvements
rDOBJSab064e3f4c80: Add unique identifier message on default api server route endpoints
rDOBJS547c219872c1: Also, add a get_random_contents access to the remote API
rDOBJS6376a2aeffdc: Change test attribute in order to skip some tests
rDOBJS553df58f2514: Add synchronous first implementation of the archiver
rDOBJSc11a89f44c23: Make the worker asynchronous optional
rDOBJSe0667961d26e: Improve test coverage of the archiver
rDOBJS5605f85df596: storage: add entity_get_one to retrieve a single entity
rDOBJS21b7717ec6fa: sql/swh-func: drop temporary entity-related tables on commit
rDOBJSd744acd7cb7e: Change the objstorage file to a module
rDOBJScc21d001c60d: Add tests for the remote objstorage API and share some code with storage.api…
rDOBJS43e8918c74fa: db: allow passing UUIDs back and forth with the database
rDOBJS355e30dabdbd: sql/swh-schema: add a fullname field to person
rDOBJSf4f7709977aa: Add an http API for objstorage and refactor the shared code with storage.api
rDOBJS3bb3c4d9097c: sql/swh-schema: rename entity{_history}.doap to metadata
rDOBJSd502912ed606: swh-schema: fold the entity.lister column in entity.lister_metadata
rDOBJS59ca69809b00: revision_log from multiple root revisions
rDOBJS349d282882a9: swh.storage: Allow storage of extra git headers in revisions
rDOBJS35a49a0b32a0: Open entity_get api
rDOBJS807d436a679f: Open content_missing_per_sha1
rDOBJS7e1232514210: Open occurrence_get(origin_id) to retrieve latest occurrences per origin
rDOBJS24a4163afadd: Deal with occurrence_history.branch, release.name as bytes
rDOBJSf708c2504046: Rename directory_get to directory_ls
rDOBJS728ddf49f266: Let the person's id flow
rDOBJSb818e0d6a837: Open directory_get to retrieve information on directory by id
rDOBJSc1f45f71f020: sql/swh-schema: Handle negative UTC offsets in revision and release dates
rDOBJS23a48526e4d2: storage: add new method object_find_by_sha1_git
rDOBJS4fd2682a1e46: revision_get_by: branch name filtering is optional
rDOBJSf10c61f16897: Wrap miscellaneous psycopg2 failures for client in StorageBackendError
rDOBJS42753cfc6c9b: storage.py: add a shortlog method only returning revision ids and parents
rDOBJS7867dab133fd: Open release_get_by to retrieve a release by origin.
rDOBJS8dfe8a87ad56: sql/upgrades/039: add 038→039 upgrade script
rDOBJSce73d5f157ba: sql: make occurrences and revisions use type agnostic targets
rDOBJS273cb6f0594f: sql: Use a new schema for occurrences
rDOBJS6a333ea4d4be: test_storage: move backend-agnostic tests to the base class
rDOBJS73043bc5e3fa: Open directory_entry_get_by_path api
rDOBJS945abad6cb68: List direct parents and children sha1s from a given revision
rDOBJS35f2b0a65e08: Open revision_get_by to retrieve a revision by occurrence criterion filtering
rDOBJSa6e11fed9ed4: swh.storage: new release schema
rDOBJSbea6691b008d: revision_log is now 'parents' aware
rDOBJS272df815b052: Unify content_find api to return dict instead of tuples
rDOBJSeb8cc20b9268: Open revision_log api
rDOBJSb232df4ae41d: Update validity timestamp filtering
rDOBJS221b1db661e9: swh.storage: use new schema for revisions
rDOBJS7d7ad16ce76f: Revert "List direct parents and children sha1s from a given revision"
rDOBJS6d7a50f91123: sql/swh-func: get_revision_by should only return one revision
rDOBJSc4f1f6e1e382: Simplify test
rDOBJS3755a19432c4: Remove redundant content_exist and use content_find which returns None if not…
rDOBJS835c05f494df: revision_log: Implement a limit on the number of revisions listed
rDOBJS646eb1eae229: Retrieve origin information
rDOBJS6c11f3247ea9: Fix [null...] parents list
rDOBJSbde53fa5665e: Fix db.origin_add returns tuple not dict
rDOBJSad8cedbfcb10: Refactor - Simplify and add test
rDOBJS126e1f3bfdcd: Open person_get api to retrieve person information
rDOBJS53feebafa3b6: fix
rDOBJS57ff8af10994: Open release_get : [sha1] -> [Release]
rDOBJS1a65f77137d5: Add endpoints to list entities from the lister metadata
rDOBJSb9b39bf57392: Unify release_get's implementation with revision_get
rDOBJS9f8531e487eb: Add origin lookup per id
rDOBJS5f679c4a5545: Add endpoints to create entities
rDOBJS617884c8cfa8: sql: Add contents to directory_walk_one
rDOBJS38197324d80a: objstorage: create files as mode 0o644
rDOBJS8a9dec7a43ae: new SQL function: swh_stat_counters()
rDOBJS550a19740b8c: storage: add revision_get
rDOBJSfa2193f32c88: tests: fix flake8
rDOBJS4d1598d27528: Add metadata field to revision table
rDOBJS4d82f9cbf56d: test_storage: use proper types for revisions
rDOBJS61bbbe1209ea: objstorage: add test for directory modes
rDOBJS80925cc93960: Write/read `synthetic` column in revision/release
rDOBJSa1795a9b8e9e: storage: Add entry points for fetch_history
rDOBJSefe2dc312611: sql: update schema of organizations and projects
rDOBJSa9c2454dd63d: test_storage: add collision detection test
rDOBJS35dd9fbabba0: swh.storage.tests: Actually check the error message
rDOBJS40ab1c42e208: sql/swh-func: add table aliases to "where not exists" queries
rDOBJS8dfb906e4926: db_testing.py: allow to customize DB restore
rDOBJSaf4005522d58: storage.content_find: better testing
rDOBJSda274930196a: swh-schema: person (name, email) -> bytea
rDOBJSd250617427ad: swh.storage.tests: clean the database on teardown
rDOBJS0847618cb425: swh.storage.storage: make occurrence_add support overlapping intervals
rDOBJSff2661c8e940: Remove atime, mtime, ctime from directory entries
rDOBJS730766edb587: storage.content_find: refactor argument checking
rDOBJSff0c444c2e16: test_api_client: use absolute import to allow rm __init__.py
rDOBJS54134618a387: Improve output response for find occurrence api
rDOBJS41becead4548: test_storage: don't break if the storage has no objstorage
rDOBJS08bbfa766ae3: Implement content_find_occurrence up to the db
rDOBJS487e4a29ba38: swh.storage.storage: add origin_get and origin_add_one
rDOBJSef7b096357b2: sql/swh-schema: unix_path to bytea
rDOBJS59cd05e6be73: Refactor - Migrate content_present to content_find
rDOBJS3079f675482f: tests: add documentation for Test{,Remote}Storage
rDOBJSb0e2a6678554: swh.storage.api: Properly escape arbitrary byte sequences in arguments
rDOBJSea0ac366f30d: swh.storage.storage: Allow storing "skipped" contents
rDOBJS6d89128b9b8e: tests: make both storage tests inherit from an abstract class
rDOBJS30fcc29356a5: Add a remote API server and a client for remote storage
rDOBJSd4a37d7c6b96: Refactor - Permit reuse of the content_find api
rDOBJSd964f372a69a: swh.storage.api: Serialize/deserialize exceptions
rDOBJSfbdf7bf2ea65: Move remote_storage and api to api.client and api.server
rDOBJS2f969e885e40: move db_test.py from swh-storage to swh-core
rDOBJS8b012b2cc280: Simplify the api content_present
rDOBJSa2a2a65fdde3: Open /content/present api
rDOBJS6e5364e3d73b: test_remote_storage: Add code to run the server automatically
rDOBJSa9102e766c0f: Implement the content_present api
rDOBJS9d5da47dfdcd: Add offsets to the dates that need them in the schema
rDOBJS79b28a205057: Commit first implementation of directory_add
rDOBJS4aa636a5aa54: DB testing: update doc and tests to use swh-storage-testdata
rDOBJSb48911d097a9: swh.storage.storage: implement occurrence_add
rDOBJSc2a5a2d88206: swh.storage.tests: refactor storage test
rDOBJS1b5f80a80532: Implement revision_add for swh.storage.Storage
rDOBJSf02b1a0bc081: swh.storage.storage: Add content_missing method to Storage
rDOBJS4d4633a1cbd0: swh.storage.db/storage: Allow listing one directory's contents
rDOBJS1811f5916880: add back needed __init__.py to non-namespace packages
rDOBJS8c5aec2d794e: get rid of __init__.py
rDOBJS8cb72d223c39: test_objstorage.py: use assert(Not)In where possible (style)
rDOBJScf387f8b2d40: db_testing: bug fix: call specific super() methods
rDOBJSad760ca40473: db_testing.py: fixture to ease testing using the DB
rDOBJSa334ddd93523: db_testing: do not try to rollback closed connections
rDOBJS4d7ca904aeb3: test_storage.py: test add_content()
rDOBJS942a0fd6750d: stored proc: generalize mktemp function
rDOBJS375e2fda05b5: Rename Storage.add_content to content_add
rDOBJS75a597cd45bb: Update ObjStorage API for the new hashes
rDOBJS000d2098afd0: test_db.py: (trivial) tests for swh.storage.db
rDOBJS48f11ca9c99d: refactor db.Db to be a proxy with storage procedure wrappers
rDOBJSa05b79a3b210: db.Db: refactor copy_to() function to COPY to arbitrary tables
rDOBJS8e416232bcf6: make swh.storage.tests a proper python module
rDOBJS9f120b6b29cb: Update swh.storage.db for the new hash API
rDOBJS09e1d5717352: objstorage: simplify exc hierarchy to have a top-level Error exc
rDOBJS995efafc3885: objstorage: return object id upon add()
rDOBJSc6e50451862a: objstorage: idempotent addition, ignore dups
rDOBJS7c183da88c37: ensure we can easily switch to sha1 algo
rDOBJS8cd1c7c88e10: tests: preseed obj_id on add wherever possible
rDOBJSddd94f2301d0: objstorage: use __contains__ for membership test
rDOBJS810bde68afe5: tests: better organization for setUp values
rDOBJS9012042f9a68: objstorage: add support for len() and __iter__()
rDOBJSd3c33d6dbef7: add check() method
rDOBJS99f01a2f9a7f: add get() method(s)
rDOBJS75750b3b1907: objstorage: switch to sha1 object ids (from sha1_git)
rDOBJSe8733fa77194: add has() method and clobbering support
rDOBJSf17eaf8ad521: add gzip compression
rDOBJSae1f67019537: push tests down, inside swh.storage
Diff Detail
Diff Detail
- Repository
- rDOBJS Object storage
- Lint
Automatic diff as part of commit; lint not applicable. - Unit
Automatic diff as part of commit; unit tests not applicable.