diff --git a/docs/images/.gitignore b/docs/images/.gitignore new file mode 100644 index 00000000..756b22fa --- /dev/null +++ b/docs/images/.gitignore @@ -0,0 +1 @@ +*.svg diff --git a/docs/images/Makefile b/docs/images/Makefile new file mode 100644 index 00000000..a029d0c2 --- /dev/null +++ b/docs/images/Makefile @@ -0,0 +1,10 @@ +UML_DIAGS_SRC = $(wildcard *.uml) +UML_DIAGS = $(patsubst %.uml,%.svg,$(UML_DIAGS_SRC)) + +all: $(UML_DIAGS) + +%.svg: %.uml + DISPLAY="" plantuml -tsvg $< + +clean: + -rm -f $(UML_DIAGS) diff --git a/docs/images/deposit-create-chart.uml b/docs/images/deposit-create-chart.uml new file mode 100644 index 00000000..d7683029 --- /dev/null +++ b/docs/images/deposit-create-chart.uml @@ -0,0 +1,26 @@ +@startuml + participant CLIENT as "SWORD client\n(eg. HAL)" + participant DEPOSIT as "swh-deposit" + participant DEPOSIT_STORAGE as "deposit storage" + + activate CLIENT + activate DEPOSIT + activate DEPOSIT_STORAGE + + CLIENT ->> DEPOSIT: GET /1// + + DEPOSIT ->> DEPOSIT_STORAGE: check authentication + DEPOSIT_STORAGE -->> DEPOSIT: return ok (if client exists and credentials ok) + + DEPOSIT -->> CLIENT: return 200, + + CLIENT ->> DEPOSIT: POST /1// + + DEPOSIT ->> DEPOSIT_STORAGE: check authentication + DEPOSIT_STORAGE -->> DEPOSIT: return ok (if client exists and credentials ok) + + DEPOSIT ->> DEPOSIT_STORAGE: create new deposit + DEPOSIT_STORAGE -->> DEPOSIT: return deposit_id + + DEPOSIT -->> CLIENT: return 201, +@enduml diff --git a/docs/images/deposit-delete-chart.uml b/docs/images/deposit-delete-chart.uml new file mode 100644 index 00000000..06b2cc7d --- /dev/null +++ b/docs/images/deposit-delete-chart.uml @@ -0,0 +1,37 @@ +@startuml + participant CLIENT as "SWORD client\n(eg. HAL)" + participant DEPOSIT as "swh-deposit" + participant DEPOSIT_STORAGE as "deposit storage" + + activate CLIENT + activate DEPOSIT + activate DEPOSIT_STORAGE + + CLIENT ->> DEPOSIT: POST /1//\nHEADER In-Progress: true + + DEPOSIT ->> DEPOSIT_STORAGE: check authentication + DEPOSIT_STORAGE -->> DEPOSIT: return ok (if client exists and credentials ok) + + DEPOSIT -->> CLIENT: return 201, + + loop while status is 'partial' + + CLIENT -> DEPOSIT: DELETE /1///media/\nDELETE /1///metadata/ + + DEPOSIT ->> DEPOSIT_STORAGE: check authentication + DEPOSIT_STORAGE -->> DEPOSIT: return ok + + DEPOSIT ->> DEPOSIT_STORAGE: check inputs() + + alt status is not 'partial' + DEPOSIT_STORAGE -->> DEPOSIT: return ko + DEPOSIT -->> CLIENT: return 403 + else status is 'partial' + DEPOSIT ->> DEPOSIT_STORAGE: delete-deposit-or-deposit-archives() + DEPOSIT_STORAGE -->> DEPOSIT: return ko + DEPOSIT -->> CLIENT: return 204 + end + + end +@enduml + diff --git a/docs/images/deposit-update-chart.uml b/docs/images/deposit-update-chart.uml new file mode 100644 index 00000000..26f44335 --- /dev/null +++ b/docs/images/deposit-update-chart.uml @@ -0,0 +1,37 @@ +@startuml + participant CLIENT as "SWORD client\n(eg. HAL)" + participant DEPOSIT as "swh-deposit" + participant DEPOSIT_STORAGE as "deposit storage" + + activate CLIENT + activate DEPOSIT + activate DEPOSIT_STORAGE + + CLIENT ->> DEPOSIT: POST /1//\nHEADER In-Progress: true + + DEPOSIT ->> DEPOSIT_STORAGE: check authentication + DEPOSIT_STORAGE -->> DEPOSIT: return ok (if client exists and credentials ok) + + DEPOSIT -->> CLIENT: return 201, + + loop while status is 'partial' + + CLIENT -> DEPOSIT: POST/PUT /1///media/\nPOST/PUT /1///metadata/ + + DEPOSIT ->> DEPOSIT_STORAGE: check authentication + DEPOSIT_STORAGE -->> DEPOSIT: return ok + + DEPOSIT ->> DEPOSIT_STORAGE: check inputs() + DEPOSIT_STORAGE -->> DEPOSIT: return ok + + alt HEADER: In-Progress = true + DEPOSIT ->> DEPOSIT_STORAGE: add-or-replace-data-and-update-status('partial') + else HEADER: In-Progress = false + DEPOSIT ->> DEPOSIT_STORAGE: add-or-replace-data-and-update-status('ready') + end + DEPOSIT_STORAGE -->> DEPOSIT: return ok + + end + + DEPOSIT -->> CLIENT: return 201, \nreturn 204 +@enduml diff --git a/docs/specs/blueprint.rst b/docs/specs/blueprint.rst index b7ab15ce..c652f84a 100644 --- a/docs/specs/blueprint.rst +++ b/docs/specs/blueprint.rst @@ -1,122 +1,186 @@ Use cases --------- Deposit creation ~~~~~~~~~~~~~~~~ From client's deposit repository server to SWH's repository server: 1. The client requests for the server's abilities and its associated collection (GET query to the *SD/service document uri*) 2. The server answers the client with the service document which gives the *collection uri* (also known as *COL/collection IRI*). 3. The client sends a deposit (optionally a zip archive, some metadata or both) through the *collection uri*. This can be done in: * one POST request (metadata + archive). * one POST request (metadata or archive) + other PUT or POST request to the *update uris* (*edit-media iri* or *edit iri*) a. Server validates the client's input or returns detailed error if any b. Server stores information received (metadata or software archive source code or both) 4. The server notifies the client it acknowledged the client's request. An ``http 201 Created`` response with a deposit receipt in the body response is sent back. That deposit receipt will hold the necessary information to eventually complete the deposit later on if it was incomplete (also known as status ``partial``). Schema representation ^^^^^^^^^^^^^^^^^^^^^ -.. raw:: html +Scenario: pushing a deposit via SWORD protocol (nominal scenario) - - -.. figure:: ../images/deposit-create-chart.png +.. figure:: ../images/deposit-create-chart.svg :alt: +Deposit statuses: + +'partial' + The deposit is new or partially received, since it can be done in + multiple requests + +'expired' + Deposit was there too long and is new deemed ready to be + garbage-collected + +'ready' + Deposit is fully received and ready for injection + +'injecting' + Injection is ongoing on SWH's side + +'success' + Injection successful + +'failure' + Injection failure + + + Updating an existing deposit ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 5. Client updates existing deposit through the *update uris* (one or more POST or PUT requests to either the *edit-media iri* or *edit iri*). 1. Server validates the client's input or returns detailed error if any 2. Server stores information received (metadata or software archive source code or both) This would be the case for example if the client initially posted a ``partial`` deposit (e.g. only metadata with no archive, or an archive without metadata, or a split archive because the initial one exceeded the limit size imposed by swh repository deposit). .. note:: It is currently only possible to update deposits in the ``partial`` state, but we are planning to allow depositing metadata in the ``done`` state as well. In this state, ``In-Progress`` is not allowed, so the deposit cannot go back in the ``partial`` state, but only to ``deposited``. Schema representation ^^^^^^^^^^^^^^^^^^^^^ -.. raw:: html - - +Scenario: updating a deposit via SWORD protocol -.. figure:: ../images/deposit-update-chart.png +.. figure:: ../images/deposit-update-chart.svg :alt: + +Deposit statuses: + +'partial' + The deposit is new or partially received, since it can be done in + multiple requests + +'expired' + Deposit was there too long and is new deemed ready to be + garbage-collected + +'ready' + Deposit is fully received and ready for injection + +'injecting' + Injection is ongoing on SWH's side + +'success' + Injection successful + +'failure' + Injection failure + Deleting deposit (or associated archive, or associated metadata) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 6. Deposit deletion is possible as long as the deposit is still in ``partial`` state. 1. Server validates the client's input or returns detailed error if any 2. Server actually delete information according to request Schema representation ^^^^^^^^^^^^^^^^^^^^^ -.. raw:: html - - +Scenario: deleting a deposit via SWORD protocol -.. figure:: ../images/deposit-delete-chart.png +.. figure:: ../images/deposit-delete-chart.svg :alt: + +Deposit statuses: + +'partial' + The deposit is new or partially received, since it can be done in + multiple requests + +'expired' + Deposit was there too long and is new deemed ready to be + garbage-collected + +'ready' + Deposit is fully received and ready for injection + +'injecting' + Injection is ongoing on SWH's side + +'success' + Injection successful + +'failure' + Injection failure + Client asks for operation status ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 7. Operation status can be read through a GET query to the *state iri*. Server: Triggering deposit checks ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Once the status ``deposited`` is reached for a deposit, checks for the associated archive(s) and metadata will be triggered. If those checks fail, the status is changed to ``rejected`` and nothing more happens there. Otherwise, the status is changed to ``verified``. Server: Triggering deposit load ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Once the status ``verified`` is reached for a deposit, loading the deposit with its associated metadata will be triggered. The loading will result on status update, either ``done`` or ``failed`` (depending on the loading's status). This is described in the `loading document <./spec-loading.html>`__.