diff --git a/docs/getting-started.rst b/docs/getting-started.rst index 6e281ff8..33f1749b 100644 --- a/docs/getting-started.rst +++ b/docs/getting-started.rst @@ -1,320 +1,286 @@ Getting Started =============== This is a guide for how to prepare and push a software deposit with the `swh deposit` commands. The API is rooted at https://deposit.softwareheritage.org/1. For more details, see the `main documentation <./index.html>`__. Requirements ------------ You need to be referenced on SWH's client list to have: * credentials (needed for the basic authentication step) - in this document we reference ```` as the client's name and ```` as its associated authentication password. * an associated collection_. .. _collection: https://bitworking.org/projects/atom/rfc5023#rfc.section.8.3.3 `Contact us for more information. `__ Prepare a deposit ----------------- * compress the files in a supported archive format: - zip: common zip archive (no multi-disk zip files). - tar: tar archive without compression or optionally any of the following compression algorithm gzip (`.tar.gz`, `.tgz`), bzip2 (`.tar.bz2`) , or lzma (`.tar.lzma`) -* prepare a metadata file (`more details <./metadata.html>`__.): - - - specify metadata schema/vocabulary (CodeMeta is strongly recommended) - - specify *MUST* metadata (url, authors, software name and - the external\_identifier) - - add all available information under the compatible metadata term. - - Here is an example of an atom entry file with CodeMeta terms: - -.. code:: xml - - - - Je suis GPL - swh - je-suis-gpl - https://forge.softwareheritage.org/source/jesuisgpl/ - 2018-01-05 - Je suis GPL is a modified version of GNU Hello whose - sole purpose is to showcase the usage of - Software Heritage for license compliance purposes. - 0.1 - GNU/Linux - stable - C - - - GNU General Public License v3.0 or later - https://spdx.org/licenses/GPL-3.0-or-later.html - - - Stefano Zacchiroli - Maintainer - - - +* (Optional) prepare a metadata file (more details :ref:`deposit-metadata`): Push deposit ------------ You can push a deposit with: * a single deposit (archive + metadata): The user posts in one query a software source code archive and associated metadata. The deposit is directly marked with status ``deposited``. * a multisteps deposit: 1. Create an incomplete deposit (marked with status ``partial``) 2. Add data to a deposit (in multiple requests if needed) 3. Finalize deposit (the status becomes ``deposited``) Single deposit ^^^^^^^^^^^^^^ Once the files are ready for deposit, we want to do the actual deposit in one shot, sending exactly one POST query: * 1 archive (content-type ``application/zip`` or ``application/x-tar``) * 1 metadata file in atom xml format (``content-type: application/atom+xml;type=entry``) For this, we need to provide the: * arguments: ``--username 'name' --password 'pass'`` as credentials -* archive's path (example: ``--archive path/to/archive-name.tgz``) : +* archive's path (example: ``--archive path/to/archive-name.tgz``) +* software's name (optional if a metadata filepath is specified and the + artifact's name is included in the metadata file). +* author's name (optional if a metadata filepath is specified and the authors + are included in the metadata file). This can be specified multiple times in + case of multiple authors. * (optionally) metadata file's path ``--metadata - path/to/file.metadata.xml``. If not provided, the archive's filename - will be used to determine the metadata file, e.g: - ``path/to/archive-name.tgz.metadata.xml`` -* (optionally) ``--slug 'your-id'`` argument, a reference to a - unique identifier the client uses for the software object. + path/to/file.metadata.xml``. +* (optionally) ``--slug 'your-id'`` argument, a reference to a unique identifier + the client uses for the software object. If not provided, A UUID will be + generated by SWH. You can do this with the following command: minimal deposit .. code:: shell $ swh deposit upload --username name --password secret \ + --author "some@noone" --author "second@noone" \ + --name 'je-suis-gpl' \ --archive je-suis-gpl.tgz with client's external identifier (``slug``) .. code:: shell $ swh deposit upload --username name --password secret \ + --author "some@noone" \ + --name 'je-suis-gpl' \ --archive je-suis-gpl.tgz \ --slug je-suis-gpl to a specific client's collection .. code:: shell $ swh deposit upload --username name --password secret \ + --author "some@noone" \ + --name 'je-suis-gpl' \ --archive je-suis-gpl.tgz \ --collection 'second-collection' - You just posted a deposit to your collection on Software Heritage If everything went well, the successful response will contain the elements below: .. code:: shell { 'deposit_status': 'deposited', 'deposit_id': '7', 'deposit_date': 'Jan. 29, 2018, 12:29 p.m.' } Note: As the deposit is in ``deposited`` status, you can no longer update the deposit after this query. It will be answered with a 403 forbidden answer. If something went wrong, an equivalent response will be given with the `error` and `detail` keys explaining the issue, e.g.: .. code:: shell { 'error': 'Unknown collection name xyz', 'detail': None, 'deposit_status': None, 'deposit_status_detail': None, 'deposit_swh_id': None, 'status': 404 } multisteps deposit ^^^^^^^^^^^^^^^^^^^^^^^^^ The steps to create a multisteps deposit: 1. Create an incomplete deposit ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ First use the ``--partial`` argument to declare there is more to come .. code:: shell $ swh deposit upload --username name --password secret \ --archive foo.tar.gz \ --partial 2. Add content or metadata to the deposit ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Continue the deposit by using the ``--deposit-id`` argument given as a response for the first step. You can continue adding content or metadata while you use the ``--partial`` argument. +To only add one new archive to the deposit: + .. code:: shell $ swh deposit upload --username name --password secret \ --archive add-foo.tar.gz \ --deposit-id 42 \ --partial - -In case you want to add only one new archive without metadata: +To only add metadata to the deposit: .. code:: shell $ swh deposit upload --username name --password secret \ - --archive add-foo.tar.gz \ - --archive-deposit \ + --metadata add-foo.tar.gz.metadata.xml \ --deposit-id 42 \ --partial -If you want to add only metadata, use: - +or: .. code:: shell $ swh deposit upload --username name --password secret \ - --metadata add-foo.tar.gz.metadata.xml \ - --metadata-deposit \ + --name 'add-foo' --author 'someone' \ --deposit-id 42 \ --partial + 3. Finalize deposit ~~~~~~~~~~~~~~~~~~~ -On your last addition, by not declaring it as ``--partial``, the -deposit will be considered as completed and its status will be changed -to ``deposited``: - -.. code:: shell - $ swh deposit upload --username name --password secret \ - --metadata add-foo.tar.gz.metadata.xml \ - --metadata-deposit \ - --deposit-id 42 +On your last addition (same command as before), by not declaring it +``--partial``, the deposit will be considered completed. Its status will be +changed to ``deposited`` Update deposit ---------------- * replace deposit: - only possible if the deposit status is ``partial`` and ``--deposit-id `` is provided - by using the ``--replace`` flag - ``--metadata-deposit`` replaces associated existing metadata - ``--archive-deposit`` replaces associated archive(s) - by default, with no flag or both, you'll replace associated metadata and archive(s): .. code:: shell $ swh deposit upload --username name --password secret \ --deposit-id 11 \ --archive updated-je-suis-gpl.tgz \ --replace * update a loaded deposit with a new version: - by using the external-id with the ``--slug`` argument, you will link the new deposit with its parent deposit: .. code:: shell $ swh deposit upload --username name --password secret \ --archive je-suis-gpl-v2.tgz \ --slug 'je-suis-gpl' \ Check the deposit's status -------------------------- You can check the status of the deposit by using the ``--deposit-id`` argument: .. code:: shell $ swh deposit status --username name --password secret \ --deposit-id 11 .. code:: json { 'deposit_id': '11', 'deposit_status': 'deposited', 'deposit_swh_id': None, 'deposit_status_detail': 'Deposit is ready for additional checks \ (tarball ok, metadata, etc...)' } The different statuses: - **partial**: multipart deposit is still ongoing - **deposited**: deposit completed - **rejected**: deposit failed the checks - **verified**: content and metadata verified - **loading**: loading in-progress - **done**: loading completed successfully - **failed**: the deposit loading has failed When the deposit has been loaded into the archive, the status will be marked ``done``. In the response, will also be available the , , , . For example: .. code:: json { 'deposit_id': '11', 'deposit_status': 'done', 'deposit_swh_id': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9', 'deposit_swh_id_context': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9;origin=https://forge.softwareheritage.org/source/jesuisgpl/', 'deposit_swh_anchor_id': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb', 'deposit_swh_anchor_id_context': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb;origin=https://forge.softwareheritage.org/source/jesuisgpl/', 'deposit_status_detail': 'The deposit has been successfully \ loaded into the Software Heritage archive' } diff --git a/docs/metadata.rst b/docs/metadata.rst index 768099e3..f8e1cda2 100644 --- a/docs/metadata.rst +++ b/docs/metadata.rst @@ -1,183 +1,185 @@ +.. _deposit-metadata: + Deposit metadata ================ When making a software deposit into the SWH archive, one can add information describing the software artifact and the software project. Metadata requirements --------------------- - **the schema/vocabulary** used *MUST* be specified with a persistent url (DublinCore, DOAP, CodeMeta, etc.) .. code:: xml or or - **the name** of the software deposit *MUST* be provided [atom:title, codemeta:name, dcterms:title] - **the authors** of the software deposit *MUST* be provided - **the url** representing the location of the source *MAY* be provided under the url tag. The url will be used for creating an origin object in the archive. .. code:: xml www.url-example.com - **the external\_identifier** *MAY* be provided as an identifier - **the external\_identifier** *SHOULD* match the Slug external-identifier in the header - **the description** of the software deposit *SHOULD* be provided [codemeta:description]: short or long description of the software - **the license/s** of the software deposit *SHOULD* be provided [codemeta:license] - other metadata *MAY* be added with terms defined by the schema in use. Examples -------- Using only Atom ~~~~~~~~~~~~~~~ .. code:: xml Awesome Compiler urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a 1785io25c695 2017-10-07T15:17:08Z some awesome author Using Atom with CodeMeta ~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: xml Awesome Compiler urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a 1785io25c695 1785io25c695 origin url other identifier, DOI, ARK Domain description key-word 1 key-word 2 creation date publication date comment article name article id Collaboration/Projet project name id see also Sponsor A Sponsor B Platform/OS dependencies Version active license url spdx .Net Framework 3.0 Python2.3 author1 Inria UPMC author2 Inria UPMC http://code.com language 1 language 2 http://issuetracker.com Using Atom with DublinCore and CodeMeta (multi-schema entry) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: xml Awesome Compiler hal urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a %s hal-01587361 doi:10.5281/zenodo.438684 The assignment problem AffectationRO author [INFO] Computer Science [cs] [INFO.INFO-RO] Computer Science [cs]/Operations Research [cs.RO] SOFTWARE Project in OR: The assignment problemA java implementation for the assignment problem first release description fr 2015-06-01 2017-10-19 en origin url 1.0.0 key word Comment Rfrence interne link Sponsor Platform/OS dependencies Ended license url spdx http://code.com language 1 language 2 Note ---- We aim on harmonizing the metadata from different origins and thus metadata will be translated to the `CodeMeta v.2 `__ vocabulary if possible.