Changeset View
Changeset View
Standalone View
Standalone View
docs/journal.rst
Show First 20 Lines • Show All 92 Lines • ▼ Show 20 Lines | |||||
with `branches` being a dictionary which keys are branch names [bytes], and values a dictionary of: | with `branches` being a dictionary which keys are branch names [bytes], and values a dictionary of: | ||||
- `target` [bytes] intrinsic identifier of the targeted object | - `target` [bytes] intrinsic identifier of the targeted object | ||||
- `target_type` [string] the type of the targeted object (can be "content", | - `target_type` [string] the type of the targeted object (can be "content", | ||||
"directory", "revision", "release", "snapshot" or "alias"). | "directory", "revision", "release", "snapshot" or "alias"). | ||||
Example: | Example: | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
'branches': { | 'branches': { | ||||
b'refs/pull/1/head': { | b'refs/pull/1/head': { | ||||
'target': b'\x07\x10\\\xfc\xae\x1f\xb1\xf9\xb5\xad\x8bI\xf1G\x10\x9a\xba>8\x0c', | 'target': b'\x07\x10\\\xfc\xae\x1f\xb1\xf9\xb5\xad\x8bI\xf1G\x10\x9a\xba>8\x0c', | ||||
'target_type': 'revision' | 'target_type': 'revision' | ||||
}, | }, | ||||
b'refs/pull/2/head': { | b'refs/pull/2/head': { | ||||
Show All 34 Lines | - `synthetic` [bool] True if the :py:class:`swh.model.model.Release` object has | ||||
computation, | computation, | ||||
- `author` [dict] the author of the release | - `author` [dict] the author of the release | ||||
- `date` [gitdate] the date of the release | - `date` [gitdate] the date of the release | ||||
- `id` [bytes] the intrinsic identifier of the | - `id` [bytes] the intrinsic identifier of the | ||||
:py:class:`swh.model.model.Release` object | :py:class:`swh.model.model.Release` object | ||||
Example: | Example: | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
'name': b'0.3', | 'name': b'0.3', | ||||
'message': b'', | 'message': b'', | ||||
'target': b'<\xd6\x15\xd9\xef@\xe0[\xe7\x11=\xa1W\x11h%\xcc\x13\x96\x8d', | 'target': b'<\xd6\x15\xd9\xef@\xe0[\xe7\x11=\xa1W\x11h%\xcc\x13\x96\x8d', | ||||
'target_type': 'revision', | 'target_type': 'revision', | ||||
'synthetic': False, | 'synthetic': False, | ||||
'author': { | 'author': { | ||||
Show All 35 Lines | - `metadata` [bytes] the metadata linked to this :py:class:`swh.model.model.Revision` (not part of the | ||||
intrinsic identifier computation), | intrinsic identifier computation), | ||||
- `parents` [list[bytes]] list of parent :py:class:`swh.model.model.Revision` intrinsic identifiers | - `parents` [list[bytes]] list of parent :py:class:`swh.model.model.Revision` intrinsic identifiers | ||||
- `id` [bytes] intrinsic identifier of the :py:class:`swh.model.model.Revision` | - `id` [bytes] intrinsic identifier of the :py:class:`swh.model.model.Revision` | ||||
- `extra_headers` [list[(bytes, bytes)]] TODO | - `extra_headers` [list[(bytes, bytes)]] TODO | ||||
Example: | Example: | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
'message': b'I now arrange to be able to create a prettyprinted version of the Pascal\ncode to make review of translation of it easier, and I have thought a bit\nmore about coping with Pastacl variant records and the like, but have yet to\nimplement everything. lufylib.red is a place for support code.\n', | 'message': b'I now arrange to be able to create a prettyprinted version of the Pascal\ncode to make review of translation of it easier, and I have thought a bit\nmore about coping with Pastacl variant records and the like, but have yet to\nimplement everything. lufylib.red is a place for support code.\n', | ||||
'author': { | 'author': { | ||||
'fullname': b'\xf3\xa7\xde7[\x8b#=\xe48\\/\xa1 \xed\x05NA\xa6\xf8\x9c\n\xad5\xe7\xe0"\xc4\xd5[\xc9z', | 'fullname': b'\xf3\xa7\xde7[\x8b#=\xe48\\/\xa1 \xed\x05NA\xa6\xf8\x9c\n\xad5\xe7\xe0"\xc4\xd5[\xc9z', | ||||
'name': None, | 'name': None, | ||||
'email': None | 'email': None | ||||
}, | }, | ||||
▲ Show 20 Lines • Show All 41 Lines • ▼ Show 20 Lines | |||||
- `blake2s256` [bytes] Blake2S256 hash of the :py:class:`swh.model.model.Content` | - `blake2s256` [bytes] Blake2S256 hash of the :py:class:`swh.model.model.Content` | ||||
- `length` [int] length of the :py:class:`swh.model.model.Content` | - `length` [int] length of the :py:class:`swh.model.model.Content` | ||||
- `status` [string] visibility status of the :py:class:`swh.model.model.Content` (can be "visible" or "hidden") | - `status` [string] visibility status of the :py:class:`swh.model.model.Content` (can be "visible" or "hidden") | ||||
- `ctime` [timestamp] creation date of the :py:class:`swh.model.model.Content` (i.e. date at which this | - `ctime` [timestamp] creation date of the :py:class:`swh.model.model.Content` (i.e. date at which this | ||||
:py:class:`swh.model.model.Content` has been seen for the first time in the |swh| Archive). | :py:class:`swh.model.model.Content` has been seen for the first time in the |swh| Archive). | ||||
Example: | Example: | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
'sha1': b'-\xe7\xc1`\x9d\xd7\x7fu+\x05l\x07\xd1}\x95\x16o-u\x1d', | 'sha1': b'-\xe7\xc1`\x9d\xd7\x7fu+\x05l\x07\xd1}\x95\x16o-u\x1d', | ||||
'sha1_git': b'\xb9B\xa7EOW[\xef\x8b\x98\xa6b\xe9\xc7\xf0\x96g\x06`\xa4', | 'sha1_git': b'\xb9B\xa7EOW[\xef\x8b\x98\xa6b\xe9\xc7\xf0\x96g\x06`\xa4', | ||||
'sha256': b'h{\xda\x8d\xaeG\xa4\xc6\x10\x05\xbc\xc9hca\x0em)\xd3A\x08\xd6\x95~(\xe5\xba\xe4\xaa\xcaT\x19', | 'sha256': b'h{\xda\x8d\xaeG\xa4\xc6\x10\x05\xbc\xc9hca\x0em)\xd3A\x08\xd6\x95~(\xe5\xba\xe4\xaa\xcaT\x19', | ||||
'blake2s256': b'\x8cl\xec\xe8S\xcd\xab\x90E\xc2\x8c\xfax\xe3\xbe\xca\x9aJ6\x1a\x9c](6\xc3\xb49\x8b:\xf9\xd8r', | 'blake2s256': b'\x8cl\xec\xe8S\xcd\xab\x90E\xc2\x8c\xfax\xe3\xbe\xca\x9aJ6\x1a\x9c](6\xc3\xb49\x8b:\xf9\xd8r', | ||||
'length': 3220, | 'length': 3220, | ||||
'status': 'visible', | 'status': 'visible', | ||||
Show All 23 Lines | |||||
- `ctime` [timestamp] creation date of the | - `ctime` [timestamp] creation date of the | ||||
:py:class:`swh.model.model.SkippedContent` (i.e. date at which this | :py:class:`swh.model.model.SkippedContent` (i.e. date at which this | ||||
:py:class:`swh.model.model.SkippedContent` has been seen for the first time in | :py:class:`swh.model.model.SkippedContent` has been seen for the first time in | ||||
the |swh| Archive) | the |swh| Archive) | ||||
Example: | Example: | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
'sha1': b'[\x0f\x19I-%+\xec\x9dS\x86\xffz\xcb\xa2\x9f\x15\xcc\xb4&', | 'sha1': b'[\x0f\x19I-%+\xec\x9dS\x86\xffz\xcb\xa2\x9f\x15\xcc\xb4&', | ||||
'sha1_git': b'\xa9\xff4\xa7\xff\x85\xb3x$Ot\xaa\x91\x0b\xd0ZB!\x04\x8a', | 'sha1_git': b'\xa9\xff4\xa7\xff\x85\xb3x$Ot\xaa\x91\x0b\xd0ZB!\x04\x8a', | ||||
'sha256': b"\xe6\x876\xb2U-\x87\xb8\xe3\x12\xa0L\rq'\x88\xd4\x95\x92\xdf\x86\xfci\xe3E\x82\xe0\x95^\xbf\x1e\xbe", | 'sha256': b"\xe6\x876\xb2U-\x87\xb8\xe3\x12\xa0L\rq'\x88\xd4\x95\x92\xdf\x86\xfci\xe3E\x82\xe0\x95^\xbf\x1e\xbe", | ||||
'blake2s256': b'\xe1 \n\x1d5\x8b\x1f\x98\\\x8e\xaa\x1d?8*\xc1\xf7\xb9\x95\r|\x1e\xee^\x10\x10\x19\xc6\x9c\x11\xedX', | 'blake2s256': b'\xe1 \n\x1d5\x8b\x1f\x98\\\x8e\xaa\x1d?8*\xc1\xf7\xb9\x95\r|\x1e\xee^\x10\x10\x19\xc6\x9c\x11\xedX', | ||||
'length': 125146729, | 'length': 125146729, | ||||
'status': 'absent', | 'status': 'absent', | ||||
Show All 17 Lines | |||||
- `name` [bytes] name of the directory entry | - `name` [bytes] name of the directory entry | ||||
- `type` [string] type of directory entry (can be "file", "dir" or "rev") | - `type` [string] type of directory entry (can be "file", "dir" or "rev") | ||||
- `perms` [int] permissions for this directory entry | - `perms` [int] permissions for this directory entry | ||||
Example: | Example: | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
'entries': [ | 'entries': [ | ||||
{'name': b'LICENSE', | {'name': b'LICENSE', | ||||
'type': 'file', | 'type': 'file', | ||||
'target': b'b\x03f\xeb\x90\x07\x1cs\xaeib\x8eg\x97]0\xf0\x9dg\x01', | 'target': b'b\x03f\xeb\x90\x07\x1cs\xaeib\x8eg\x97]0\xf0\x9dg\x01', | ||||
'perms': 33188}, | 'perms': 33188}, | ||||
{'name': b'README.md', | {'name': b'README.md', | ||||
Show All 28 Lines | |||||
Topic for :py:class:`swh.model.model.Origin` objects. | Topic for :py:class:`swh.model.model.Origin` objects. | ||||
Message format: | Message format: | ||||
- `url` [string] URL of the :py:class:`swh.model.model.Origin` | - `url` [string] URL of the :py:class:`swh.model.model.Origin` | ||||
Example: | Example: | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
"url": "https://github.com/vujkovicm/pml" | "url": "https://github.com/vujkovicm/pml" | ||||
} | } | ||||
`swh.journal.objects.origin_visit` | `swh.journal.objects.origin_visit` | ||||
++++++++++++++++++++++++++++++++++ | ++++++++++++++++++++++++++++++++++ | ||||
Topic for :py:class:`swh.model.model.OriginVisit` objects. | Topic for :py:class:`swh.model.model.OriginVisit` objects. | ||||
Message format: | Message format: | ||||
- `origin` [string] URL of the visited :py:class:`swh.model.model.Origin` | - `origin` [string] URL of the visited :py:class:`swh.model.model.Origin` | ||||
- `date` [timestamp] date of the visit | - `date` [timestamp] date of the visit | ||||
- `type` [string] type of the loader used to perform the visit | - `type` [string] type of the loader used to perform the visit | ||||
- `visit` [int] number of the visit for this `origin` | - `visit` [int] number of the visit for this `origin` | ||||
Example: | Example: | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
'origin': 'https://pypi.org/project/wasp-eureka/', | 'origin': 'https://pypi.org/project/wasp-eureka/', | ||||
'date': Timestamp(seconds=1606260407, nanoseconds=818259954), | 'date': Timestamp(seconds=1606260407, nanoseconds=818259954), | ||||
'type': 'pypi', | 'type': 'pypi', | ||||
'visit': 505} | 'visit': 505} | ||||
} | } | ||||
Show All 10 Lines | |||||
- `date` [timestamp] date of the visit status update | - `date` [timestamp] date of the visit status update | ||||
- `status` [string] status (can be "created", "ongoing", "full" or "partial"), | - `status` [string] status (can be "created", "ongoing", "full" or "partial"), | ||||
- `snapshot` [bytes] identifier of the :py:class:`swh.model.model.Snaphot` this | - `snapshot` [bytes] identifier of the :py:class:`swh.model.model.Snaphot` this | ||||
visit resulted in (if `status` is "full" or "partial") | visit resulted in (if `status` is "full" or "partial") | ||||
- `metadata`: deprecated | - `metadata`: deprecated | ||||
Example: | Example: | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
'origin': 'https://pypi.org/project/stricttype/', | 'origin': 'https://pypi.org/project/stricttype/', | ||||
'visit': 524, | 'visit': 524, | ||||
'date': Timestamp(seconds=1606260407, nanoseconds=818259954), | 'date': Timestamp(seconds=1606260407, nanoseconds=818259954), | ||||
'status': 'full', | 'status': 'full', | ||||
'snapshot': b"\x85\x8f\xcb\xec\xbd\xd3P;Z\xb0~\xe7\xa2(\x0b\x11'\x05i\xf7", | 'snapshot': b"\x85\x8f\xcb\xec\xbd\xd3P;Z\xb0~\xe7\xa2(\x0b\x11'\x05i\xf7", | ||||
'metadata': None | 'metadata': None | ||||
Show All 17 Lines | |||||
Message format: | Message format: | ||||
- `type` [string] | - `type` [string] | ||||
- `url` [string] | - `url` [string] | ||||
- `metadata` [dict] | - `metadata` [dict] | ||||
Examples: | Examples: | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
'type': 'forge', | 'type': 'forge', | ||||
'url': 'https://guix.gnu.org/sources.json', | 'url': 'https://guix.gnu.org/sources.json', | ||||
'metadata': {} | 'metadata': {} | ||||
} | } | ||||
{ | { | ||||
Show All 12 Lines | |||||
Message format: | Message format: | ||||
- `type` [string] | - `type` [string] | ||||
- `version` [string] | - `version` [string] | ||||
- `metadata` [dict] | - `metadata` [dict] | ||||
Example: | Example: | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
'name': 'swh.loader.package.cran.loader.CRANLoader', | 'name': 'swh.loader.package.cran.loader.CRANLoader', | ||||
'version': '0.15.0', | 'version': '0.15.0', | ||||
'metadata': {} | 'metadata': {} | ||||
} | } | ||||
Show All 17 Lines | |||||
- `snapshot` [SWHID] | - `snapshot` [SWHID] | ||||
- `release` [SWHID] | - `release` [SWHID] | ||||
- `revision` [SWHID] | - `revision` [SWHID] | ||||
- `path` [bytes] | - `path` [bytes] | ||||
- `directory` [SWHID] | - `directory` [SWHID] | ||||
Example: | Example: | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
'type': 'snapshot', | 'type': 'snapshot', | ||||
'id': 'swh:1:snp:f3b180979283d4931d3199e6171840a3241829a3', | 'id': 'swh:1:snp:f3b180979283d4931d3199e6171840a3241829a3', | ||||
'discovery_date': Timestamp(seconds=1606260407, nanoseconds=818259954), | 'discovery_date': Timestamp(seconds=1606260407, nanoseconds=818259954), | ||||
'authority': { | 'authority': { | ||||
'type': 'forge', | 'type': 'forge', | ||||
'url': 'https://pypi.org/', | 'url': 'https://pypi.org/', | ||||
▲ Show 20 Lines • Show All 65 Lines • ▼ Show 20 Lines | - dates for git-like objects (:py:class:`swh.model.model.Revision` and | ||||
- `offset` [int] offset of the date (in minutes) | - `offset` [int] offset of the date (in minutes) | ||||
- `negative_utc` [bool] only True for the very edge case where the date has a | - `negative_utc` [bool] only True for the very edge case where the date has a | ||||
zero but negative offset value (which does not makes much sense, but | zero but negative offset value (which does not makes much sense, but | ||||
technically the git format permits) | technically the git format permits) | ||||
Example: | Example: | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
'timestamp': {'seconds': 1480432642, 'microseconds': 0}, | 'timestamp': {'seconds': 1480432642, 'microseconds': 0}, | ||||
'offset': 180, | 'offset': 180, | ||||
'negative_utc': False | 'negative_utc': False | ||||
} | } | ||||
These are denoted as `gitdate` below. | These are denoted as `gitdate` below. | ||||
- other dates (resulting of the |swh| processing stack) are encoded using | - other dates (resulting of the |swh| processing stack) are encoded using | ||||
msgpack's Timestamp_ extended type. | msgpack's Timestamp_ extended type. | ||||
These are denoted as `timestamp` below. | These are denoted as `timestamp` below. | ||||
Note that these dates used to be encoded as a dictionary (beware: keys are bytes): | Note that these dates used to be encoded as a dictionary (beware: keys are bytes): | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
b"swhtype": "datetime", | b"swhtype": "datetime", | ||||
b"d": '2020-09-15T16:19:13.037809+00:00' | b"d": '2020-09-15T16:19:13.037809+00:00' | ||||
} | } | ||||
Person | Person | ||||
++++++ | ++++++ | ||||
:py:class:`swh.model.model.Person` objects represent a person in the |swh| | :py:class:`swh.model.model.Person` objects represent a person in the |swh| | ||||
Merkle DAG, namely a :py:class:`swh.model.model.Revision` author or committer, | Merkle DAG, namely a :py:class:`swh.model.model.Revision` author or committer, | ||||
or a :py:class:`swh.model.model.Release` author. | or a :py:class:`swh.model.model.Release` author. | ||||
:py:class:`swh.model.model.Person` objects are serialized as a dictionary like: | :py:class:`swh.model.model.Person` objects are serialized as a dictionary like: | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
'fullname': 'John Doe <john.doe@example.com>', | 'fullname': 'John Doe <john.doe@example.com>', | ||||
'name': 'John Doe', | 'name': 'John Doe', | ||||
'email': 'john.doe@example.com' | 'email': 'john.doe@example.com' | ||||
} | } | ||||
For anonymized topics, :py:class:`swh.model.model.Person` entities have seen | For anonymized topics, :py:class:`swh.model.model.Person` entities have seen | ||||
anonymized prior to being serialized. The anonymized | anonymized prior to being serialized. The anonymized | ||||
:py:class:`swh.model.model.Person` object is a dictionary like: | :py:class:`swh.model.model.Person` object is a dictionary like: | ||||
.. code:: json | .. code:: python | ||||
{ | { | ||||
'fullname': <hashed value>, | 'fullname': <hashed value>, | ||||
'name': null, | 'name': null, | ||||
'email': null | 'email': null | ||||
} | } | ||||
Show All 11 Lines |