diff --git a/docs/graph/dataset.rst b/docs/graph/dataset.rst
index 1a725a5..f4b41b1 100644
--- a/docs/graph/dataset.rst
+++ b/docs/graph/dataset.rst
@@ -1,292 +1,294 @@
.. _swh-dataset-list:
Dataset
=======
We aim to provide regular exports of the Software Heritage graph in two
different formats:
- **Columnar data storage**: a set of relational tables stored in a columnar
format such as `Apache ORC `_, which is particularly
suited for scale-out analyses on data lakes and big data processing
ecosystems such as the Hadoop environment.
- **Compressed graph**: a compact and highly-efficient representation of the
graph dataset, suited for scale-up analysis on high-end machines with large
amounts of memory. The graph is compressed in *Boldi-Vigna representation*,
designed to be loaded by the `WebGraph framework
`_, specifically using our `swh-graph
library `_.
Summary of dataset versions
---------------------------
**Full graph**:
.. list-table::
:header-rows: 1
* - Name
- # Nodes
- # Edges
- Columnar
- Compressed
* - `2022-04-25`_
- 25,340,003,875
- 375,867,687,011
- ✔
- ✔
* - `2021-03-23`_
- 20,667,308,808
- 232,748,148,441
- ✔
- ✔
* - `2020-12-15`_
- 19,330,739,526
- 213,848,749,638
- ✗
- ✔
* - `2020-05-20`_
- 17,075,708,289
- 203,351,589,619
- ✗
- ✔
* - `2019-01-28`_
- 11,683,687,950
- 159,578,271,511
- ✔
- ✔
**Teaser datasets**:
.. list-table::
:header-rows: 1
* - Name
- # Nodes
- # Edges
- Columnar
- Compressed
* - `2021-03-23-popular-3k-python`_
- 45,691,499
- 1,221,283,907
- ✔
- ✔
* - `2020-12-15-gitlab-all`_
- 1,083,011,764
- 27,919,670,049
- ✗
- ✔
* - `2020-12-15-gitlab-100k`_
- 304,037,235
- 9,516,984,175
- ✗
- ✔
* - `2019-01-28-popular-4k`_
- ?
- ?
- ✔
- ✗
* - `2019-01-28-popular-3k-python`_
- 27,363,226
- 346,413,337
- ✔
- ✔
Full graph datasets
-------------------
+Because of their size, some of the latest datasets are only available for
+downside from Amazon S3.
+
2022-04-25
~~~~~~~~~~
A full export of the graph dated from April 2022
- **Columnar tables (Apache ORC)**:
- **Total size**: 11 TiB
- **S3**: ``s3://softwareheritage/graph/2022-04-25/orc``
- **Compressed graph**:
- **S3**: ``s3://softwareheritage/graph/2022-04-25/compressed``
2021-03-23
~~~~~~~~~~
A full export of the graph dated from March 2021.
- **Columnar tables (Apache ORC)**:
- **Total size**: 8.4 TiB
- **URL**: `/graph/2021-03-23/orc/
`_
- **S3**: ``s3://softwareheritage/graph/2021-03-23/orc``
- **Compressed graph**:
- - **URL**: `/graph/2021-03-23/compressed/
- `_
+ - **S3**: ``s3://softwareheritage/graph/2021-03-23/compressed``
2020-12-15
~~~~~~~~~~
A full export of the graph dated from December 2020. Only available in
compressed representation.
- **Compressed graph**:
- **URL**: `/graph/2020-12-15/compressed/
`_
2020-05-20
~~~~~~~~~~
A full export of the graph dated from May 2020. Only available in
compressed representation.
**(DEPRECATED: known issue with missing snapshot edges.)**
- **Compressed graph**:
- **URL**: `/graph/2020-05-20/compressed/
`_
2019-01-28
~~~~~~~~~~
A full export of the graph dated from January 2019. The export was done in two
phases, one of them called "2018-09-25" and the other "2019-01-28". They both
refer to the same dataset, but the different formats have various
inconsistencies between them.
**(DEPRECATED: early export pipeline, various inconsistencies).**
- **Columnar tables (Apache Parquet)**:
- **Total size**: 1.2 TiB
- **URL**: `/graph/2019-01-28/parquet/
`_
- **S3**: ``s3://softwareheritage/graph/2018-09-25/parquet``
- **Compressed graph**:
- **URL**: `/graph/2019-01-28/compressed/
`_
Teaser datasets
---------------
If the above datasets are too big, we also provide "teaser"
datasets that can get you started and have a smaller size fingerprint.
2021-03-23-popular-3k-python
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The ``popular-3k-python`` teaser contains a subset of 2197 popular
repositories **tagged as being written in the Python language**, from GitHub,
Gitlab, PyPI and Debian. The selection criteria to pick the software origins
was the following:
- the 580 most popular GitHub projects written in Python (by number of stars),
- the 135 Gitlab projects written in Python that have 2 stars or more,
- the 827 most popular PyPI projects (by usage statistics, according to the
`Top PyPI Packages `_ database),
- the 655 most popular Debian packages with the
`debtag `_ ``implemented-in::python`` (by
"votes" according to the `Debian Popularity Contest
`_ database).
- **Columnar (Apache ORC)**:
- **Total size**: 36 GiB
- **S3**: ``s3://softwareheritage/graph/2021-03-23-popular-3k-python/orc/``
- **Compressed graph**:
- **Total size**: 15 GiB
- **S3**: ``s3://softwareheritage/graph/2021-03-23-popular-3k-python/compressed/``
2020-12-15-gitlab-all
~~~~~~~~~~~~~~~~~~~~~
A teaser dataset containing the entirety of Gitlab, exported in December 2020.
Available in compressed graph format.
- **Compressed graph**:
- **URL**: `/graph/2020-12-15-gitlab-all/compressed/
`_
2020-12-15-gitlab-100k
~~~~~~~~~~~~~~~~~~~~~~
A teaser dataset containing the 100k most popular Gitlab repositories,
exported in December 2020. Available in compressed graph format.
- **Compressed graph**:
- **URL**: `/graph/2020-12-15-gitlab-100k/compressed/
`_
2019-01-28-popular-4k
~~~~~~~~~~~~~~~~~~~~~
This teaser dataset contains a subset of 4000 popular repositories from GitHub,
Gitlab, PyPI and Debian. The selection criteria to pick the software origins
was the following:
- The 1000 most popular GitHub projects (by number of stars)
- The 1000 most popular Gitlab projects (by number of stars)
- The 1000 most popular PyPI projects (by usage statistics, according to the
`Top PyPI Packages `_ database),
- The 1000 most popular Debian packages (by "votes" according to the `Debian
Popularity Contest `_ database)
- **Columnar (Apache Parquet)**:
- **Total size**: 27 GiB
- **URL**: `/graph/2019-01-28-popular-4k/parquet/
`_
- **S3**: ``s3://softwareheritage/graph/2019-01-28-popular-4k/parquet/``
2019-01-28-popular-3k-python
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The ``popular-3k-python`` teaser contains a subset of 3052 popular
repositories **tagged as being written in the Python language**, from GitHub,
Gitlab, PyPI and Debian. The selection criteria to pick the software origins
was the following, similar to ``popular-4k``:
- the 1000 most popular GitHub projects written in Python (by number of stars),
- the 131 Gitlab projects written in Python that have 2 stars or more,
- the 1000 most popular PyPI projects (by usage statistics, according to the
`Top PyPI Packages `_ database),
- the 1000 most popular Debian packages with the
`debtag `_ ``implemented-in::python`` (by
"votes" according to the `Debian Popularity Contest
`_ database).
- **Columnar (Apache Parquet)**:
- **Total size**: 5.3 GiB
- **URL**: `/graph/2019-01-28-popular-3k-python/parquet/
`_
- **S3**: ``s3://softwareheritage/graph/2019-01-28-popular-3k-python/parquet/``