Changeset View
Changeset View
Standalone View
Standalone View
docs/graph/dataset.rst
Show All 27 Lines | .. list-table:: | ||||
:header-rows: 1 | :header-rows: 1 | ||||
* - Name | * - Name | ||||
- # Nodes | - # Nodes | ||||
- # Edges | - # Edges | ||||
- Columnar | - Columnar | ||||
- Compressed | - Compressed | ||||
* - `2022-04-25`_ | |||||
- 25,340,003,875 | |||||
- 375,867,687,011 | |||||
- ✔ | |||||
- ✔ | |||||
* - `2021-03-23`_ | * - `2021-03-23`_ | ||||
- 20,667,308,808 | - 20,667,308,808 | ||||
- 232,748,148,441 | - 232,748,148,441 | ||||
- ✔ | - ✔ | ||||
- ✔ | - ✔ | ||||
* - `2020-12-15`_ | * - `2020-12-15`_ | ||||
- 19,330,739,526 | - 19,330,739,526 | ||||
Show All 20 Lines | .. list-table:: | ||||
:header-rows: 1 | :header-rows: 1 | ||||
* - Name | * - Name | ||||
- # Nodes | - # Nodes | ||||
- # Edges | - # Edges | ||||
- Columnar | - Columnar | ||||
- Compressed | - Compressed | ||||
* - `2021-03-23-popular-3k-python`_ | |||||
- 45,691,499 | |||||
- 1,221,283,907 | |||||
- ✔ | |||||
- ✔ | |||||
* - `2020-12-15-gitlab-all`_ | * - `2020-12-15-gitlab-all`_ | ||||
- 1,083,011,764 | - 1,083,011,764 | ||||
- 27,919,670,049 | - 27,919,670,049 | ||||
- ✗ | - ✗ | ||||
- ✔ | - ✔ | ||||
* - `2020-12-15-gitlab-100k`_ | * - `2020-12-15-gitlab-100k`_ | ||||
- 304,037,235 | - 304,037,235 | ||||
Show All 11 Lines | * - `2019-01-28-popular-3k-python`_ | ||||
- 27,363,226 | - 27,363,226 | ||||
- 346,413,337 | - 346,413,337 | ||||
- ✔ | - ✔ | ||||
- ✔ | - ✔ | ||||
Full graph datasets | Full graph datasets | ||||
------------------- | ------------------- | ||||
2022-04-25 | |||||
~~~~~~~~~~ | |||||
A full export of the graph dated from April 2022 | |||||
- **Columnar tables (Apache ORC)**: | |||||
- **Total size**: 11 TiB | |||||
- **S3**: ``s3://softwareheritage/graph/2022-04-25/orc`` | |||||
- **Compressed graph**: | |||||
- **S3**: ``s3://softwareheritage/graph/2022-04-25/compressed`` | |||||
2021-03-23 | 2021-03-23 | ||||
~~~~~~~~~~ | ~~~~~~~~~~ | ||||
A full export of the graph dated from March 2021. | A full export of the graph dated from March 2021. | ||||
- **Columnar tables (Apache ORC)**: | - **Columnar tables (Apache ORC)**: | ||||
▲ Show 20 Lines • Show All 57 Lines • ▼ Show 20 Lines | |||||
Teaser datasets | Teaser datasets | ||||
--------------- | --------------- | ||||
If the above datasets are too big, we also provide "teaser" | If the above datasets are too big, we also provide "teaser" | ||||
datasets that can get you started and have a smaller size fingerprint. | datasets that can get you started and have a smaller size fingerprint. | ||||
2021-03-23-popular-3k-python | |||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |||||
The ``popular-3k-python`` teaser contains a subset of 2197 popular | |||||
repositories **tagged as being written in the Python language**, from GitHub, | |||||
Gitlab, PyPI and Debian. The selection criteria to pick the software origins | |||||
was the following: | |||||
- the 580 most popular GitHub projects written in Python (by number of stars), | |||||
- the 135 Gitlab projects written in Python that have 2 stars or more, | |||||
- the 827 most popular PyPI projects (by usage statistics, according to the | |||||
`Top PyPI Packages <https://hugovk.github.io/top-pypi-packages/>`_ database), | |||||
- the 655 most popular Debian packages with the | |||||
`debtag <https://debtags.debian.org/>`_ ``implemented-in::python`` (by | |||||
"votes" according to the `Debian Popularity Contest | |||||
<https://popcon.debian.org/>`_ database). | |||||
- **Columnar (Apache ORC)**: | |||||
- **Total size**: 36 GiB | |||||
- **S3**: ``s3://softwareheritage/graph/2021-03-23-popular-3k-python/orc/`` | |||||
2020-12-15-gitlab-all | 2020-12-15-gitlab-all | ||||
~~~~~~~~~~~~~~~~~~~~~ | ~~~~~~~~~~~~~~~~~~~~~ | ||||
A teaser dataset containing the entirety of Gitlab, exported in December 2020. | A teaser dataset containing the entirety of Gitlab, exported in December 2020. | ||||
Available in compressed graph format. | Available in compressed graph format. | ||||
- **Compressed graph**: | - **Compressed graph**: | ||||
▲ Show 20 Lines • Show All 59 Lines • Show Last 20 Lines |