diff --git a/docs/getting-started.rst b/docs/getting-started.rst
index c009eb1..1521f67 100644
--- a/docs/getting-started.rst
+++ b/docs/getting-started.rst
@@ -1,67 +1,65 @@
.. highlight:: bash
.. _getting-started:
Run your own Software Heritage
==============================
This tutorial will guide from the basic step of obtaining the source code of
the Software Heritage stack to running a local copy of it with which you can
archive source code and browse it on the web. To that end, just follow the
steps detailed below.
Using Docker
-++++++++++++
+------------
The easiest way to run a Software Heritage instance is to use Docker and
docker-compose. Please refer to the `docker-compose documentation
`_ if you do not have a working docker setup.
Then::
git clone https://forge.softwareheritage.org/source/swh-environment.git
cd swh-environment/docker
docker-compose up -d
When all the containers are up and running, you have a running Software
Heritage platform. You should open:
- http://localhost:5080/ to navigate your (empty for now) SWH archive,
- http://localhost:5080/rabbitmq to access the rabbitmq dashboard (guest/guest),
- http://localhost:5080/grafana to explore the platform's metrics (admin/admin),
All the internal APIs are also exposed:
- http://localhost:5080/scheduler
- http://localhost:5080/storage
- http://localhost:5080/indexer-storage
- http://localhost:5080/deposit
- http://localhost:5080/objstorage
At this point, the simplest way to start loading software is to use the 'Save
Code Now' feature of the archive web interface:
http://localhost:5080/browse/origin/save/
For details and an overview of what you can do with your Software Heritage,
-see `docker/README.md`_
+see the full :ref:`docker-environment` documentation.
Enjoy filling your hard drives!
-.. _docker/README.md: https://forge.softwareheritage.org/source/swh-environment/browse/master/docker/
-
Hacking the archive
-+++++++++++++++++++
+-------------------
If you want to hack the code of the Software Heritage Archive, a bit more work
will be required.
To be able to write patches, you will need a development setup.
The best way to have a development-friendly environment is to build a mixed
docker/virtualenv setup.
Such a setup is described in the
:ref:`Developer Setup Guide `.
diff --git a/docs/getting-started-index.rst b/docs/getting-started/index.rst
similarity index 54%
rename from docs/getting-started-index.rst
rename to docs/getting-started/index.rst
index 754ce1e..0cfc3d5 100644
--- a/docs/getting-started-index.rst
+++ b/docs/getting-started/index.rst
@@ -1,12 +1,13 @@
Getting started
===============
.. this index file is used to force a hierarchy in the toctree,
- even though these two files are in the same directory
+ even though these two files are in the parent directory
.. toctree::
:maxdepth: 2
:titlesonly:
- getting-started
- developer-setup
+ ../getting-started
+ ../developer-setup
+ using-docker
diff --git a/docs/getting-started/using-docker.rst b/docs/getting-started/using-docker.rst
new file mode 100644
index 0000000..06fda43
--- /dev/null
+++ b/docs/getting-started/using-docker.rst
@@ -0,0 +1,3 @@
+.. _docker-environment:
+
+.. include:: ../../../docker/README.rst
diff --git a/docs/index.rst b/docs/index.rst
index 165ac49..6e46c05 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -1,217 +1,217 @@
.. _swh-docs:
Software Heritage - Development Documentation
=============================================
Getting started
---------------
* :ref:`getting-started` → deploy a local copy of the Software Heritage
software stack in less than 5 minutes, or
* :ref:`developer-setup` → get a working development setup that allows to hack
on the Software Heritage software stack
Contributing
------------
* :ref:`patch-submission` → learn how to submit your patches to the
Software Heritage codebase
* :ref:`code-review` → rules and guidelines to review code in
Software Heritage
* :ref:`python-style-guide` → how to format the Python code you write
Architecture
------------
* :ref:`architecture` → get a glimpse of the Software Heritage software
architecture
* :ref:`mirror` → learn what a Software Heritage mirror is and how to set up
one
* :ref:`keycloak ` → learn what keycloak is and how to register and configure users
Data Model and Specifications
-----------------------------
* :ref:`persistent-identifiers` Specifications of the SoftWare Heritage persistent IDentifiers (SWHID).
* :ref:`data-model` Documentation of the main |swh| archive data model.
* :ref:`journal-specs` Documentation of the Kafka journal of the |swh| archive.
Tutorials
---------
* :ref:`testing-guide`
* :doc:`/tutorials/issue-debugging-monitoring`
* :ref:`Listing the content of your favorite forge `
and :ref:`running a lister in Docker `
Roadmap
-------
* :ref:`roadmap-2021`
Components
----------
Here is brief overview of the most relevant software components in the Software
Heritage stack. Each component name is linked to the development documentation
of the corresponding Python module.
:ref:`swh.auth `
low-level library used by modules needing keycloak authentication
:ref:`swh.core `
low-level utilities and helpers used by almost all other modules in the
stack
:ref:`swh.counters `
service providing efficient estimates of the number of objects in the SWH archive,
using Redis's Hyperloglog
:ref:`swh.dataset `
public datasets and periodic data dumps of the archive released by Software
Heritage
:ref:`swh.deposit `
push-based deposit of software artifacts to the archive
swh.docs
developer documentation (used to generate this doc you are reading)
:ref:`swh.fuse `
Virtual file system to browse the Software Heritage archive, based on
`FUSE `_
:ref:`swh.graph `
Fast, compressed, in-memory representation of the archive, with tooling to
generate and query it.
:ref:`swh.indexer `
tools and workers used to crawl the content of the archive and extract
derived information from any artifact stored in it
:ref:`swh.journal `
persistent logger of changes to the archive, with publish-subscribe support
:ref:`swh.lister `
collection of listers for all sorts of source code hosting and distribution
places (forges, distributions, package managers, etc.)
:ref:`swh.loader-core `
low-level loading utilities and helpers used by all other loaders
:ref:`swh.loader-git `
loader for `Git `_ repositories
:ref:`swh.loader-mercurial `
loader for `Mercurial `_ repositories
:ref:`swh.loader-svn `
loader for `Subversion `_ repositories
:ref:`swh.model `
implementation of the :ref:`data-model` to archive source code artifacts
:ref:`swh.objstorage `
content-addressable object storage
:ref:`swh.objstorage.replayer `
Object storage replication tool
:ref:`swh.scanner `
source code scanner to analyze code bases and compare them with source code
artifacts archived by Software Heritage
:ref:`swh.scheduler `
task manager for asynchronous/delayed tasks, used for recurrent (e.g.,
listing a forge, loading new stuff from a Git repository) and one-off
activities (e.g., loading a specific version of a source package)
:ref:`swh.search `
search engine for the archive
:ref:`swh.storage `
abstraction layer over the archive, allowing to access all stored source
code artifacts as well as their metadata
:ref:`swh.vault `
implementation of the vault service, allowing to retrieve parts of the
archive as self-contained bundles (e.g., individual releases, entire
repository snapshots, etc.)
:ref:`swh.web `
Web application(s) to browse the archive, for both interactive (HTML UI)
and mechanized (REST API) use
:ref:`swh.web.client `
Python client for :ref:`swh.web `
Dependencies
------------
The dependency relationships among the various modules are depicted below.
.. _py-deps-swh:
.. figure:: images/py-deps-swh.svg
:width: 1024px
:align: center
Dependencies among top-level Python modules (click to zoom).
Archive
-------
* :ref:`Archive ChangeLog `: notable changes to the archive
over time
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* `URLs index `_
* :ref:`search`
* :ref:`glossary`
.. ensure sphinx does not complain about index files not being included
.. toctree::
:maxdepth: 2
:caption: Contents:
:titlesonly:
:hidden:
- getting-started-index
+ getting-started/index
architecture/index
contributing/index
tutorials/index
API documentation
roadmap/roadmap-2021.rst
swh.auth
swh.core
swh.counters
swh.dataset
swh.deposit
swh.fuse
swh.graph
swh.indexer
swh.journal
swh.lister
swh.loader
swh.model
swh.objstorage
swh.objstorage.replayer
swh.scanner
swh.scheduler
swh.search
swh.storage
swh.vault
swh.web
swh.web.client
archive-changelog
journal