diff --git a/docs/faq/index.rst b/docs/faq/index.rst
index a34932f..c4e4485 100644
--- a/docs/faq/index.rst
+++ b/docs/faq/index.rst
@@ -1,254 +1,259 @@
.. _faq:
Frequently Asked Questions
**************************
+.. contents::
+ :depth: 3
+ :local:
+..
+
.. _faq_prerequisites:
Prerequisites for code contributions
====================================
What are the Skills required to be a code contributor?
------------------------------------------------------
It depends on what area you want to work on. The `internships`_ postings
list specific skills
required. Generally, only Python and basic Git knowledge are required. Feel free to
contact us on one of the `development channels
`__ for details.
What are the minimum system requirements (hardware/software) to run SWH locally?
--------------------------------------------------------------------------------
Python3.7 or newer is required. See the :ref:`developer setup documentation
` for more details.
.. _faq_getting_started:
Getting Started
===============
What are the must read docs before I start contributing?
--------------------------------------------------------
We recommend you read the top links listed at from the :ref:`documentation home page
` in order: getting started,
contributing, and architecture, as well as the data model.
Where can I see the getting started guide for developers?
---------------------------------------------------------
See :ref:`getting-started` tutorial.
How do I find an easy ticket to get started?
--------------------------------------------
We maintain a `list of easy tickets
`__ to work on, see
the `Easy hacks page `__ for more
details.
I am skilled in one specific technology, can I find tickets requiring that skill?
---------------------------------------------------------------------------------
Unfortunately, not at the moment. But you can look at the `internships`_
list to look for something matching
this skill, and this may allow you to find topics to search for in the `bug tracking
system`_.
Either way, feel free to contact our developers through any of the
`development channels`_, we would love to work with
you.
Where should I ask for technical help?
--------------------------------------
You can choose one of the following:
* `development channels`_
* `contact form`_ for any enquiries
.. _faq_run_swh:
Running an SWH instance locally
===============================
I have SWH stack running in my local. How do I get some initial data to play around?
------------------------------------------------------------------------------------
You can setup a job on your local machine, for this you can
:ref:`schedule a listing task `
for example. Doing so on small forge, will allow you to load some repositories.
Or you can also trigger directly :ref:`loading from the cli `.
I have a SWH stack running in local, How do I setup a lister/loader job?
------------------------------------------------------------------------
See the :ref:`"Managing tasks" chapter `
in the Docker environment documentation.
How can I create a user in my local instance?
---------------------------------------------
We cannot right now. Stay either anonymous or use the user "test" (password "test") or
the user ambassador (password "ambassador").
Should I run/test the web app in any particular browser?
--------------------------------------------------------
We expect the web app to work on all major browsers. It uses mostly straightforward
HTML/CSS and a little Javascript for search and source code highlighting, so testing in
a single browser is usually enough.
.. _faq_dataset:
Getting sample datasets
=======================
Is there a way to connect to SWH archived (production) database from my local machine?
--------------------------------------------------------------------------------------
We provide the archive as a dataset on public clouds, see the :ref:`swh-dataset
documentation `. We can
also provide read access to one of the main databases on request, `contact us`_.
.. _faq_error_bugs:
Errors and bugs
===============
I found a bug/improvement in the system, where should I report it?
------------------------------------------------------------------
Please report it on our `bug tracking system`_.
First create an account, then create a bug report using the "Create task" button. You
should get some feedback within a week (at least someone triaging your issue). If not,
`get in touch with us `_ to
make sure we did not miss it.
.. _faq_legal:
Legal matters
=============
Do I need to sign a form to contribute code?
--------------------------------------------
Yes, on your first diff, you will have to sign such document.
As long as it's not signed, your diff content won't be visible.
Will my name be added to a CONTRIBUTORS file?
---------------------------------------------
You will be asked during review to add yourself.
.. _faq_code_review:
Code Review
===========
I found a straightforward typo fix, should my fix go through the entire code review process?
--------------------------------------------------------------------------------------------
You are welcome to drop us a message at one of the `development
channels`_, we will pick it up
and fix it so you don't have to follow the whole :ref:`code review process `.
What tests I should run before committing the code?
---------------------------------------------------
Mostly run `tox` (or `pytest`) to run the unit tests suite. When you will propose a
patch in our forge, the continuous integration factory will trigger a build (using `tox`
as well).
I am getting errors while trying to commit. What is going wrong?
----------------------------------------------------------------
Ensure you followed the proper guide to :ref:`setup your
environment `
and try again. If the error persists, you are welcome to drop us a message at one of the
`development channels`_
Is there a format/guideline for writing commit messages?
--------------------------------------------------------
See the :ref:`git-style-guide`
Is there some recommended git branching strategy?
-------------------------------------------------
It's left at the developer's discretion. Mostly people hack on their feature, then
propose a diff from a git branch or directly from the master branch. There is no
imperative. The only imperative is that for a feature to be packaged and deployed, it
needs to land first in the master branch.
how should I document the code I contribute to SWH?
---------------------------------------------------
Any new feature should include documentation in the form of comments and/or docstrings.
Ideally, they should also be documented in plain English in the repository's `docs/`
folder if relevant to a single package, or in the main `swh-docs` repository if it is a
transversal feature.
.. _faq_api:
Software Heritage API
=====================
How do I generate API usage credentials?
----------------------------------------
See the :ref:`Authentication guide `.
Is there a page where I can see all the API endpoints?
------------------------------------------------------
See the `API endpoint listing page`_.
What are the usage limits for SWH APIs?
---------------------------------------
Maximum number of permitted requests per hour:
* 120 for anonymous users
* 1200 for authenticated users
It's described in the `rate limit documentation page`_.
.. It's temporarily here but it should be moved into its own sphinx instance at some
point in the future.
.. _faq_sysadm:
System Administration
=====================
How does SWH release?
---------------------
Release is mostly done:
- first in docker (somewhat as part of the development process)
- secondly packaged and deployed on staging (mostly)
- thirdly the same package is deployed on production
Is there a release cycle?
-------------------------
When a functionality is ready (tests ok, landed in master, docker run ok), the module is
tagged. The tag is pushed. This triggers a packaging build process. When the package is
ready, depending on the module [1], sysadms deploy the package with the help of puppet.
[1] swh-web module is mostly automatic. Other modules are not yet automatic as some
internal state migration (dbs) often enters the release cycle and due to the data
volume, that may need human intervention.
.. _API endpoint listing page: https://archive.softwareheritage.org/api/1/
.. _rate limit documentation page: https://archive.softwareheritage.org/api/#rate-limiting
.. _bug tracking system: https://forge.softwareheritage.org/
.. _contact form: https://www.softwareheritage.org/contact/
.. _contact us: https://www.softwareheritage.org/contact/
.. _development channels: https://www.softwareheritage.org/community/developers/
.. _internships: https://wiki.softwareheritage.org/wiki/Internships
diff --git a/docs/index.rst b/docs/index.rst
index 42f95b4..4f94193 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -1,242 +1,235 @@
.. _swh-docs:
Software Heritage - Development Documentation
=============================================
Getting started
---------------
* :ref:`getting-started` → deploy a local copy of the Software Heritage
software stack in less than 5 minutes, or
* :ref:`developer-setup` → get a working development setup that allows to hack
on the Software Heritage software stack
+* :ref:`faq`
Contributing
------------
* :ref:`patch-submission` → learn how to submit your patches to the
Software Heritage codebase
* :ref:`code-review` → rules and guidelines to review code in
Software Heritage
* :ref:`python-style-guide` → how to format the Python code you write
Architecture
------------
* :ref:`architecture-overview` → get a glimpse of the Software Heritage software
architecture
* :ref:`mirror` → learn what a Software Heritage mirror is and how to set up
one
* :ref:`Metadata workflow ` → learn how Software Heritage
stores and handles metadata
* :ref:`Keycloak ` → learn how to use Keycloak,
the authentication system used by |swh|'s web interface and public APIs
Data Model and Specifications
-----------------------------
* :ref:`persistent-identifiers` Specifications of the SoftWare Heritage persistent IDentifiers (SWHID).
* :ref:`data-model` Documentation of the main |swh| archive data model.
* :ref:`journal-specs` Documentation of the Kafka journal of the |swh| archive.
Tutorials
---------
* :ref:`testing-guide`
* :doc:`/tutorials/issue-debugging-monitoring`
* :ref:`Listing the content of your favorite forge `
and :ref:`running a lister in Docker `
* :ref:`Add a new swh package `
* :ref:`doc-contribution`
-Frequently Asked Questions
---------------------------
-
-.. toctree::
- :maxdepth: 2
-
- faq/index
-
Roadmap
-------
* :ref:`roadmap-2021`
Engineering
-----------
* :ref:`infrastructure`
Components
----------
Here is brief overview of the most relevant software components in the Software
Heritage stack, in alphabetical order.
For a better introduction to the architecture, see the :ref:`architecture-overview`,
which presents each of them in a didactical order.
Each component name is linked to the development documentation
of the corresponding Python module.
:ref:`swh.auth `
low-level library used by modules needing keycloak authentication
:ref:`swh.core `
low-level utilities and helpers used by almost all other modules in the
stack
:ref:`swh.counters `
service providing efficient estimates of the number of objects in the SWH archive,
using Redis's Hyperloglog
:ref:`swh.dataset `
public datasets and periodic data dumps of the archive released by Software
Heritage
:ref:`swh.deposit `
push-based deposit of software artifacts to the archive
swh.docs
developer documentation (used to generate this doc you are reading)
:ref:`swh.fuse `
Virtual file system to browse the Software Heritage archive, based on
`FUSE `_
:ref:`swh.graph `
Fast, compressed, in-memory representation of the archive, with tooling to
generate and query it.
:ref:`swh.indexer `
tools and workers used to crawl the content of the archive and extract
derived information from any artifact stored in it
:ref:`swh.journal `
persistent logger of changes to the archive, with publish-subscribe support
:ref:`swh.lister `
collection of listers for all sorts of source code hosting and distribution
places (forges, distributions, package managers, etc.)
:ref:`swh.loader-core `
low-level loading utilities and helpers used by all other loaders
:ref:`swh.loader-git `
loader for `Git `_ repositories
:ref:`swh.loader-mercurial `
loader for `Mercurial `_ repositories
:ref:`swh.loader-svn `
loader for `Subversion `_ repositories
:ref:`swh.loader-cvs `
loader for `CVS `_ repositories
:ref:`swh.model `
implementation of the :ref:`data-model` to archive source code artifacts
:ref:`swh.objstorage `
content-addressable object storage
:ref:`swh.objstorage.replayer `
Object storage replication tool
:ref:`swh.scanner `
source code scanner to analyze code bases and compare them with source code
artifacts archived by Software Heritage
:ref:`swh.scheduler `
task manager for asynchronous/delayed tasks, used for recurrent (e.g.,
listing a forge, loading new stuff from a Git repository) and one-off
activities (e.g., loading a specific version of a source package)
:ref:`swh.search `
search engine for the archive
:ref:`swh.storage `
abstraction layer over the archive, allowing to access all stored source
code artifacts as well as their metadata
:ref:`swh.vault `
implementation of the vault service, allowing to retrieve parts of the
archive as self-contained bundles (e.g., individual releases, entire
repository snapshots, etc.)
:ref:`swh.web `
Web application(s) to browse the archive, for both interactive (HTML UI)
and mechanized (REST API) use
:ref:`swh.web.client `
Python client for :ref:`swh.web `
Dependencies
------------
The dependency relationships among the various modules are depicted below.
.. _py-deps-swh:
.. figure:: images/py-deps-swh.svg
:width: 1024px
:align: center
Dependencies among top-level Python modules (click to zoom).
Archive
-------
* :ref:`Archive ChangeLog `: notable changes to the archive
over time
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* `URLs index `_
* :ref:`search`
* :ref:`glossary`
.. ensure sphinx does not complain about index files not being included
.. toctree::
:maxdepth: 2
:caption: Contents:
:titlesonly:
:hidden:
getting-started/index
architecture/index
contributing/index
tutorials/index
faq/index
roadmap/roadmap-2021.rst
infrastructure/index
swh.auth
swh.core
swh.counters
swh.dataset
swh.deposit
swh.fuse
swh.graph
swh.indexer
swh.journal
swh.lister
swh.loader
swh.model
swh.objstorage
swh.objstorage.replayer
swh.scanner
swh.scheduler
swh.search
swh.storage
swh.vault
swh.web
swh.web.client
archive-changelog
journal
Python modules autodocumentation
diff --git a/user/faq/index.rst b/user/faq/index.rst
index 7bab05a..e80fa38 100644
--- a/user/faq/index.rst
+++ b/user/faq/index.rst
@@ -1,81 +1,86 @@
.. _faq:
Frequently Asked Questions
**************************
+.. contents::
+ :depth: 3
+ :local:
+..
+
.. _faq_roadmap:
Roadmap
=======
Where can I find the SWH roadmap?
---------------------------------
The roadmap is accessible on the :ref:`development docs `.
.. _faq_savecodenow:
Save Code Now
=============
What file formats are supported with the save code now?
-------------------------------------------------------
The save code now is intended publicly for code repositories with git, mercurial or svn
version control systems. For authenticated user with role ambassador, it is possible to
do save code now requests on zip or tarballs.
Is there a license existence check (the one included in the source code)?
-------------------------------------------------------------------------
No.
How are Save code now requests handled?
---------------------------------------
`Save code now `__ requests on known forges
for origins are scheduled as soon as possible. Unknown origins are put in a moderation
queue waiting for human vetting (Ambassadors or staff).
what file formats are supported with the save code now?
-------------------------------------------------------
The "save code now" supports public code repositories with git, mercurial or svn version
control systems. Ambassadors, however can also deposits for a given origins multiple
artifacts (zip, tarballs).
My "Save code now" request is stuck in pending status. what should I do?
------------------------------------------------------------------------
If the repository you want to save is already in the authorized list (e.g. GitLab,
GitHub, ...), the repositories will be saved without approval, so this should not last
more than a few hours. If it's not the case, requests should be approved within a few
days (minus French bank holidays), and loaded in a few hours.
If your repository is still pending after this time, this is most likely a bug. `Get in
touch with us `__ to check
whether we are aware of this potential issue or are working on it.
.. _faq_search:
Search
======
I cannot find all my "releases" in a git repository archived in Software Heritage, what should I do?
----------------------------------------------------------------------------------------------------
Do not worry, the repository has been saved in full. What you are witnessing is just a
terminological difference between what platforms like GitHub calls “releases” (any non
annotated git tag) and what we call “releases” (a node in the Merkle tree, which
corresponds to a git annotated tag). Let’s say your “release” is FinalSubmission. If you
click on the branch dropdown menu on the Software Heritage Web interface you’ll find
what you are looking listed as “refs/tags/FinalSubmission”. If you want a “release” to
appear in our web interface you should create your tags using “git tag -a”, instead of
simply “git tag” (and then archive your repository again).
How can I search the SWH archive? is it possible to search over metadata?
-------------------------------------------------------------------------
At the moment searching is possible using the url of a repository, package or deposit
(a.k.a the origin of the source code). You can use the checkbox "search in metadata
(instead of URL)" to search over intrinsic metadata.
diff --git a/user/index.rst b/user/index.rst
index eb5e7f3..75bbe39 100644
--- a/user/index.rst
+++ b/user/index.rst
@@ -1,44 +1,35 @@
.. _swh-docs:
.. Software Heritage documentation master file, created by
sphinx-quickstart on Thu Jun 17 11:23:12 2021.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Software Heritage User documentation
====================================
Getting started
---------------
-
-...
+* :ref:`faq`
Tutorials
---------
...
-Frequently Asked Questions
----------------------------
-
-.. toctree::
- :maxdepth: 2
-
- faq/index
-
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
.. toctree::
:maxdepth: 2
:caption: Contents:
:titlesonly:
:hidden:
faq/index