Page MenuHomeSoftware Heritage
Files F9345927

D5968.id21495.diff
No OneTemporary
Actions

D5968.id21495.diff
View Options

diff --git a/docs/index.rst b/docs/index.rst
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -44,6 +44,7 @@
* :doc:`/tutorials/issue-debugging-monitoring`
* :ref:`Listing the content of your favorite forge <lister-tutorial>`
and :ref:`running a lister in Docker <run-lister-tutorial>`
+* ref:`Add a new swh module <add-a-new-module>`
Frequently Asked Questions
--------------------------
diff --git a/docs/tutorials/add-a-new-module.rst b/docs/tutorials/add-a-new-module.rst
new file mode 100644
--- /dev/null
+++ b/docs/tutorials/add-a-new-module.rst
@@ -0,0 +1,143 @@
+.. _add-a-new-module:
+
+Add a new module
+================
+
+The following document demonstrates how to create a new swh module in the current `swh
+phabricator instance`_ and reference it so the `Continuous Integration (CI)`_ is able to
+build the new module.
+
+We will need to (optionally) create a project, then create a repository, reference it in
+the CI and finally update the documentation repository so the docs get built with that
+new module. Optionally, we can also create the necessary debian branches for the debian
+build deployment process to work. That's not something immediately urgent when
+bootstraping a module though.
+
+Create a project
+----------------
+
+Create a `new Project`_ (seen also as a ``Tag``) and fill in the form:
+
+ - Fill in the "name" field (e.g. ``Origin Sourceforge``, ``Loader XYZ``, ...)
+
+ - Additional hashtags should be filled in with easy to remember hashstags (e.g.
+ ``#sourceforge``, ``#loader-xyz``)
+
+ - Add a small description about what the project is about
+
+Create a repository
+-------------------
+
+- Create a `new Git repository`_ and fill in the form:
+
+- ``name`` of the repository should be human readable (e.g. ``Loader Git``, ``Lister Gitlab``)
+
+- ``callsign`` should follow the current `naming convention`_
+
+- ``short-name`` should be lower case and dash separated (e.g. swh-loader-git, swh-lister, ...)
+
+- ``Tags`` should be filled with:
+
+ - the first Project/tag created in the `optional first step <#create-a-project>`__
+
+ - ``Language-Python``: Enable unit tests execution on commit
+
+ - ``Has debian packaging branches``: Activate debian builds in the CI
+
+ - ``Sync to github``: (optional) Activate mirroring sync to github
+
+- Add the staging area, click in order from ``BUILDS`` (left menu) > ``Staging Area`` >
+ ``Edit staging`` > fill in ``Staging Area URI`` with
+ https://forge.softwareheritage.org/source/staging.git
+
+- Finally, activate the repository
+
+Add the repo on the swh-environment project
+-------------------------------------------
+
+(Only if necessary)
+
+Unless specified otherwise, the following commands need to run from the base directory
+``swh-environment``.
+
+- clone the new repository
+
+.. code:: bash
+
+ git clone https://forge.softwareheritage.org/source/swh-my-new-repo.git
+
+- launch the command ``bin/init-py-repo`` to initialize the repository with a project
+ template
+
+.. code:: bash
+
+ bin/init-py-repo swh-new-repo
+
+- Edit the default content of the template (`Example <https://forge.softwareheritage.org/rDCNT142fff84305b793974e6f7b837988e5fb95d8db1>`__)
+
+- Configure **your local** pre-commit hook
+
+ - In the ``swh-environment/swh-my-new-repo`` directory, execute:
+
+ .. code:: bash
+
+ grep -q pre-commit.com .git/hooks/pre-commit || pre-commit install
+
+- Declare the repository on the mr configuration
+
+ - Edit the ``.mrconfig`` file and declare the new repository (just
+ duplicate one existing entry and adapt with the new module name)
+
+ - Commit file modification (`Example <https://forge.softwareheritage.org/rCJSWHede4a65bc9e103db99dd8b0690caa3a769b378bd>`__)
+
+Install CI jobs
+---------------
+
+- In the swh-jenkins-jobs_ repository, open the
+ ``jobs/swh-packages.yaml`` and add a section for the new repository as for the others
+ (`Example <https://forge.softwareheritage.org/rCJSWHdd5b3a1192cb45c07103be199af8c2a74478746e>`__)
+
+- Configure the `post-receive hook`_ on the phabricator instance
+
+- `Setting up the debian jenkins jobs`_
+
+`Setting up debian builds <https://wiki.softwareheritage.org/wiki/Debian_packaging#Git_repositories_for_Debian_packages>`__
+---------------------------------------------------------------------------------------------------------------------------
+
+As mentioned early in the introduction, this can be delayed for when the module is
+actually ready to be deployed.
+
+Documentation updates
+---------------------
+
+- Documentation repository is located in the swh-docs_ repository.
+
+- Add the new module dependency in the top-level ``requirements-swh.txt`` and
+ ``requirements-swh-dev.txt``
+
+- Add the new module entry in ``docs/index.rst`` with a concise description of the
+ module
+
+::
+
+ :ref:`swh.auth <swh-auth>`
+ low-level library used by modules needing keycloak authentication
+ ...
+
+ # at the end of the index page
+ swh.auth <swh-auth/index>
+
+- ensure this builds fine locally (e.g run `tox`, then `make -C docs`)
+
+- Then open a diff to advertise the new documentation entrypoints (`Example <https://forge.softwareheritage.org/D5327>`__)
+
+
+.. _`swh phabricator instance`: https://forge.softwareheritage.org/
+.. _`Continuous Integration (CI)`: https://jenkins.softwareheritage.org
+.. _`new Project`: https://forge.softwareheritage.org/project/edit/form/3/
+.. _`new Git repository`: https://forge.softwareheritage.org/diffusion/edit/form/default/?vcs=git
+.. _`naming convention`: https://wiki.softwareheritage.org/wiki/Phabricator_callsign_naming_convention
+.. _swh-jenkins-jobs: https://forge.softwareheritage.org/source/swh-jenkins-jobs
+.. _`post-receive hook`: https://wiki.softwareheritage.org/wiki/Debian_packaging#Setting_up_the_repository_on_Phabricator
+.. _`Setting up the debian jenkins jobs`: https://wiki.softwareheritage.org/wiki/Debian_packaging#Setting_up_the_Jenkins_jobs
+.. _swh-docs: https://forge.softwareheritage.org/source/swh-docs/
diff --git a/docs/tutorials/index.rst b/docs/tutorials/index.rst
--- a/docs/tutorials/index.rst
+++ b/docs/tutorials/index.rst
@@ -9,3 +9,4 @@
testing
issue-debugging-monitoring
+ add-a-new-module

File Metadata

Mime Type
text/plain
Expires
Thu, Jul 3, 3:36 PM (1 w, 2 d ago)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
3229465

Event Timeline

About · Developer information · Legal