Differential D5968 Diff 21486 docs/tutorials/add-a-new-module.rst

Changeset View

Standalone View

View Options

docs/tutorials/add-a-new-module.rst

This file was added.

.. _add-a-new-module:

vlorentzUnsubmitted

Done

I try to avoid stop words in anchor names. What about .. _tutorial-new-package:?

vlorentz: I try to avoid stop words in anchor names. What about `.. _tutorial-new-package:`?

ardumontAuthorUnsubmitted

Done

Sure.

Note that i think it's an how to though (see diff description) but fine.

ardumont: Sure. Note that i think it's an how to though (see diff description) but fine.

Add a new module

================

The following document demonstrates how to create a new swh module in the current `swh

phabricator instance`_ and reference it so the `Continuous Integration (CI)`_ is able to

build the new module.

We will need to (optionally) create a project, then create a repository, reference it in

the CI and finally update the documentation repository so the docs get built with that

new module. Optionally, we can also create the necessary debian branches for the debian

build deployment process to work. That's not something immediately urgent when

bootstraping a module though.

Create a project

----------------

Create a `new Project`_ (seen also as a ``Tag``) and fill in the form:

- Fill in the "name" field (e.g. ``Origin Sourceforge``, ``Loader XYZ``, ...)

- Additional hashtags should be filled in with easy to remember hashstags (e.g.

``#sourceforge``, ``#loader-xyz``)

vlorentzUnsubmitted

Done

- Fill in the "name" field (e.g. ``Origin Sourceforge``, ``Loader XYZ``, ...)

- - Additional hashtags should be filled in with easy to remember hashstags (e.g.

+ - Additional hashtags should be filled in with easy to remember hashtags (e.g.

``#sourceforge``, ``#loader-xyz``)

- Add a small description about what the project is about

vlorentz:

- Add a small description about what the project is about

Create a repository

-------------------

- Create a `new Git repository`_ and fill in the form:

- ``name`` of the repository should be human readable (e.g. ``Loader Git``, ``Lister Gitlab``)

vlorentzUnsubmitted

Done

- Create a `new Git repository`_ and fill in the form:

- - ``name`` of the repository should be human readable (e.g. ``Loader Git``, ``Lister Gitlab``)

+ - ``name`` of the repository should be human readable (e.g. ``Git Loader``, ``Gitlab Lister``)

- ``callsign`` should follow the current `naming convention`_

vlorentz:

- ``callsign`` should follow the current `naming convention`_

- ``short-name`` should be lower case and dash separated (e.g. swh-loader-git, swh-lister, ...)

- ``Tags`` should be filled with the project/tag you created in the `optional first step <#create-a-project>`__

- Add tags in the ``Tags`` form:

vlorentzUnsubmitted

Done

Define an anchor for Create a project and use a :ref: to point to it.

Written this way, it depends on how the HTML renderer auto-generates anchors, which is somewhat brittle. (And also HTML-specific)

vlorentz: Define an anchor for `Create a project` and use a `:ref:` to point to it. Written this way, it…

- ``Language-Python``: Enable unit tests execution on commit

- ``Has debian packaging branches``: Activate debian builds in the CI

- ``Sync to github``: Activate sync to github

vlorentzUnsubmitted

Done

This should probably explain when they should be enabled

vlorentz: This should probably explain when they should be enabled

ardumontAuthorUnsubmitted

Done

yes, i did not enter much into it because 1. i don't remember what needs to be done after that 2. and i think that'd need a dedicated doc...

ardumont: yes, i did not enter much into it because 1. i don't remember what needs to be done after that…

- Add the staging area, click in order from ``BUILDS`` (left menu) > ``Staging Area`` >

``Edit staging`` > fill in ``Staging Area URI`` with

https://forge.softwareheritage.org/source/staging.git

- Finally, activate the repository

Add the repo on the swh-environment project

-------------------------------------------

(Only if necessary)

Unless specified otherwise, the following commands need to run from the base directory

``swh-environment``.

- clone the new repository

.. code:: bash

git clone https://forge.softwareheritage.org/source/swh-my-new-repo.git

- launch the command ``bin/init-py-repo`` to initialize the repository with a project

template

.. code:: bash

bin/init-py-repo swh-new-repo

- Edit the default content of the template (`Example <https://forge.softwareheritage.org/rDCNT142fff84305b793974e6f7b837988e5fb95d8db1>`__)

- Configure **your local** pre-commit hook

- In the ``swh-environment/swh-my-new-repo`` directory, execute:

.. code:: bash

grep -q pre-commit.com .git/hooks/pre-commit || pre-commit install

- Declare the repository on the mr configuration

- Edit the ``.mrconfig`` file and declare the new repository (just

duplicate one existing entry and adapt with the new module name)

- Commit file modification (`Example <https://forge.softwareheritage.org/rCJSWHede4a65bc9e103db99dd8b0690caa3a769b378bd>`__)

Install CI jobs

---------------

- In the swh-jenkins-jobs_ repository, open the

``jobs/swh-packages.yaml`` and add a section for the new repository as for the others

(`Example <https://forge.softwareheritage.org/rCJSWHdd5b3a1192cb45c07103be199af8c2a74478746e>`__)

- Configure the `post-receive hook`_ on the phabricator instance

- `Setting up the debian jenkins jobs`_

`Setting up debian builds <https://wiki.softwareheritage.org/wiki/Debian_packaging#Git_repositories_for_Debian_packages>`__

---------------------------------------------------------------------------------------------------------------------------

vlorentzUnsubmitted

Done

Can you move the link inside the paragraph?

vlorentz: Can you move the link inside the paragraph?

ardumontAuthorUnsubmitted

Done

sure, i did not know how to properly add this.
Also, initially the paragraph was empty.

ardumont: sure, i did not know how to properly add this. Also, initially the paragraph was empty.

As mentioned early in the introduction, this can be delayed for when the module is

actually ready to be deployed.

Documentation updates

---------------------

- Documentation repository is located in the swh-docs_ repository.

- Add the new module dependency in the top-level ``requirements-swh.txt`` and

``requirements-swh-dev.txt``

- Add the new module entry in ``docs/index.rst`` with a concise description of the

module

:ref:`swh.auth <swh-auth>`

low-level library used by modules needing keycloak authentication

...

# at the end of the index page

swh.auth <swh-auth/index>

vlorentzUnsubmitted

Done

- :ref:`swh.auth <swh-auth>`

- low-level library used by modules needing keycloak authentication

+ :ref:`swh.auth <swh-my-new-repo>`

+ short description of the repository

...

# at the end of the index page

- swh.auth <swh-auth/index>

+ swh.my_new_repo <swh-my-new-repo/index>

- ensure this builds fine locally (e.g run `tox`, then `make -C docs`)

vlorentz:

- ensure this builds fine locally (e.g run `tox`, then `make -C docs`)

- Then open a diff to advertise the new documentation entrypoints (`Example <https://forge.softwareheritage.org/D5327>`__)

.. _`swh phabricator instance`: https://forge.softwareheritage.org/

.. _`Continuous Integration (CI)`: https://jenkins.softwareheritage.org

.. _`new Project`: https://forge.softwareheritage.org/project/edit/form/3/

.. _`new Git repository`: https://forge.softwareheritage.org/diffusion/edit/form/default/?vcs=git

.. _`naming convention`: https://wiki.softwareheritage.org/wiki/Phabricator_callsign_naming_convention

.. _swh-jenkins-jobs: https://forge.softwareheritage.org/source/swh-jenkins-jobs

.. _`post-receive hook`: https://wiki.softwareheritage.org/wiki/Debian_packaging#Setting_up_the_repository_on_Phabricator

.. _`Setting up the debian jenkins jobs`: https://wiki.softwareheritage.org/wiki/Debian_packaging#Setting_up_the_Jenkins_jobs

.. _swh-docs: https://forge.softwareheritage.org/source/swh-docs/