Page MenuHomeSoftware Heritage

loader.git.from_disk: Register loader in `swh loader run` cli
ClosedPublic

Authored by ardumont on Fri, Nov 13, 10:04 AM.

Details

Summary

This drops the older cli in swh.loader.git.from_disk which was broken and not
covered by test.

Related to T2770#52497

Supersedes D4467

Test Plan

tox happy

And manual check:

$ cd $SWH_ENVIRONMENT_HOME/swh-loader-git
$ pip install -e .
$ swh loader run --help
WARNING:swh.core.cli:Could not load subcommand graph: module 'swh.graph.cli' has no attribute 'cli'
WARNING:swh.core.cli:Could not load subcommand dataset: module 'swh.dataset.cli' has no attribute 'cli'
Usage: swh loader run [OPTIONS] [archive|cran|debian|deposit|git|git_disk|merc
                      urial|nixguix|npm|pypi|svn] URL [OPTIONS]...

  Ingest with loader <type> the origin located at <url>

Options:
  -h, --help  Show this message and exit.

swh loader run git_disk is there alright.

docker:

$ doco up -d  # docker-compose.override.yml with loader-git mount
$ doco exec swh-loader /bin/bash
swh@40b47a9a723a:/$ cd
swh@40b47a9a723a:/$ swh loader run git_disk https://swh/loader.git directory=/src/swh-loader-git/
INFO:swh.loader.git.Loader:Load origin 'https://swh/loader.git' with type 'git'
{'status': 'eventful'}

$ psql service=swh-dev -c "select url from origin where url like 'https://swh/%'"
          url
------------------------
 https://swh/loader.git  # <- right url
(1 row)

unset PAGER; psql service=swh-dev -c "select count(*) from content"
-------
  2719
(1 row)

ok then!

Diff Detail

Repository
rDLDG Git loader
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Fri, Nov 13, 10:04 AM

Build is green

Patch application report for D4470 (id=15864)

Rebasing onto 9eb1c08f1c...

Current branch diff-target is up to date.
Changes applied before test
commit 18e32073e6cf043de35ddd82ab0c58586e2d6471
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Nov 12 17:32:30 2020 +0100

    loader.git.from_disk: Register loader in `swh loader run` cli
    
    This drops the older cli in swh.loader.git.from_disk which was broken and not
    covered by test.
    
    Related to T2770#52497

See https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/69/ for more details.

ardumont edited the test plan for this revision. (Show Details)Fri, Nov 13, 10:07 AM
olasd accepted this revision.Fri, Nov 13, 10:08 AM
olasd added a subscriber: olasd.
olasd added inline comments.
swh/loader/git/__init__.py
22

Should probably be an empty list, to avoid getting duplicates; Even though I don't know if this really matters.

This revision is now accepted and ready to land.Fri, Nov 13, 10:08 AM
ardumont added inline comments.Fri, Nov 13, 10:09 AM
swh/loader/git/__init__.py
22

yes, i don't really know what either entails...
i'll check for the empty list.

ardumont edited the test plan for this revision. (Show Details)Fri, Nov 13, 10:19 AM
ardumont updated this revision to Diff 15865.Fri, Nov 13, 10:20 AM

Make the task list the empty one, those task are already registered for the
main git loader.

Build is green

Patch application report for D4470 (id=15865)

Rebasing onto 9eb1c08f1c...

Current branch diff-target is up to date.
Changes applied before test
commit 9ad25545de101ab24af4d23b230ccc60607747f9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Nov 12 17:32:30 2020 +0100

    loader.git.from_disk: Register loader in `swh loader run` cli
    
    This drops the older cli in swh.loader.git.from_disk which was broken and not
    covered by test.
    
    Related to T2770#52497

See https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/70/ for more details.