$ cd $SWH_ENVIRONMENT_HOME/swh-loader-git
$ pip install -e .
$ swh loader run --help
WARNING:swh.core.cli:Could not load subcommand graph: module 'swh.graph.cli' has no attribute 'cli'
WARNING:swh.core.cli:Could not load subcommand dataset: module 'swh.dataset.cli' has no attribute 'cli'
Usage: swh loader run [OPTIONS] [archive|cran|debian|deposit|git|git_disk|merc
                      urial|nixguix|npm|pypi|svn] URL [OPTIONS]...

  Ingest with loader <type> the origin located at <url>

Options:
  -h, --help  Show this message and exit.

swh loader run git_disk is there alright.

docker:

$ doco up -d  # docker-compose.override.yml with loader-git mount
$ doco exec swh-loader /bin/bash
swh@40b47a9a723a:/$ cd
swh@40b47a9a723a:/$ swh loader run git_disk https://swh/loader.git directory=/src/swh-loader-git/
INFO:swh.loader.git.Loader:Load origin 'https://swh/loader.git' with type 'git'
{'status': 'eventful'}

$ psql service=swh-dev -c "select url from origin where url like 'https://swh/%'"
          url
------------------------
 https://swh/loader.git  # <- right url
(1 row)

unset PAGER; psql service=swh-dev -c "select count(*) from content"
-------
  2719
(1 row)

ok then!

Diff Detail

Repository

rDLDG Git loader

Branch

master

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 17074
Build 26354: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 26353: arc lint + arc unit

Event Timeline

ardumont created this revision.Nov 13 2020, 10:04 AM

Herald added a reviewer: Reviewers. · View Herald TranscriptNov 13 2020, 10:04 AM

Build is green

Patch application report for D4470 (id=15864)

Rebasing onto 9eb1c08f1c...

Current branch diff-target is up to date.

Changes applied before test

commit 18e32073e6cf043de35ddd82ab0c58586e2d6471
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Nov 12 17:32:30 2020 +0100

    loader.git.from_disk: Register loader in `swh loader run` cli
    
    This drops the older cli in swh.loader.git.from_disk which was broken and not
    covered by test.
    
    Related to T2770#52497

See https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/69/ for more details.

Harbormaster completed remote builds in B17073: Diff 15864.Nov 13 2020, 10:05 AM

ardumont edited the test plan for this revision. (Show Details)Nov 13 2020, 10:07 AM

olasd accepted this revision.Nov 13 2020, 10:08 AM

olasd added a subscriber: olasd.

olasd added inline comments.

swh/loader/git/__init__.py
21	Should probably be an empty list, to avoid getting duplicates; Even though I don't know if this really matters.

This revision is now accepted and ready to land.Nov 13 2020, 10:08 AM

ardumont added inline comments.Nov 13 2020, 10:09 AM

swh/loader/git/__init__.py
21	yes, i don't really know what either entails... i'll check for the empty list.

ardumont edited the test plan for this revision. (Show Details)Nov 13 2020, 10:19 AM

Make the task list the empty one, those task are already registered for the
main git loader.

ardumont mentioned this in D4467: loader.git.from_disk: Fix cli.Nov 13 2020, 10:21 AM

Build is green

Patch application report for D4470 (id=15865)

Rebasing onto 9eb1c08f1c...

Current branch diff-target is up to date.

Changes applied before test

commit 9ad25545de101ab24af4d23b230ccc60607747f9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Nov 12 17:32:30 2020 +0100

    loader.git.from_disk: Register loader in `swh loader run` cli
    
    This drops the older cli in swh.loader.git.from_disk which was broken and not
    covered by test.
    
    Related to T2770#52497

See https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/70/ for more details.

Harbormaster completed remote builds in B17074: Diff 15865.Nov 13 2020, 10:22 AM

Closed by commit rDLDG9ad25545de10: loader.git.from_disk: Register loader in `swh loader run` cli (authored by ardumont). · Explain WhyNov 13 2020, 10:22 AM

This revision was automatically updated to reflect the committed changes.

ardumont added a commit: rDLDG9ad25545de10: loader.git.from_disk: Register loader in `swh loader run` cli.