Changeset View
Changeset View
Standalone View
Standalone View
README.md
swh-loader-mercurial | swh-loader-mercurial | ||||
========================= | ========================= | ||||
# Configuration file | The Software Heritage Mercurial Loader is a tool and a library to walk a local mercurial | ||||
repository and inject into the SWH dataset all contained files that weren't known | |||||
before. | |||||
In usual location for a loader, *{/etc/softwareheritage/ | ~/.swh/ | | The main entry points are: | ||||
~/.config/swh/}loader/hg.yml*: | - :class:`swh.loader.mercurial.loader.HgBundle20Loader` which reads and loads a remote | ||||
repository (or local). | |||||
- :class:`swh.loader.mercurial.from_disk.HgLoaderFromDisk` which reads and loads a local | |||||
repository into an SWH archive. | |||||
- :class:`swh.loader.mercurial.from_disk.HgArchiveLoaderFromDisk` which reads and loads | |||||
a local repository wrapped within a tarball | |||||
# CLI run | |||||
## Configuration file | |||||
/tmp/mercurial.yml: | |||||
``` YAML | ``` YAML | ||||
storage: | storage: | ||||
cls: remote | cls: remote | ||||
args: | args: | ||||
url: http://localhost:5002/ | url: http://localhost:5002/ | ||||
``` | ``` | ||||
# Basic use | ## Basic use | ||||
Alphare: Maybe add a mention that existing config files exist in the `docker` folder? | |||||
Done Inline ActionsThanks for the suggestion but im not sure it's really the place. Also that'd mean to quadruple this in other loaders (svn, core, mercurial, git) ardumont: Thanks for the suggestion but im not sure it's really the place.
Also that'd mean to quadruple… | |||||
The main entry point to import a Mercurial repository is the `main` function | |||||
defined in the `swh.loader.mercurial.cli` module: | |||||
``` bash | |||||
python3 -m swh.loader.mercurial.cli | |||||
``` | |||||
If the Python package has been installed via `pip`, you should be able | |||||
to type: | |||||
``` bash | |||||
user@host:~$ swh-loader-hg --help | |||||
Usage: swh-loader-hg [OPTIONS] ORIGIN_URL | |||||
Options: | |||||
-d, --hg-directory TEXT Path to the hg (local) directory to load | |||||
from. If unset, the hg repo will ben cloned | |||||
from the given (origin) url | |||||
-a, --hg-archive TEXT Path to the hg (local) archive file to load | |||||
from. | |||||
-D, --visit-date TEXT Visit date (defaults to now) | |||||
-l, --log-level [NOTSET|DEBUG|INFO|WARNING|ERROR|CRITICAL] | |||||
Log level | |||||
--help Show this message and exit. | |||||
``` | |||||
For example: | |||||
``` bash | ``` bash | ||||
user@host:~$ swh-loader-hg https://www.mercurial-scm.org/repo/hello | swh loader --C /tmp/mercurial.yml run mercurial https://www.mercurial-scm.org/repo/hello | ||||
[...] | |||||
``` | |||||
# From Python | |||||
From python3's toplevel: | |||||
## Remote | |||||
``` Python | |||||
project = 'hello' | |||||
# remote repository | |||||
origin_url = 'https://www.mercurial-scm.org/repo/%s' % project | |||||
# local clone | |||||
directory = '/home/storage/hg/repo/%s' % project | |||||
import logging | |||||
logging.basicConfig(level=logging.DEBUG) | |||||
from swh.loader.mercurial.tasks import LoadMercurial | |||||
t = LoadMercurial() | |||||
t.run(origin_url=origin_url, directory=directory, visit_date='2016-05-03T15:16:32+00:00') | |||||
``` | |||||
## local directory | |||||
Only origin, contents, and directories are filled so far. | |||||
Remaining objects are empty (revision, release, occurrence). | |||||
``` Python | |||||
project = '756015-ipv6' | |||||
directory = '/home/storage/hg/repo/%s' % project | |||||
origin_url = 'https://%s.googlecode.com' % project | |||||
import logging | |||||
logging.basicConfig(level=logging.DEBUG) | |||||
from swh.loader.mercurial.tasks import LoadMercurial | |||||
t = LoadMercurial() | |||||
t.run(origin_url=origin_url, directory=directory, visit_date='2016-05-03T15:16:32+00:00') | |||||
``` | |||||
## local archive | |||||
``` Python | |||||
project = '756015-ipv6-source-archive.zip' | |||||
archive_path = '/home/storage/hg/repo/%s' % project | |||||
origin_url = 'https://%s-archive.googlecode.com' % project | |||||
import logging | |||||
logging.basicConfig(level=logging.DEBUG) | |||||
from swh.loader.mercurial.tasks import LoadArchiveMercurial | |||||
t = LoadArchiveMercurial() | |||||
t.run(origin_url=origin_url, archive_path=archive_path, visit_date='2016-05-03T15:16:32+00:00') | |||||
``` | ``` | ||||
or `mercurial_from_disk` |
Maybe add a mention that existing config files exist in the docker folder?