diff --git a/PKG-INFO b/PKG-INFO index cfaf252..1c4d61b 100644 --- a/PKG-INFO +++ b/PKG-INFO @@ -1,10 +1,10 @@ Metadata-Version: 1.0 Name: swh.loader.dir -Version: 0.0.5 +Version: 0.0.6 Summary: Software Heritage Directory Loader Home-page: https://forge.softwareheritage.org/diffusion/DLDDIR Author: Software Heritage developers Author-email: swh-devel@inria.fr License: UNKNOWN Description: UNKNOWN Platform: UNKNOWN diff --git a/README b/README index 0370631..2287724 100644 --- a/README +++ b/README @@ -1,87 +1,95 @@ SWH-loader-dir ============== The Software Heritage Directory Loader is a tool and a library to walk a local directory and inject into the SWH dataset all unknown contained files. -Configuration sample -==================== -### Directory loader +Directory loader +================ -#### Configuration + +### Configuration This is the loader's (or task's) configuration file. loader/dir.ini: [main] # access to swh's storage storage_class = remote_storage storage_args = http://localhost:5000/ # parameters to condition loading into swh storage send_contents = True send_directories = True send_revisions = True send_releases = True send_occurrences = True content_packet_size = 10000 content_packet_size_bytes = 1073741824 directory_packet_size = 25000 revision_packet_size = 100000 release_packet_size = 100000 occurrence_packet_size = 100000 Present in possible locations: - ~/.config/swh/loader/dir.ini - ~/.swh/loader/dir.ini - /etc/softwareheritage/loader/dir.ini -#### toplevel + +#### Toplevel Load directory directly from code or toplevel: from swh.loader.dir.tasks import LoadDirRepository dir_path = '/path/to/directory # Fill in those origin = {} release = None revision = {} occurrence = {} LoadDirRepository().run(dir_path, origin, revision, release, [occurrence]) -#### celery + +#### Celery Load directory using celery. Providing you have a properly configured celery up and running worker.ini needs to be updated with the following keys: task_modules = swh.loader.dir.tasks task_queues = swh_loader_dir cf. https://forge.softwareheritage.org/diffusion/DCORE/browse/master/README.md for more details You can send the following message to the task queue: from swh.loader.dir.tasks import LoadDirRepository # Fill in those origin = {} release = None revision = {} occurrence = {} # Send message to the task queue LoadDirRepository().apply_async(('/path/to/dir, origin, revision, release, [occurrence])) + + +Directory producer +================== + +None diff --git a/debian/control b/debian/control index 813affc..447ab64 100644 --- a/debian/control +++ b/debian/control @@ -1,19 +1,22 @@ Source: swh-loader-dir Maintainer: Software Heritage developers Section: python Priority: optional Build-Depends: debhelper (>= 9), dh-python, python3-all, python3-nose, python3-setuptools, - python3-swh.core, + python3-swh.core (>= 0.0.14~), python3-swh.storage, python3-vcversioner Standards-Version: 3.9.6 Homepage: https://forge.softwareheritage.org/diffusion/DLDDIR/ Package: python3-swh.loader.dir Architecture: all -Depends: ${misc:Depends}, ${python3:Depends} +Depends: python3-swh.core (>= 0.0.14~), + python3-swh.storage, + ${misc:Depends}, + ${python3:Depends} Description: Software Heritage Directory Loader diff --git a/swh.loader.dir.egg-info/PKG-INFO b/swh.loader.dir.egg-info/PKG-INFO index cfaf252..1c4d61b 100644 --- a/swh.loader.dir.egg-info/PKG-INFO +++ b/swh.loader.dir.egg-info/PKG-INFO @@ -1,10 +1,10 @@ Metadata-Version: 1.0 Name: swh.loader.dir -Version: 0.0.5 +Version: 0.0.6 Summary: Software Heritage Directory Loader Home-page: https://forge.softwareheritage.org/diffusion/DLDDIR Author: Software Heritage developers Author-email: swh-devel@inria.fr License: UNKNOWN Description: UNKNOWN Platform: UNKNOWN diff --git a/version.txt b/version.txt index 0297abe..59e18f8 100644 --- a/version.txt +++ b/version.txt @@ -1 +1 @@ -v0.0.5-0-g7c8feab \ No newline at end of file +v0.0.6-0-g8bba640 \ No newline at end of file