Page MenuHomeSoftware Heritage

Gather CLI tools spread in SWH packages
Closed, ResolvedPublic

Event Timeline

douardda triaged this task as Normal priority.Apr 18 2019, 9:36 AM
douardda created this task.

Currently, we have:

  • swh.archiver
    • run an Archiver Director (dispatching batch of contents to archiver workers)
    • run an Updater (Journal client in charge of updating new contents in the content_archiver db)
    • run a Checker (several content integrity checkers)
  • swh.core
    • db initialization tool
  • swh.deposit
    • cli/ main swh-deposit command
    • cli/ deposit (client) tool to upload deposit artifacts
    • cli/ management tools for the deposit server
  • swh.indexer
    • main cli entry point (swh-indexer) with mapping and task management tools plus the API/RPC server command
    • example tool to log origins and origin_visit received in the journal
    • storage/api/ deprecated (in favor of swh-indexer)
    • metadata_dictionary/ not sure what this tool is for
    • seems like a utility tool to look for origins in the origin indexer DB
  • swh.journal
    • main cli entry point (swh-journal) + the replay subcommand (fill a new storage by reading a journal) and the backfill one (manipulate the backfiller)
  • swh.lister
    • main swh-lister command (initialize a lister's DB)
    • debian/ debian specific utilities (list registered distributions, add one, etc.)
  • swh.loader.debian
    • debian/ (example) tool to load a given debian package in the archive
  • swh.loader.dir
    • dir/ (example) tool to load a given source code directory in the archive
  • swh.loader.git
    • git/ (example) tool to load a given remote git repo in the archive
    • git/ (example) tool to load a given local git repo in the archive
  • swh.loader.mercurial
    • mercurial/ (example) tool to load a given hg repository in the archive
  • swh.loader.pypi
    • pypi/ (example) tool to load a given Python source package in the archive
  • swh.loader.svn
    • svn/ (example) tool to load a given svn repository in the archive
  • swh.model
    • swh-identity tool (Compute the Software Heritage persistent identifier (PID) for the given source code object(s))
  • swh.objstorage
    • main swh-objstorage command with serve, fsck and import subcommands
  • swh.scheduler
    • main swh-scheduler command with many server related subcommands (API server, runner, listener, ghtorrent client and updater) and task management (client) related subcommands
    • bin/swh-worker-control
    • API server
  • swh.vault
    • API server
  • swh.web
    • django app management tools
zack claimed this task.
zack added a subscriber: zack.

looks like this is done now, as you're deep in the implementation already! closing