Page MenuHomeSoftware Heritage
Feed Advanced Search

Mar 10 2016

zack removed a project from T121: Populate fetch_history: Staff.
Mar 10 2016, 5:49 PM · Storage manager
zack removed a project from T111: Performance analysis of read queries: Staff.
Mar 10 2016, 5:49 PM · Storage manager
zack removed a project from T147: Migrate code to use latest schema improvments from T3: Staff.
Mar 10 2016, 5:49 PM · Directory loader, Tarball loader
zack removed a project from T163: Add entry points to create / request entities in swh.storage: Staff.
Mar 10 2016, 5:49 PM · Storage manager
zack removed a project from T149: Populate fetch_history table: Staff.
Mar 10 2016, 5:49 PM · Directory loader
zack removed a project from T148: Populate fetch_history: Staff.
Mar 10 2016, 5:49 PM · Tarball loader
zack removed a project from T170: Random order in tarball messages computation: Staff.
Mar 10 2016, 5:49 PM · Directory loader, Tarball loader
zack removed a project from T174: Analyse and fix last errors: Staff.
Mar 10 2016, 5:49 PM · Directory loader, Tarball loader
zack removed a project from T171: Fix content loading only when needed: Staff.
Mar 10 2016, 5:49 PM · Directory loader
zack removed a project from T183: gnu/old-gnu injection to production: Staff.
Mar 10 2016, 5:49 PM · Directory loader, Tarball loader
zack removed a project from T195: /revision/<SHA1_GIT>: show commit information: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T176: Fix `Permission denied: write to /usr/local/lib/gcl-3...` : Staff.
Mar 10 2016, 5:49 PM · Directory loader, Tarball loader
zack removed a project from T178: Replay loading mirror gnu and old-gnu from scratch: Staff.
Mar 10 2016, 5:49 PM · Directory loader, Tarball loader
zack removed a project from T197: /api/1/directory/<SHA1_GIT>/path/to/file-or-dir: ditto, but for file or directory pointed by path: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T198: /content/[<HASH_ALGO>:]<HASH>: show content information: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T196: /directory/<SHA1_GIT>: show directory information (including ls): Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T200: /person/<PERSON_ID>: show person information: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T199: /release/<SHA1_GIT>: show release information: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T202: /api/1/project/<PROJECT_ID>: show project information: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T201: /origin/<ORIGIN_ID>: show origin information: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T204: /api/1/directory/<SHA1_GIT>/path/to/file-or-dir?timestamp=<TIMESTAMP>&origin=<ORIGIN>&branch<BRANCH>: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T203: /api/1/organization/<ORGANIZATION_ID>: show organization information: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T207: /api/1/revision/<SHA1_GIT>/log/: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T205: /api/1/revision/<SHA1_GIT>?timestamp=<TIMESTAMP>&origin=<ORIGIN>: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T209: Retrieve content with other checksums than sha1: Staff.
Mar 10 2016, 5:49 PM · Storage manager
zack removed a project from T211: Load files and directories from a set of Debian packages: Staff.
Mar 10 2016, 5:49 PM · Origin-Debian, Debian loader
zack removed a project from T213: Create synthetic releases for Debian packages: Staff.
Mar 10 2016, 5:49 PM · Origin-Debian, Debian loader
zack removed a project from T212: Create synthetic revisions for Debian packages: Staff.
Mar 10 2016, 5:49 PM · Origin-Debian, Debian loader
zack removed a project from T214: Create occurrences for Debian packages: Staff.
Mar 10 2016, 5:49 PM · Origin-Debian, Debian loader
zack removed a project from T215: Internal Error 500 on /api/1/revision/<sha1>: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T216: /api/1/revision/<sha1> - Single revision with no parent: Staff.
Mar 10 2016, 5:49 PM · Storage manager, Web app
zack removed a project from T225: Add view /browse/content/<algo_hash:hash>/raw : Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T224: Add coverage on views: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T226: Add view /browse/directory/<sha1_git>: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T234: api - peer review adaptations: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T249: Open /api/1/revision/<SHA1_GIT>/directory/[<PATH>] : Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T245: Autodoc browsable api endpoint: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T259: Open /api/1/revision/origin/<ORIG_ID>: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T256: Improve browsable api rendering view: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T257: Open /revision/<SHA1_GIT_ROOT>/history/<SHA1_GIT>/directory/[<PATH>]: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T269: Deal nicely with swh-storage's backend down time in clients: Staff.
Mar 10 2016, 5:49 PM · Web app, Storage manager
zack removed a project from T264: directory_get: Retrieve information on a directory per hash.: Staff.
Mar 10 2016, 5:49 PM · Web app, Storage manager
zack removed a project from T277: Open /browse/revision/<SHA1_GIT_ROOT>/history/<SHA1_GIT>/directory/[<PATH>] view: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T272: Fix double-encoded release.name entries: Staff.
Mar 10 2016, 5:49 PM · Data Model, Storage manager
zack removed a project from T276: Open /browse/revision/<sha1_git_root>/directory/<path>/ view: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T278: Open /browse/revision/origin/<ORIG_ID>[/branch/<BRANCH>][/ts/<TIMESTAMP>]/ view: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T279: Open /browse/revision/origin/<ORIG_ID>[/branch/<BRANCH>][/ts/<TIMESTAMP>]/history/<SHA1>/ view: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T281: Open /revision/origin/<ORIG_ID>[/branch/<BRANCH>][/ts/<TIMESTAMP>]/directory/[<PATH>] view: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T280: Open /browse/revision/origin/<ORIG_ID>[/branch/<BRANCH>][/ts/<TIMESTAMP>]/history/<SHA1>/directory/[<PATH>] view: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T285: Open /browse/revision/<sha1_git>/log/: Staff.
Mar 10 2016, 5:49 PM · Web app
zack removed a project from T308: retrieve content from s3 and store it in SWH storage: Staff.
Mar 10 2016, 5:49 PM · Storage manager, Antelink loader
zack removed a project from T302: swh-loader-tar origin validities are the current time instead of the mirroring time: Staff.
Mar 10 2016, 5:49 PM · Tarball loader
zack removed a project from T309: Delete duplicated antelink/antepedia content from s3: Staff.
Mar 10 2016, 5:49 PM · Antelink loader
zack removed a project from T316: List and compute hashes of actual sesi files: Staff.
Mar 10 2016, 5:49 PM · Antelink loader
zack removed a project from T317: Inject sesi files hashes in antelink db: Staff.
Mar 10 2016, 5:49 PM · Antelink loader
zack removed a project from T319: S3 content files downloader and injection in swh: Staff.
Mar 10 2016, 5:49 PM · Antelink loader
zack removed a project from T321: Download antepedia's s3 contents not in sesi nor in swh: Staff.
Mar 10 2016, 5:49 PM · Antelink loader
zack removed a project from T320: sesi content downloader and injection in swh worker: Staff.
Mar 10 2016, 5:49 PM · Antelink loader
zack removed a project from T322: Inject antepedia contents (backuped in sesi and not in swh) into swh: Staff.
Mar 10 2016, 5:49 PM · Antelink loader
zack removed a project from T326: swh-scheduler: puppetize event listener: Staff.
Mar 10 2016, 5:49 PM · Puppet recipes, Language-Puppet, Scheduling utilities
zack removed a project from T327: swh-worker: puppetize daemon: Staff.
Mar 10 2016, 5:49 PM · Puppet recipes, Language-Puppet, Scheduling utilities
zack removed a project from T343: retrieve gitorious repositories from the gitorious valhalla: Staff.
Mar 10 2016, 5:49 PM · Origin-Gitorious, Format-Git
ardumont created T348: Improve security on web form.
Mar 10 2016, 4:01 PM · Web app

Mar 9 2016

olasd closed T164: Port GitHub lister to swh.storage instead of sqlalchemy as Resolved.

As of rDLSGH533f6fa1a3dc, the GitHub lister uses swh.storage instead of SQLAlchemy.

Mar 9 2016, 7:05 PM · GitHub lister
zack changed the status of T343: retrieve gitorious repositories from the gitorious valhalla, a subtask of T312: Gitorious import: ingest repositories, from Open to Work in Progress.
Mar 9 2016, 10:27 AM · Archive coverage, Restricted Project, Origin-Gitorious, Format-Git
zack changed the status of T343: retrieve gitorious repositories from the gitorious valhalla from Open to Work in Progress.

The transfer is now in progress on uffizi:/srv/softwareheritage/mirros/gitorious.org/, within a screen session of my user with title "gitorious-transfer".

Mar 9 2016, 10:27 AM · Origin-Gitorious, Format-Git
zack claimed T343: retrieve gitorious repositories from the gitorious valhalla.
Mar 9 2016, 9:53 AM · Origin-Gitorious, Format-Git
ardumont added a comment to T339: add missing "length" property to revision.metadata JSON for all imported tar/zip.

ardumont closed this task as "Wontfix".

Mar 9 2016, 12:27 AM · Tarball loader
ardumont changed the status of T339: add missing "length" property to revision.metadata JSON for all imported tar/zip from Wontfix to Resolved.
Mar 9 2016, 12:26 AM · Tarball loader
ardumont closed T345: Update db swh.revision.metadata data with tarball length, a subtask of T339: add missing "length" property to revision.metadata JSON for all imported tar/zip, as Resolved.
Mar 9 2016, 12:26 AM · Tarball loader
ardumont closed T345: Update db swh.revision.metadata data with tarball length as Resolved.
Mar 9 2016, 12:26 AM · Tarball loader

Mar 8 2016

ardumont renamed T322: Inject antepedia contents (backuped in sesi and not in swh) into swh from Download antepedia contents (backuped in sesi) not in swh and injects in swh to Inject antepedia contents (backuped in sesi and not in swh) into swh.
Mar 8 2016, 7:31 PM · Antelink loader
ardumont closed T339: add missing "length" property to revision.metadata JSON for all imported tar/zip as Wontfix.
Mar 8 2016, 7:29 PM · Tarball loader
ardumont added a comment to T321: Download antepedia's s3 contents not in sesi nor in swh.

As of yesterday, 10M were done.

Mar 8 2016, 7:28 PM · Antelink loader
olasd renamed T338: manifest browser / renderer from maniphest browser / renderer to manifest browser / renderer.
Mar 8 2016, 5:44 PM · Web app
ardumont added a comment to T322: Inject antepedia contents (backuped in sesi and not in swh) into swh.

Multiple problems:

  • slow
  • ssh connection problems when too many concurrent jobs (either with ssh multiplex or with simple scp).
Mar 8 2016, 12:06 PM · Antelink loader

Mar 7 2016

ardumont added a comment to T345: Update db swh.revision.metadata data with tarball length.

Query sample to update jsonb field:

Mar 7 2016, 2:46 PM · Tarball loader
ardumont closed T344: Add missing length tarball property in revision.metadata json in loader as Resolved by committing rDLDTAR5e770d8eed41: Add length property in revision.metadata json entry.
Mar 7 2016, 12:12 PM · Tarball loader
ardumont closed T344: Add missing length tarball property in revision.metadata json in loader, a subtask of T339: add missing "length" property to revision.metadata JSON for all imported tar/zip, as Resolved.
Mar 7 2016, 12:12 PM · Tarball loader
ardumont renamed T344: Add missing length tarball property in revision.metadata json in loader from Add missing length tarball property in revision.metadata json to Add missing length tarball property in revision.metadata json in loader.
Mar 7 2016, 12:05 PM · Tarball loader
ardumont created T345: Update db swh.revision.metadata data with tarball length.
Mar 7 2016, 12:00 PM · Tarball loader
ardumont created T344: Add missing length tarball property in revision.metadata json in loader.
Mar 7 2016, 11:59 AM · Tarball loader
ardumont added a comment to T322: Inject antepedia contents (backuped in sesi and not in swh) into swh.

The files mentioned are only the ones with size <= 100Mib for now.

Mar 7 2016, 11:03 AM · Antelink loader
ardumont added a comment to T322: Inject antepedia contents (backuped in sesi and not in swh) into swh.

worker01 is producing messages in the swh_antelink_sesi_download queue.
Each message represents either roughly 100Mib of compressed files or 1000 compressed files to scp.

Mar 7 2016, 10:59 AM · Antelink loader
ardumont renamed T322: Inject antepedia contents (backuped in sesi and not in swh) into swh from Download antepedia's sesi contents not in swh to Download antepedia contents (backuped in sesi) not in swh and injects in swh.
Mar 7 2016, 10:54 AM · Antelink loader

Mar 5 2016

rdicosmo added a comment to T337: specify a manifest format for documenting archived software.

Important: we want to provide these informations *for all the content stored in SWH*

Mar 5 2016, 12:31 PM · General
rdicosmo added a comment to T337: specify a manifest format for documenting archived software.

Depending on the form of the archived code, and the needs of the user that reference it, a variety of different references may be needed
Here is the original list from the team meeting, that completes the one mentioned in the ticket.

Mar 5 2016, 12:31 PM · General
zack lowered the priority of T312: Gitorious import: ingest repositories from High to Normal.
Mar 5 2016, 10:41 AM · Archive coverage, Restricted Project, Origin-Gitorious, Format-Git
zack created T343: retrieve gitorious repositories from the gitorious valhalla.
Mar 5 2016, 10:41 AM · Origin-Gitorious, Format-Git
zack lowered the priority of T120: regular debian archive producer from High to Normal.
Mar 5 2016, 10:39 AM · Debian loader, Origin-Debian
zack raised the priority of T312: Gitorious import: ingest repositories from Normal to High.
Mar 5 2016, 10:38 AM · Archive coverage, Restricted Project, Origin-Gitorious, Format-Git
zack added a comment to T312: Gitorious import: ingest repositories.

We are now all set to start (after having automated it properly…) the transfer of Gitorious stuff to SWH.

Mar 5 2016, 10:36 AM · Archive coverage, Restricted Project, Origin-Gitorious, Format-Git

Mar 4 2016

zack created T341: store ingested tarball artifacts (in a separate storage area).
Mar 4 2016, 6:36 PM · General
zack closed T62: DB schema: add directory→tarball provenance information as Resolved.

This has been fixed a while ago, for both debian and tarball ingestion.

Mar 4 2016, 6:26 PM · Storage manager
zack created T340: add missing "archive_type" property to revision.metadata JSON for all imported dsc.
Mar 4 2016, 6:21 PM · Git loader
zack created T339: add missing "length" property to revision.metadata JSON for all imported tar/zip.
Mar 4 2016, 6:17 PM · Tarball loader
zack renamed T286: view /browse/origin/ORIGIN_ID/ - calendar-like UI from View /browse/origin/ - Add visit times to view /browse/origin/ORIGIN_ID/ - calendar-like UI.
Mar 4 2016, 1:12 PM · Web app
zack added a subtask for T338: manifest browser / renderer: T337: specify a manifest format for documenting archived software.
Mar 4 2016, 1:08 PM · Web app
zack added a parent task for T337: specify a manifest format for documenting archived software: T338: manifest browser / renderer.
Mar 4 2016, 1:08 PM · General
zack created T338: manifest browser / renderer.
Mar 4 2016, 1:07 PM · Web app