Page MenuHomeSoftware Heritage
Feed Advanced Search

Nov 19 2018

zack retitled D673: tests: Move test fixture generation to a dedicated module from tests: Move generation tests fixture in a dedicated module to tests: Move test fixture generation to a dedicated module.
Nov 19 2018, 1:09 PM
zack added a comment to T1330: Ensure documentation building process does not generates any rst syntax warning.

Would removing your 3rd requirement (per-package indexes) simplify things? I don't see that as a goal at all, just what happens by default with the minimalistic sphinx setup we used back in the days. I'd be much happier if we could have a single index / navigation bar, where the doc of all modules appear.

Nov 19 2018, 1:08 PM · Documentation

Nov 16 2018

zack triaged T1352: ingest Guix (SD) packages as Normal priority.
Nov 16 2018, 12:09 PM · Archive coverage
zack renamed T1351: (periodically) ingest GNU package releases from periodically ingest GNU package releases to (periodically) ingest GNU package releases.
Nov 16 2018, 12:08 PM · Archive coverage
zack added a project to T1351: (periodically) ingest GNU package releases: Archive coverage.
Nov 16 2018, 12:08 PM · Archive coverage
zack triaged T1351: (periodically) ingest GNU package releases as Normal priority.
Nov 16 2018, 12:08 PM · Archive coverage

Nov 15 2018

zack triaged T1350: vault: pre-cook "interesting" bundles as Normal priority.
Nov 15 2018, 4:12 PM · Vault

Nov 14 2018

zack added a comment to T1346: Define persistent URL to be used as archive url property in Wikidata.

Thanks for filing this @moranegg !

Nov 14 2018, 6:15 PM · Web app, Software Stories
zack added a comment to T1339: Handle malformed author and committer dates.

The simplest solution would be to check if the computed timezone offset lies in the adequate bounds [UTC−14:00, UTC+14:00] and set it to 0 if not.

Nov 14 2018, 12:03 PM · Storage manager, Git loader
zack raised the priority of T1338: Change BBUs on orsay from Normal to High.
Nov 14 2018, 12:00 PM · System administration
zack committed rDWAPPS2bed324bed30: simplify user-message for revision ordering choice (authored by zack).
simplify user-message for revision ordering choice
Nov 14 2018, 11:23 AM
zack closed D653: simplify user-message for revision ordering choice.
Nov 14 2018, 11:23 AM
zack created D653: simplify user-message for revision ordering choice.
Nov 14 2018, 11:10 AM
zack committed rDWAPPS52afd71c010a: spelling fixes in comments (authored by zack).
spelling fixes in comments
Nov 14 2018, 11:09 AM
zack added a comment to T1323: Check battery status on storage adapters.

@ftigeot thanks for checking!
Please file tasks for changing the failing batteries (priority high) and automate the monitoring of failing batteries so that in the future we will be automatically notified of future similar failures (priority normal).

Nov 14 2018, 8:13 AM · System administration
zack added a project to T1332: Set up a planning/kanban-like organization for the swh project : Unknown Object (Project).
Nov 14 2018, 8:11 AM · Unknown Object (Project)

Nov 13 2018

zack added a project to T1324: Deploy metadata indexers in production: Indexer.
Nov 13 2018, 2:34 PM · Indexer
zack changed the visibility for T358: doc: high-level architecture diagram.
Nov 13 2018, 2:33 PM · Documentation
zack reassigned T358: doc: high-level architecture diagram from zack to douardda.
Nov 13 2018, 2:33 PM · Documentation
zack renamed T358: doc: high-level architecture diagram from prepare high-level architecture diagram to doc: high-level architecture diagram.
Nov 13 2018, 2:33 PM · Documentation
zack triaged T1282: Revisit backups as Normal priority.
Nov 13 2018, 10:56 AM · System administration
zack renamed T1302: test pease ignore - Add a new feature from Add a new feature to test pease ignore - Add a new feature.
Nov 13 2018, 10:56 AM · v0.0.2
zack triaged T1317: Improve the CI job dedicated to Diff to handle stacked Diff properly as Low priority.
Nov 13 2018, 10:55 AM · Continuous Integration

Nov 12 2018

zack added a comment to T1315: Web API: lift the restriction that /raw only allows to download text-ish contents.

@anlambert: let's do this. Green light, go ahead!

Nov 12 2018, 5:03 PM · Web app
zack triaged T1315: Web API: lift the restriction that /raw only allows to download text-ish contents as Low priority.
Nov 12 2018, 2:37 PM · Web app
zack triaged T1314: Web API: /raw should not return 404 for available but non text-like contents as Normal priority.
Nov 12 2018, 2:37 PM · Web app

Nov 9 2018

zack added a comment to T1219: add tests to git loader.

Resolved by D622.

Nov 9 2018, 1:49 PM · Sprint 2018 12, Git loader

Nov 6 2018

zack triaged T1309: Web UI: fails rendering archived repository with 500 error "directory" as High priority.
Nov 6 2018, 9:50 PM · Web app

Nov 5 2018

zack added a comment to T1304: Web UI: fail to render some repositories with 500 error "'NoneType' object is not subscriptable".

thanks! can you ping me when this fix (T1304) is deployed?

Nov 5 2018, 5:54 PM · Web app
zack added a comment to T1303: Web UI: empty repositories shown as 404 errors with "snapshot not found" message.

thanks! can you ping me when this fix (T1303) is deployed?

Nov 5 2018, 5:53 PM · Archive content, Web app
zack accepted D628: msgpack.unpackb: Use raw=False when possible..

Oh, silly me, I commented without checking that first.
Fine by me then :) Thanks!

Nov 5 2018, 5:09 PM
zack requested changes to D628: msgpack.unpackb: Use raw=False when possible..

I've tried this before too, but we can't do that yet, because the version of msgpack we use in Debian stable doesn't yet have raw=False.

Nov 5 2018, 5:05 PM
zack committed rMSLD2ce01c037a7b: check-in slides for Google NYC talk (authored by zack).
check-in slides for Google NYC talk
Nov 5 2018, 3:15 PM
zack committed rMSLDa8c16f143837: status: add link to the archive and update object storage stats (authored by zack).
status: add link to the archive and update object storage stats
Nov 5 2018, 3:15 PM
zack committed rMSLDa2d5c5f1ac7c: bibliography module: add ipres 2018 and CACM entries (authored by zack).
bibliography module: add ipres 2018 and CACM entries
Nov 5 2018, 3:15 PM
zack committed rMSLDadae56c49a1d: add new R&D challenges module (authored by zack).
add new R&D challenges module
Nov 5 2018, 3:15 PM
zack updated subscribers of T1304: Web UI: fail to render some repositories with 500 error "'NoneType' object is not subscriptable".

This seems due to releases pointing to missing revisions in the archive, see for instance http://localhost:5004/api/1/release/21d03ad29cfa315d8b7e5d7577964fbc21797f74/

Nov 5 2018, 12:41 PM · Web app
zack added a comment to T1249: create a jenkins job to build and publish the sphinx documentation.

I suggest to call it good as is, and set a proper solution up if and only if it really obviously cause problems...

Nov 5 2018, 12:37 PM · Continuous Integration, Documentation

Nov 2 2018

zack renamed T1304: Web UI: fail to render some repositories with 500 error "'NoneType' object is not subscriptable" from Web UI: fail to render some repositories with error "'NoneType' object is not subscriptable" to Web UI: fail to render some repositories with 500 error "'NoneType' object is not subscriptable".
Nov 2 2018, 9:12 PM · Web app
zack triaged T1304: Web UI: fail to render some repositories with 500 error "'NoneType' object is not subscriptable" as High priority.
Nov 2 2018, 9:12 PM · Web app
zack triaged T1303: Web UI: empty repositories shown as 404 errors with "snapshot not found" message as High priority.
Nov 2 2018, 9:08 PM · Archive content, Web app

Oct 31 2018

zack added a comment to T1301: Web UI: origins returning 505: 'NoneType' object has no attribute 'split'.

(I also note that the search results page treat both cases similarly: showing an X for both kind of origins. Good!)

Oct 31 2018, 11:25 AM · Web app
zack added a comment to T1301: Web UI: origins returning 505: 'NoneType' object has no attribute 'split'.

To elaborate, the user-visible result should be the same of origins that haven't been visited yet, e.g.:

Oct 31 2018, 11:25 AM · Web app
zack updated the task description for T1301: Web UI: origins returning 505: 'NoneType' object has no attribute 'split'.
Oct 31 2018, 11:09 AM · Web app
zack changed the visibility for F3343181: software_heritage_http_code_500.txt.
Oct 31 2018, 11:08 AM
zack triaged T1301: Web UI: origins returning 505: 'NoneType' object has no attribute 'split' as High priority.
Oct 31 2018, 11:08 AM · Web app

Oct 30 2018

zack accepted D626: docs/getting-started: Update pip environment build dependency.
Oct 30 2018, 8:08 PM

Oct 29 2018

zack added inline comments to D617: Always output valid JSON-LD..
Oct 29 2018, 5:34 PM

Oct 26 2018

zack renamed T1295: move debian packaging stuff to a dedicated debian git branch from move debian packaging stuff in a dedicated debian git branch to move debian packaging stuff to a dedicated debian git branch.
Oct 26 2018, 9:59 AM · Development environment
zack committed rDSTOfba230dc9cfb: DB schema: manual fix for some arrow source/dest (authored by zack).
DB schema: manual fix for some arrow source/dest
Oct 26 2018, 9:57 AM
zack committed rDSTOa579a1a636c0: DB schema: add light foreign key arrows from snapshot to * (authored by zack).
DB schema: add light foreign key arrows from snapshot to *
Oct 26 2018, 9:52 AM
zack committed rDSTOb032d2ac66d7: DB schema: remove left-over occurrence stuff from snapshot cluster (authored by zack).
DB schema: remove left-over occurrence stuff from snapshot cluster
Oct 26 2018, 9:52 AM

Oct 25 2018

zack added a comment to D592: tests: replace nose with pytest.
In D592#11583, @olasd wrote:

I don't quite understand what the "fs" marks are supposed to convey, but they're carried over from nose without change, so *shrug*.

Oct 25 2018, 7:32 AM

Oct 24 2018

zack committed rDMOD33027d77f584: PID doc: add reference to iPRES 2018 paper (authored by zack).
PID doc: add reference to iPRES 2018 paper
Oct 24 2018, 3:48 PM
zack committed rDSTO7628710a3792: SQL schema: update to reflect recent changes (authored by zack).
SQL schema: update to reflect recent changes
Oct 24 2018, 2:40 PM
zack committed rDSTOa6742b73c3a0: DB schema doc: error out if SQL schema cannot be found (authored by zack).
DB schema doc: error out if SQL schema cannot be found
Oct 24 2018, 2:40 PM
zack closed T1283: doc: update DB diagram after occurrence-* removal as Resolved by committing rDSTO7628710a3792: SQL schema: update to reflect recent changes.
Oct 24 2018, 2:40 PM · Documentation, Storage manager
zack committed rDSTO582d72d025dc: docs Makefile: only echo SQL queries on error (authored by zack).
docs Makefile: only echo SQL queries on error
Oct 24 2018, 2:40 PM
zack committed rDSTO5fe922185289: docs Makefile: forward "distclean" to sql dir (authored by zack).
docs Makefile: forward "distclean" to sql dir
Oct 24 2018, 2:40 PM
zack committed rDSTOb91482af6116: gitignore and remove generate archive copies diagram (authored by zack).
gitignore and remove generate archive copies diagram
Oct 24 2018, 2:11 PM
zack closed T1269: swh-storage: `make distclean` in docs/ deletes committed files as Resolved by committing rDSTOb91482af6116: gitignore and remove generate archive copies diagram.
Oct 24 2018, 2:11 PM · Documentation, Storage manager
zack triaged T1287: storage: doc generation fails if postgres dev DB exists as Normal priority.
Oct 24 2018, 2:11 PM · Documentation, Storage manager
zack raised the priority of T1285: Ensure /var/lib/docker dir are not backupped from High to Unbreak Now!.
Oct 24 2018, 10:02 AM · System administration
zack changed the edit policy for T1285: Ensure /var/lib/docker dir are not backupped .
Oct 24 2018, 10:02 AM · System administration

Oct 23 2018

zack accepted D557: Add the origin intrinsic metadata storage database..
Oct 23 2018, 3:34 PM
zack accepted D574: docs: fix a rst formatting problem.
Oct 23 2018, 3:26 PM
zack committed rDTPL1c547d864937: tox.ini: use pytest instead of nose (authored by zack).
tox.ini: use pytest instead of nose
Oct 23 2018, 3:22 PM
zack closed D562: tox.ini: use pytest instead of nose.
Oct 23 2018, 3:22 PM
zack added a comment to D568: getting-started: replace usage of swh-storage/sql/bin/db-init by swh-db-init.

To try this i've re-installed locally swh-core and got the swh-db-init entry point. However, it fails as follows:

$ swh-db-init 
Traceback (most recent call last):
  File "/home/zack/.local/bin/swh-db-init", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3105, in <module>
    @_call_aside
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3089, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3118, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 578, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 895, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 781, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'msgpack-python' distribution was not found and is required by swh.core

even if I have msgpack installed (it loads fine in ipython3 and i can successfully run all swh-core tests).

Oct 23 2018, 1:19 PM
zack abandoned D567: use raw=False for msgpack.unpackb instead of deprecated encoding.

abandoning, as raw=False isn't yet part of debian stable version of msgpack

Oct 23 2018, 12:27 PM
Herald added a reviewer for D567: use raw=False for msgpack.unpackb instead of deprecated encoding: Reviewers.
Oct 23 2018, 12:23 PM
zack committed rDCOREb8f23b5b1232: logger docstring: fix invalid escape sequence (authored by zack).
logger docstring: fix invalid escape sequence
Oct 23 2018, 12:19 PM
zack updated the diff for D562: tox.ini: use pytest instead of nose.

use .[testing] in tox.ini

Oct 23 2018, 10:36 AM
zack accepted D563: tox: install test deps from setup.py file's [testing] extras.
Oct 23 2018, 9:35 AM
zack created D562: tox.ini: use pytest instead of nose.
Oct 23 2018, 9:23 AM

Oct 22 2018

zack triaged T1283: doc: update DB diagram after occurrence-* removal as High priority.
Oct 22 2018, 10:00 PM · Documentation, Storage manager
zack added a comment to T1262: wiki: Update suggestion box if `all Debian derivatives` can be noted as ingested.

the internship topic on this is now available here: https://wiki.softwareheritage.org/wiki/Ingest_all_Debian_derivatives_(internship)

Oct 22 2018, 7:59 PM · Archive coverage
zack added inline comments to D557: Add the origin intrinsic metadata storage database..
Oct 22 2018, 12:45 PM
zack requested changes to D557: Add the origin intrinsic metadata storage database..
Oct 22 2018, 11:13 AM

Oct 19 2018

zack committed rDMODf58f4c2e7ff4: doc: document PID resolution possibilities other than Web UI / (authored by zack).
doc: document PID resolution possibilities other than Web UI /
Oct 19 2018, 4:01 PM
zack closed T1223: identifiers.org URL resolution should support swh:id contextual parameters as Resolved by committing rDMODf58f4c2e7ff4: doc: document PID resolution possibilities other than Web UI /.
Oct 19 2018, 4:01 PM · Metadata workflow
zack closed D536: doc: document PID resolution possibilities other than Web UI /.
Oct 19 2018, 4:01 PM
zack closed T1015: document registration of swh: identifiers on identifiers.org as Resolved by committing rDMODf58f4c2e7ff4: doc: document PID resolution possibilities other than Web UI /.
Oct 19 2018, 4:01 PM · Documentation
zack updated the diff for D536: doc: document PID resolution possibilities other than Web UI /.

add N2T resolution

Oct 19 2018, 2:50 PM
zack added a comment to D537: Origin metadata pipeline..

The last comment was submitted rapidly, sorry..

Oct 19 2018, 12:54 PM

Oct 18 2018

zack edited projects for T439: Indexers: compute (and maintain up-to-date) the filetype of all blobs, added: Indexer; removed Developers.
Oct 18 2018, 9:17 PM · Indexer

Oct 16 2018

zack added a comment to T1262: wiki: Update suggestion box if `all Debian derivatives` can be noted as ingested.
In T1262#23681, @olasd wrote:

Automating the addition of distributions from the Debian derivatives census to Software Heritage would probably be a good topic for an internship, e.g. a Google Summer of Code/Outreachy project.

Oct 16 2018, 5:41 PM · Archive coverage
zack planned changes to D536: doc: document PID resolution possibilities other than Web UI /.

I'll add n2t.net and resubmit this. Thanks @moranegg

Oct 16 2018, 5:35 PM
zack added a comment to T830: Remove tables occurrence and occurrence_history.

\o/

Oct 16 2018, 5:34 PM · Storage manager, Archive content
zack accepted D534: Drop confusing functions revision_get_by/revision_log_by.
Oct 16 2018, 11:37 AM
zack accepted D535: Drop table occurrence_history.
Oct 16 2018, 11:36 AM
zack committed rMSLDa62c995216f5: check in slides for Speck & Tech talk (authored by zack).
check in slides for Speck & Tech talk
Oct 16 2018, 11:28 AM
zack committed rMSLD9e5ace1efb61: slides makefile: clean cruft related to code snippets (authored by zack).
slides makefile: clean cruft related to code snippets
Oct 16 2018, 11:28 AM

Oct 15 2018

zack edited reviewers for D537: Origin metadata pipeline., added: Reviewers; removed: test-public.
Oct 15 2018, 8:34 PM
zack added a reviewer for D536: doc: document PID resolution possibilities other than Web UI /: anlambert.
Oct 15 2018, 5:34 PM
zack added a revision to T1223: identifiers.org URL resolution should support swh:id contextual parameters: D536: doc: document PID resolution possibilities other than Web UI /.
Oct 15 2018, 5:34 PM · Metadata workflow
zack added a revision to T1015: document registration of swh: identifiers on identifiers.org: D536: doc: document PID resolution possibilities other than Web UI /.
Oct 15 2018, 5:34 PM · Documentation
zack created D536: doc: document PID resolution possibilities other than Web UI /.
Oct 15 2018, 5:34 PM
zack committed rMSLD79a8dd34054a: refactor pypi loading slides to a separate module (authored by zack).
refactor pypi loading slides to a separate module
Oct 15 2018, 4:35 PM
zack raised the priority of T1261: migrate tests from nose to pytest from Wishlist to Normal.

(i've retitled this according to consensus)

Oct 15 2018, 3:14 PM · Development environment