Page MenuHomeSoftware Heritage
Feed All Stories

Yesterday

vlorentz committed R186:161bcd9639e1: Finish the long benchmarks. (authored by vlorentz).
Finish the long benchmarks.
Wed, Apr 24, 5:37 PM
vlorentz committed R186:b5196b9d8298: Add long benchmarks. (authored by vlorentz).
Add long benchmarks.
Wed, Apr 24, 5:23 PM
vlorentz closed T1691: metadata indexer: investigate metadata entries with empty mappings as Resolved.

We should investigate why they are there.

Wed, Apr 24, 5:22 PM · Archive content, Indexer
vlorentz closed T1691: metadata indexer: investigate metadata entries with empty mappings, a subtask of T1549: Clean up entries in {origin_intrinsic,revision}_metadata with no metadata, as Resolved.
Wed, Apr 24, 5:22 PM · Archive content, Indexer
zack renamed T1691: metadata indexer: investigate metadata entries with empty mappings from metadata indexer: investigate empty mappings to metadata indexer: investigate metadata entries with empty mappings.
Wed, Apr 24, 5:21 PM · Archive content, Indexer
zack triaged T1691: metadata indexer: investigate metadata entries with empty mappings as Normal priority.
Wed, Apr 24, 5:20 PM · Archive content, Indexer
zack closed T1549: Clean up entries in {origin_intrinsic,revision}_metadata with no metadata as Resolved.

This is now done, aside from a minor issue noted below:

softwareheritage-indexer=# select count(*) from revision_intrinsic_metadata where metadata = '{"@context": "https://doi.org/10.5063/schema/codemeta-2.0"}'::jsonb;
 count 
-------
     0
(1 row)
Wed, Apr 24, 5:18 PM · Archive content, Indexer
swh-public-ci added a comment to D1415: Add support for client side rendering of Jupyter notebooks.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/403/ for more details.

Wed, Apr 24, 5:12 PM
anlambert updated the diff for D1415: Add support for client side rendering of Jupyter notebooks.

Rebase

Wed, Apr 24, 5:08 PM
zack accepted D1432: Drop constraint on references of revision_metadata from origin_metadata..
Wed, Apr 24, 5:04 PM
anlambert committed rDWAPPS65501f32f226: assets/webapp: Add missing filterXSS function export (authored by anlambert).
assets/webapp: Add missing filterXSS function export
Wed, Apr 24, 5:04 PM
swh-public-ci added a comment to D1432: Drop constraint on references of revision_metadata from origin_metadata..

Build is green
See https://jenkins.softwareheritage.org/job/DCIDX/job/tox/525/ for more details.

Wed, Apr 24, 5:02 PM
vlorentz created D1432: Drop constraint on references of revision_metadata from origin_metadata..
Wed, Apr 24, 4:59 PM
anlambert committed rCDFD875c50aa3920: conf/web.yml: Disable swh web api rate limiting (authored by anlambert).
conf/web.yml: Disable swh web api rate limiting
Wed, Apr 24, 4:43 PM
swh-public-ci added a comment to D1335: Don't store empty metadata dicts..

Build has FAILED

Wed, Apr 24, 3:31 PM
Harbormaster failed remote builds in B5535: Diff 4656 for D1335: Don't store empty metadata dicts.!
Wed, Apr 24, 3:31 PM
vlorentz committed rDCIDX22175dbc20be: Don't store empty metadata dicts. (authored by vlorentz).
Don't store empty metadata dicts.
Wed, Apr 24, 3:30 PM
vlorentz closed D1335: Don't store empty metadata dicts..
Wed, Apr 24, 3:30 PM
vlorentz updated the diff for D1335: Don't store empty metadata dicts..
  • rebase
Wed, Apr 24, 3:30 PM
zack added a comment to T1683: Decide where should be gathered specs.

I don't think it's a good idea to move specs from the specific components they specify to a different repository (swh-docs), because that will just increase the chances (which are already quite high) that they will get out of sync.

Wed, Apr 24, 3:14 PM · Development documentation
moranegg added a comment to T1683: Decide where should be gathered specs.
In T1683#31104, @zack wrote:

That was my point about "swh-docs as a fallback".

Wed, Apr 24, 3:05 PM · Development documentation
zack added a comment to D1430: Correct typo "scrapping" -> "scraping".

I knew about the manual landing from us but I am not a big fan of all this noise added to the commit message by arc.
Based on my understanding, if the original commit author is able to push after we accept a diff, this noise will not
be present. Anyway, that's not a blocker to integrate changes so I will also proceed like this in the future.

Wed, Apr 24, 2:26 PM
anlambert added a comment to D1430: Correct typo "scrapping" -> "scraping".

That's only because I've (erroneously, this time) used arc land instead of arc land --squash.

Wed, Apr 24, 2:24 PM
zack added a project to T1683: Decide where should be gathered specs: Development documentation.
Wed, Apr 24, 2:24 PM · Development documentation
zack triaged T1689: enable landing patches via the web UI for all repos as High priority.
Wed, Apr 24, 2:24 PM · Development environment, Phabricator
zack added a comment to D1430: Correct typo "scrapping" -> "scraping".

Third: everyone else: once a change is accepted, you can land it yourself by doing first arc diff D1430 and then arc land

I think you mean arc patch D1430 then arc land

Wed, Apr 24, 2:14 PM
anlambert added a comment to D1430: Correct typo "scrapping" -> "scraping".

Third: everyone else: once a change is accepted, you can land it yourself by doing first arc diff D1430 and then arc land

Wed, Apr 24, 2:11 PM
zack committed rDDOC550b9d6e598f: CONTRIBUTORS: create file adding first external contributor (authored by zack).
CONTRIBUTORS: create file adding first external contributor
Wed, Apr 24, 2:08 PM
zack added a comment to D1430: Correct typo "scrapping" -> "scraping".

@mcv21: first, apologies for the conflicting communication. We don't have a lot of external contributors (yet), so our processes are not very oiled on that front (yet).

Wed, Apr 24, 2:05 PM
zack committed rDDOCd863304089b6: Correct typo "scrapping" -> "scraping" (authored by mcv21).
Correct typo "scrapping" -> "scraping"
Wed, Apr 24, 2:03 PM
zack committed rDDOCd146bfdc4ca2: Correct typo "scrapping" -> "scraping" (authored by mcv21).
Correct typo "scrapping" -> "scraping"
Wed, Apr 24, 2:03 PM
zack closed D1430: Correct typo "scrapping" -> "scraping".
Wed, Apr 24, 2:02 PM
zack added a comment to T1683: Decide where should be gathered specs.

The problem with having it in the docs, is that specs of higher level (like metadata workflow) must be in a specific repository, or can I have a specs folder in swh-environment?

Wed, Apr 24, 1:55 PM · Development documentation
anlambert added a comment to D1430: Correct typo "scrapping" -> "scraping".

@vlorentz, looking at the commits history of the documentation repository, only team members have pushed on it (which are de facto members of the Developers project)

Wed, Apr 24, 1:17 PM
vlorentz added a comment to D1430: Correct typo "scrapping" -> "scraping".

@anlambert non-developers have been able to push accepted diffs to master in the past

Wed, Apr 24, 1:11 PM
anlambert added a comment to D1430: Correct typo "scrapping" -> "scraping".

Ok, so the fact that you are not able to push is that you are not a member of the Developers phabricator project.
As you can see on the repository policies, there is a push restriction on that specific users group.

Wed, Apr 24, 12:32 PM
mcv21 added a comment to D1430: Correct typo "scrapping" -> "scraping".

@vlorentz thus:

mv3@deskpro108655:/upstreams/SoftwareHeritage/swh-docs$ git remote -v
origin  ssh://git@forge.softwareheritage.org/source/swh-docs.git (fetch)
origin  ssh://git@forge.softwareheritage.org/source/swh-docs.git (push)
Wed, Apr 24, 12:22 PM
vlorentz added a comment to D1430: Correct typo "scrapping" -> "scraping".

@anlambert That feature is only enabled on swh-storage, and I think it's broken anyway.

Wed, Apr 24, 12:11 PM
mcv21 added a comment to D1430: Correct typo "scrapping" -> "scraping".

@anlambert no, I don't have any such link - the top left of this page ( https://forge.softwareheritage.org/D1430 ) has "Accepted" and "Public" below the title of this Differential, but nothing else...

Wed, Apr 24, 12:04 PM
anlambert triaged T1688: Refactor content rendering as Normal priority.
Wed, Apr 24, 11:52 AM · Web app
moranegg added a comment to T1683: Decide where should be gathered specs.

@zack Thanks for the answer.
It's not a blocker per se but I prefer to have a long term decision before moving the specs from different spaces, so for me it's a high priority.
It wasn't clear to me that it should be in the docs and that this is the long term decision.

Wed, Apr 24, 11:42 AM · Development documentation
swh-public-ci added a comment to D1431: Revisions walker: Add methods to query produced history state.

Build is green
See https://jenkins.softwareheritage.org/job/DSTO/job/tox/377/ for more details.

Wed, Apr 24, 11:39 AM
Herald added a reviewer for D1431: Revisions walker: Add methods to query produced history state: Reviewers.
Wed, Apr 24, 11:34 AM
anlambert added a comment to D1430: Correct typo "scrapping" -> "scraping".

Hi @mcv21 , did you try to land that diff directly from the web interface by clicking on the "Land revision" link on the top left part of that page ?

Wed, Apr 24, 11:28 AM
anlambert triaged T1687: Add filename as an optional part in persistent identifiers as Normal priority.
Wed, Apr 24, 11:24 AM · Data Model
anlambert added a comment to D1369: Guess extension from detected mime type to add to filename.

Currently, you can pass a filename as a query parameter to the content view of swh-web
(see https://archive.softwareheritage.org/browse/content/sha1_git:d0158ee2e79b461bf25b5b66e6778671c2114263/?path=xmlrpc.php
as an example).

Wed, Apr 24, 11:20 AM
anlambert added a comment to T1685: move main website from www.s.o to s.o.

According to Gandi documentation, if we switch to their LiveDNS feature, there will be no downtime.

Wed, Apr 24, 11:14 AM · System administration, Website
zack added a project to T1685: move main website from www.s.o to s.o: System administration.

@zack, I tried to handle that task but I do not have access to domains configuration from the Gandi interface.

Wed, Apr 24, 11:08 AM · System administration, Website
anlambert added a comment to T1685: move main website from www.s.o to s.o.

@zack, I tried to handle that task but I do not have access to domains configuration from the Gandi interface.

Wed, Apr 24, 10:51 AM · System administration, Website
mcv21 added a comment to D1430: Correct typo "scrapping" -> "scraping".

Hm, no, I did that, but I still get the same permission denied message.

Wed, Apr 24, 10:48 AM
mcv21 updated the diff for D1430: Correct typo "scrapping" -> "scraping".

Tidy commit message.

Wed, Apr 24, 10:46 AM
mcv21 added a comment to D1430: Correct typo "scrapping" -> "scraping".

Oh, I'm going to have to do arc diff --update first, aren't I? Sorry, not used this tooling before, and it's a bit confusing!

Wed, Apr 24, 10:46 AM
mcv21 added a comment to D1430: Correct typo "scrapping" -> "scraping".

I tried (having done git rebase -i to tidy up the commit message, and then git merge), but get refused:

mv3@deskpro108655:/upstreams/SoftwareHeritage/swh-docs$ git push
X11 forwarding request failed on channel 0
Exception: You do not have permission to push to this repository.
fatal: Could not read from remote repository.
Wed, Apr 24, 10:44 AM
zack added a comment to D1430: Correct typo "scrapping" -> "scraping".
In D1430#31457, @mcv21 wrote:

[I think I can't merge this myself]

Wed, Apr 24, 10:42 AM
vlorentz committed R186:8c8c3f808478: Don't create the indexes 6 times... (authored by vlorentz).
Don't create the indexes 6 times...
Wed, Apr 24, 10:40 AM
vlorentz committed R186:9701a5c1b7a8: remove debug print. (authored by vlorentz).
remove debug print.
Wed, Apr 24, 10:40 AM
vlorentz committed R186:a9727ef6886d: New run of pg benchmark. (authored by vlorentz).
New run of pg benchmark.
Wed, Apr 24, 10:40 AM
vlorentz committed R186:6aa0a942a624: move plot data to a text file. (authored by vlorentz).
move plot data to a text file.
Wed, Apr 24, 10:40 AM
mcv21 added a comment to D1430: Correct typo "scrapping" -> "scraping".

[I think I can't merge this myself]

Wed, Apr 24, 10:27 AM
zack committed rMSLD68a1d19e7498: reprod-bad-sota: add back original Collberg picture (authored by zack).
reprod-bad-sota: add back original Collberg picture
Wed, Apr 24, 10:25 AM
zack committed rMSLD5496f371c595: merkle tree slide: add vspace to fit into 16:9 (authored by zack).
merkle tree slide: add vspace to fit into 16:9
Wed, Apr 24, 10:25 AM

Tue, Apr 23

Harbormaster failed remote builds in B5532: Diff 4652 for D1365: Implemented a lister for phabricator instance!
Tue, Apr 23, 7:51 PM
swh-public-ci added a comment to D1365: Implemented a lister for phabricator instance.

Build has FAILED

Tue, Apr 23, 7:51 PM
nahimilega updated the diff for D1365: Implemented a lister for phabricator instance.
  • fixed conversion of datatype
Tue, Apr 23, 7:48 PM
Harbormaster failed remote builds in B5531: Diff 4651 for D1365: Implemented a lister for phabricator instance!
Tue, Apr 23, 7:45 PM
swh-public-ci added a comment to D1365: Implemented a lister for phabricator instance.

Build has FAILED

Tue, Apr 23, 7:45 PM
nahimilega updated the diff for D1365: Implemented a lister for phabricator instance.
  • Fixed null exception in shortName
Tue, Apr 23, 7:42 PM
Harbormaster failed remote builds in B5530: Diff 4650 for D1365: Implemented a lister for phabricator instance!
Tue, Apr 23, 6:38 PM
swh-public-ci added a comment to D1365: Implemented a lister for phabricator instance.

Build has FAILED

Tue, Apr 23, 6:38 PM
nahimilega updated the diff for D1365: Implemented a lister for phabricator instance.
  • Changed indexable and uid type to string
Tue, Apr 23, 6:35 PM
zack accepted D1430: Correct typo "scrapping" -> "scraping".
Tue, Apr 23, 6:20 PM
mcv21 updated the summary of D1430: Correct typo "scrapping" -> "scraping".
Tue, Apr 23, 6:07 PM
Herald added 1 required legal document(s) to D1430: Correct typo "scrapping" -> "scraping": L3 Software Heritage Contributor License Agreement, version 1.0.
Tue, Apr 23, 6:01 PM
zack added a comment to T1683: Decide where should be gathered specs.

I need this decision for different specs, for example:

  • Legacy software deposit
  • Sparse / Metadata deposit (now in docs)
  • Metadata workflow- How do we dill with software metadata (T1344)
Tue, Apr 23, 5:46 PM · Development documentation
Harbormaster failed remote builds in B5528: Diff 4648 for D1365: Implemented a lister for phabricator instance!
Tue, Apr 23, 4:36 PM
swh-public-ci added a comment to D1365: Implemented a lister for phabricator instance.

Build has FAILED

Tue, Apr 23, 4:36 PM
nahimilega updated the diff for D1365: Implemented a lister for phabricator instance.
  • Changed bad response for phabricator lister
Tue, Apr 23, 4:33 PM
vlorentz committed R186:94e5f782b9a1: Apply permissions after db init. (authored by vlorentz).
Apply permissions after db init.
Tue, Apr 23, 4:15 PM
vlorentz committed R186:6b45fcfd8d5a: Add plot.py (authored by vlorentz).
Add plot.py
Tue, Apr 23, 4:15 PM
vlorentz committed R186:7a318f9b246a: Fix pgsql config. (authored by vlorentz).
Fix pgsql config.
Tue, Apr 23, 4:15 PM
vlorentz committed R186:7db9c7c2fbb9: Tell GNU Parallel not to eat error messages. (authored by vlorentz).
Tell GNU Parallel not to eat error messages.
Tue, Apr 23, 4:15 PM
anlambert closed T1655: Web API: more user friendly answer when checking state of non existent save code now requests as Resolved by committing rDWAPPS9a2be798a039: api/origin_save: Return a 404 error when querying non existent request state.
Tue, Apr 23, 3:17 PM · Web app
anlambert committed rDWAPPS9a2be798a039: api/origin_save: Return a 404 error when querying non existent request state (authored by anlambert).
api/origin_save: Return a 404 error when querying non existent request state
Tue, Apr 23, 3:17 PM
nahimilega created P385 Output of docker-compose up swh-web in the S1 Public space.
Tue, Apr 23, 3:08 PM
nahimilega created P384 Output of docker-compose ps in the S1 Public space.
Tue, Apr 23, 2:47 PM
marla.dasilva created T1686: To write visual corporate identity (charte graphique).
Tue, Apr 23, 2:11 PM · Unknown Object (Project)
moranegg added a comment to T1683: Decide where should be gathered specs.

I need this decision for different specs, for example:

  • Legacy software deposit
  • Sparse / Metadata deposit (now in docs)
  • Metadata workflow- How do we dill with software metadata (T1344)
Tue, Apr 23, 11:54 AM · Development documentation
zack triaged T1685: move main website from www.s.o to s.o as Low priority.
Tue, Apr 23, 9:58 AM · System administration, Website
vlorentz triaged T1684: Race condition on person insertion in pgsql storage as Low priority.
Tue, Apr 23, 9:38 AM · Storage manager

Mon, Apr 22

rdicosmo committed rMSLDd6f8ba0e46f7: Added images on provenance graph (authored by rdicosmo).
Added images on provenance graph
Mon, Apr 22, 6:41 PM
moranegg updated the task description for T1683: Decide where should be gathered specs.
Mon, Apr 22, 11:11 AM · Development documentation
moranegg triaged T1683: Decide where should be gathered specs as High priority.
Mon, Apr 22, 11:11 AM · Development documentation
timokratia added a comment to D1369: Guess extension from detected mime type to add to filename.

Thanks for reviewing this diff, and apologies for my late reply and not closing it, I didn't find out how to close it when I first saw your message.

Mon, Apr 22, 7:29 AM

Sun, Apr 21

zack triaged T1682: https://softwareheritage.org does not redirect to https://www.s.o as High priority.
Sun, Apr 21, 7:56 AM · System administration

Fri, Apr 19

vlorentz triaged T1681: Use project metadata as a "lister" as Low priority.
Fri, Apr 19, 11:03 PM · Archive coverage, Indexer, Metadata workflow
anlambert updated subscribers of D1415: Add support for client side rendering of Jupyter notebooks.

There are a million different file formats out there, why are we rendering Markdown and Jupyter notebooks and not something else?

Fri, Apr 19, 8:09 PM
zack added a comment to D1415: Add support for client side rendering of Jupyter notebooks.

I think we need to have a conversation about how we decide to add support for rendering specific file formats in the webapp.
There are a million different file formats out there, why are we rendering Markdown and Jupyter notebooks and not something else?
When we have support for a million (or even just a dozen, really), are we going to test for which-is-which in a long chained series of IFs in a template? That doesn't seem really wise…
How do we decide if something is a given file format or not, especially considering that getting it wrong might even entail security vulnerabilities, injections, etc.? (because we're rendering content we do not control directly)

Fri, Apr 19, 7:49 PM
swh-public-ci added a comment to D1415: Add support for client side rendering of Jupyter notebooks.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/402/ for more details.

Fri, Apr 19, 7:44 PM
anlambert updated the diff for D1415: Add support for client side rendering of Jupyter notebooks.

Update: Check that hljs supports the notebook language prior calling the highlight function

Fri, Apr 19, 7:40 PM
vlorentz committed R186:a1fdedf5e77a: Change VM type for the client, as it's now CPU-bound. (authored by vlorentz).
Change VM type for the client, as it's now CPU-bound.
Fri, Apr 19, 6:00 PM