Page MenuHomeSoftware Heritage
Feed All Stories

Apr 17 2019

ardumont created D1422: deposit: Wrongly formatted xml should return a 400 bad request.
Apr 17 2019, 2:19 PM · SWORD deposit
ardumont added a comment to D1419: cli.deposit: Add mandatory flags to generate the metadata xml.

I'm going to try and run it, but am a bit blocked with swh-deposit command................

Apr 17 2019, 2:16 PM
moranegg added a comment to D1419: cli.deposit: Add mandatory flags to generate the metadata xml.

I'm going to try and run it, but am a bit blocked with swh-deposit command................
Maybe @douardda could validate he can do a deposit?

Apr 17 2019, 1:50 PM
vlorentz created P382 test.tf.
Apr 17 2019, 1:37 PM
swh-public-ci added a comment to D1419: cli.deposit: Add mandatory flags to generate the metadata xml.

Build is green
See https://jenkins.softwareheritage.org/job/DDEP/job/tox/120/ for more details.

Apr 17 2019, 12:01 PM
ardumont added a comment to T1638: Deposit: error when submitting through cli.

the metadata file linuxperf-v4.3.1.27.metadata.xml (as per the parsing error xml.parsers.expat.ExpatError: syntax error: line 1, column 49 implies something badly formatted).

Apr 17 2019, 12:00 PM · SWORD deposit
ardumont updated the diff for D1419: cli.deposit: Add mandatory flags to generate the metadata xml.

Rebase on latest master

Apr 17 2019, 11:58 AM
ardumont updated the task description for T1648: Change `url` and `external-id` from mandatory to optional metadata.
Apr 17 2019, 11:56 AM · SWORD deposit
ardumont committed rDDEP56e903728210: check: Remove url and external_identifier from mandatory fields (authored by ardumont).
check: Remove url and external_identifier from mandatory fields
Apr 17 2019, 11:56 AM
ardumont committed rDDEP45aeec443d7a: utils: Enforce necessary setup for deposit and associated client (authored by ardumont).
utils: Enforce necessary setup for deposit and associated client
Apr 17 2019, 11:56 AM
ardumont committed rDDEP3bff42e429ce: deposit_read: Compute origin-url from the deposit's setup (authored by ardumont).
deposit_read: Compute origin-url from the deposit's setup
Apr 17 2019, 11:56 AM
ardumont closed D1421: Change `url` and `external-id` from mandatory to optional metadata.
Apr 17 2019, 11:56 AM
swh-public-ci added a comment to D1421: Change `url` and `external-id` from mandatory to optional metadata.

Build is green
See https://jenkins.softwareheritage.org/job/DDEP/job/tox/119/ for more details.

Apr 17 2019, 11:54 AM
ardumont updated the diff for D1421: Change `url` and `external-id` from mandatory to optional metadata.

Plug to master

Apr 17 2019, 11:51 AM
zack accepted D1421: Change `url` and `external-id` from mandatory to optional metadata.
Apr 17 2019, 11:50 AM
swh-public-ci added a comment to D1421: Change `url` and `external-id` from mandatory to optional metadata.

Build is green
See https://jenkins.softwareheritage.org/job/DDEP/job/tox/118/ for more details.

Apr 17 2019, 11:49 AM
ardumont updated the diff for D1421: Change `url` and `external-id` from mandatory to optional metadata.
  • utils: Enforce necessary setup for deposit and associated client
Apr 17 2019, 11:46 AM
zack requested changes to D1421: Change `url` and `external-id` from mandatory to optional metadata.
Apr 17 2019, 11:32 AM
ardumont added inline comments to D1421: Change `url` and `external-id` from mandatory to optional metadata.
Apr 17 2019, 11:29 AM
ardumont moved T1638: Deposit: error when submitting through cli from High priority to In progress on the SWORD deposit board.
Apr 17 2019, 11:19 AM · SWORD deposit
ardumont moved T1637: Adapt deposit's creation/publication date to the SWH data-structure from In code review to Landed/Tests/Validations (staging) on the SWORD deposit board.
Apr 17 2019, 11:19 AM · SWORD deposit
ardumont closed T1637: Adapt deposit's creation/publication date to the SWH data-structure as Resolved.
Apr 17 2019, 11:19 AM · SWORD deposit
ardumont closed T1637: Adapt deposit's creation/publication date to the SWH data-structure, a subtask of T1646: Enable compliance scenario for deposits, as Resolved.
Apr 17 2019, 11:19 AM · SWORD deposit
ardumont closed T1637: Adapt deposit's creation/publication date to the SWH data-structure, a subtask of T1581: Deposit: improvements, as Resolved.
Apr 17 2019, 11:19 AM · SWORD deposit
ardumont added a comment to D1416: deposit: Adapt revision dates setting according to new rules.

This revision was not accepted when it landed; it landed in state Needs Review.Wed, Apr 17, 10:52

Apr 17 2019, 11:17 AM · SWORD deposit
Harbormaster failed to build B5476: rDDEPc183276262e1: Refactor: Extract date utils manipulation function into utils module for rDDEPc183276262e1: Refactor: Extract date utils manipulation function into utils module!
Apr 17 2019, 11:05 AM
swh-public-ci added a comment to D1421: Change `url` and `external-id` from mandatory to optional metadata.

Build is green
See https://jenkins.softwareheritage.org/job/DDEP/job/tox/117/ for more details.

Apr 17 2019, 11:03 AM
ardumont updated the diff for D1421: Change `url` and `external-id` from mandatory to optional metadata.

Fix in-between interactive rebase

Apr 17 2019, 11:00 AM
swh-public-ci added a comment to D1421: Change `url` and `external-id` from mandatory to optional metadata.

Build has FAILED

Apr 17 2019, 10:56 AM
Harbormaster failed remote builds in B5479: Diff 4609 for D1421: Change `url` and `external-id` from mandatory to optional metadata!
Apr 17 2019, 10:56 AM
swh-public-ci added a comment to D1419: cli.deposit: Add mandatory flags to generate the metadata xml.

Build is green
See https://jenkins.softwareheritage.org/job/DDEP/job/tox/115/ for more details.

Apr 17 2019, 10:55 AM
ardumont updated the diff for D1421: Change `url` and `external-id` from mandatory to optional metadata.

Rebase on latest master

Apr 17 2019, 10:53 AM
ardumont updated the diff for D1419: cli.deposit: Add mandatory flags to generate the metadata xml.

Rebase on latest master

Apr 17 2019, 10:52 AM
ardumont committed rDDEPa7a7170df586: utils: Make parsing date result in a reasonable date (authored by ardumont).
utils: Make parsing date result in a reasonable date
Apr 17 2019, 10:52 AM
ardumont committed rDDEPc183276262e1: Refactor: Extract date utils manipulation function into utils module (authored by ardumont).
Refactor: Extract date utils manipulation function into utils module
Apr 17 2019, 10:52 AM
ardumont committed rDDEP00a8abd44f36: tests: Factorize metadata xml duplication to explicit tested date (authored by ardumont).
tests: Factorize metadata xml duplication to explicit tested date
Apr 17 2019, 10:52 AM
ardumont committed rDDEP37b48717a83f: loader: Use metadata date as author/committer date if provided (authored by ardumont).
loader: Use metadata date as author/committer date if provided
Apr 17 2019, 10:52 AM
ardumont committed rDDEP0138edcadef7: deposit_read: Refactor to clarify intents and docstring (authored by ardumont).
deposit_read: Refactor to clarify intents and docstring
Apr 17 2019, 10:52 AM
ardumont closed D1416: deposit: Adapt revision dates setting according to new rules.
Apr 17 2019, 10:52 AM · SWORD deposit
swh-public-ci added a comment to D1416: deposit: Adapt revision dates setting according to new rules.

Build is green
See https://jenkins.softwareheritage.org/job/DDEP/job/tox/114/ for more details.

Apr 17 2019, 10:51 AM · SWORD deposit
swh-public-ci added a comment to D1421: Change `url` and `external-id` from mandatory to optional metadata.

Build is green
See https://jenkins.softwareheritage.org/job/DDEP/job/tox/113/ for more details.

Apr 17 2019, 10:48 AM
ardumont updated the diff for D1416: deposit: Adapt revision dates setting according to new rules.

Plug on master branch

Apr 17 2019, 10:46 AM · SWORD deposit
ardumont updated the diff for D1421: Change `url` and `external-id` from mandatory to optional metadata.

Rebase on dedicated branch

Apr 17 2019, 10:45 AM
zack added inline comments to D1421: Change `url` and `external-id` from mandatory to optional metadata.
Apr 17 2019, 10:14 AM
ardumont lowered the priority of T1638: Deposit: error when submitting through cli from High to Normal.
Apr 17 2019, 10:02 AM · SWORD deposit
ardumont added a comment to T1638: Deposit: error when submitting through cli.

I have badly dealt with this issue.

Apr 17 2019, 10:02 AM · SWORD deposit
ardumont added inline comments to D1416: deposit: Adapt revision dates setting according to new rules.
Apr 17 2019, 9:34 AM · SWORD deposit
ardumont added a comment to D1411: Refactor deposit cli and some other improvments.

@douardda @olasd I have changed the migration to run the 4 steps
mentioned \m/

To do that, i iterated with git (amending) and reused the data files
generated through multiple calls to make db-prepare (wrapper on
swh.deposit.manage makemigrations)...

In the end, I checked the following command to see the sql generated
(in between last version with the old type and the new one):

python3 -m swh.deposit.manage sqlmigrate deposit 0015_depositrequest_typemigration
BEGIN;
--
-- Add field type2 to depositrequest
--
ALTER TABLE "deposit_request" ADD COLUMN "type2" varchar(8) NULL;
--
-- MIGRATION NOW PERFORMS OPERATION THAT CANNOT BE WRITTEN AS SQL:
-- Raw Python operation
--
--
-- Remove field type from depositrequest
--
SET CONSTRAINTS "deposit_request_type_id_e9882b75_fk_deposit_request_type_id" IMMEDIATE; ALTER TABLE "deposit_request" DROP CONSTRAINT "deposit_request_type_id_e9882b75_fk_deposit_request_type_id";
ALTER TABLE "deposit_request" DROP COLUMN "type_id" CASCADE;
--
-- Rename field type2 on depositrequest to type
--
ALTER TABLE "deposit_request" RENAME COLUMN "type2" TO "type";
--
-- Delete model DepositRequestType
--
DROP TABLE "deposit_request_type" CASCADE;
COMMIT;

Expectedly, that does not show the data migration part which happens
through python. But that indeed mentions the migration step ;)

Apr 17 2019, 9:26 AM

Apr 16 2019

anlambert claimed T1655: Web API: more user friendly answer when checking state of non existent save code now requests.

Indeed, returning a 404 error seems the best option here. Will fix that asap.

Apr 16 2019, 7:33 PM · Web app
ardumont added reviewers for D1419: cli.deposit: Add mandatory flags to generate the metadata xml: moranegg, douardda.
Apr 16 2019, 7:31 PM
swh-public-ci added a comment to D1416: deposit: Adapt revision dates setting according to new rules.

Build is green
See https://jenkins.softwareheritage.org/job/DDEP/job/tox/112/ for more details.

Apr 16 2019, 7:29 PM · SWORD deposit
ardumont updated the diff for D1416: deposit: Adapt revision dates setting according to new rules.

Update test docstring to match the reality of the test

Apr 16 2019, 7:26 PM · SWORD deposit
ardumont moved T1648: Change `url` and `external-id` from mandatory to optional metadata from In progress to In code review on the SWORD deposit board.
Apr 16 2019, 7:21 PM · SWORD deposit
zack triaged T1655: Web API: more user friendly answer when checking state of non existent save code now requests as Low priority.
Apr 16 2019, 7:16 PM · Web app
swh-public-ci added a comment to D1421: Change `url` and `external-id` from mandatory to optional metadata.

Build is green
See https://jenkins.softwareheritage.org/job/DDEP/job/tox/111/ for more details.

Apr 16 2019, 7:12 PM
ardumont created D1421: Change `url` and `external-id` from mandatory to optional metadata.
Apr 16 2019, 7:09 PM
zack triaged T1654: expired SSL certificate for archive.internal.softwareheritage.org as Normal priority.
Apr 16 2019, 7:05 PM · System administration
ardumont claimed T1648: Change `url` and `external-id` from mandatory to optional metadata.
Apr 16 2019, 6:32 PM · SWORD deposit
ardumont moved T1648: Change `url` and `external-id` from mandatory to optional metadata from High priority to In progress on the SWORD deposit board.
Apr 16 2019, 6:31 PM · SWORD deposit
anlambert planned changes to D1415: Add support for client side rendering of Jupyter notebooks.
Apr 16 2019, 5:58 PM
swh-public-ci added a comment to D1415: Add support for client side rendering of Jupyter notebooks.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/392/ for more details.

Apr 16 2019, 5:43 PM
anlambert updated the diff for D1415: Add support for client side rendering of Jupyter notebooks.

Fix tests

Apr 16 2019, 5:39 PM
Harbormaster failed remote builds in B5467: Diff 4601 for D1415: Add support for client side rendering of Jupyter notebooks!
Apr 16 2019, 5:32 PM
swh-public-ci added a comment to D1415: Add support for client side rendering of Jupyter notebooks.

Build has FAILED

Apr 16 2019, 5:32 PM
anlambert updated the diff for D1415: Add support for client side rendering of Jupyter notebooks.
  • add support for math typesetting upon notebook rendering through the use of the MathJax library
Apr 16 2019, 5:28 PM
anlambert committed rDWAPPS0706c426accd: package.json: Upgrade depedencies (authored by anlambert).
package.json: Upgrade depedencies
Apr 16 2019, 5:26 PM
vlorentz requested changes to D1420: Made changes to adapt it to new content_find return type.
Apr 16 2019, 5:25 PM
ardumont moved T1650: Generate xml for SWORD protocol with deposit client from In progress to In code review on the SWORD deposit board.
Apr 16 2019, 5:21 PM · SWORD deposit
ardumont moved T1650: Generate xml for SWORD protocol with deposit client from High priority to In progress on the SWORD deposit board.
Apr 16 2019, 5:21 PM · SWORD deposit
ardumont updated the language for P381 deposit-client: metadata xml generated from new cli flags from autodetect to python.
Apr 16 2019, 5:16 PM · SWORD deposit
ardumont added a comment to P381 deposit-client: metadata xml generated from new cli flags.

Related D1419

Apr 16 2019, 5:16 PM · SWORD deposit
ardumont edited P381 deposit-client: metadata xml generated from new cli flags.
Apr 16 2019, 5:16 PM · SWORD deposit
ardumont updated the test plan for D1419: cli.deposit: Add mandatory flags to generate the metadata xml.
Apr 16 2019, 5:13 PM
ardumont updated the test plan for D1419: cli.deposit: Add mandatory flags to generate the metadata xml.
Apr 16 2019, 5:12 PM
ardumont added a comment to D1419: cli.deposit: Add mandatory flags to generate the metadata xml.

ardumont: could you do a paste with the generated xml? I can't seem to be able to run the client today (i'll have more time to reconstruct that part locally tomorrow)

Apr 16 2019, 5:10 PM
ftigeot added a comment to T1442: Replace Munin graphs with Grafana/Prometheus dashboards.

Even though most/all of the Munin metrics are provided by Prometheus, Munin also provides graphs.
It is these graphs we are still missing.

Apr 16 2019, 5:09 PM · Sprint 2018 12, System administration
ardumont edited P381 deposit-client: metadata xml generated from new cli flags.
Apr 16 2019, 5:06 PM · SWORD deposit
ardumont created P381 deposit-client: metadata xml generated from new cli flags.
Apr 16 2019, 5:06 PM · SWORD deposit
swh-public-ci added a comment to D1416: deposit: Adapt revision dates setting according to new rules.

Build is green
See https://jenkins.softwareheritage.org/job/DDEP/job/tox/110/ for more details.

Apr 16 2019, 5:05 PM · SWORD deposit
ardumont updated the diff for D1416: deposit: Adapt revision dates setting according to new rules.
  • utils: Make parsing date result in a reasonable date
Apr 16 2019, 5:02 PM · SWORD deposit
ftigeot triaged T1653: Prometheus rate functions considered unreliable as Normal priority.
Apr 16 2019, 4:55 PM · Sprint 2018 12, System administration
Harbormaster failed remote builds in B5464: Diff 4599 for D1420: Made changes to adapt it to new content_find return type!
Apr 16 2019, 4:41 PM
swh-public-ci added a comment to D1420: Made changes to adapt it to new content_find return type.

Build has FAILED

Apr 16 2019, 4:41 PM
faux updated the diff for D1420: Made changes to adapt it to new content_find return type.

Made the required changes

Apr 16 2019, 4:37 PM
vlorentz requested changes to D1420: Made changes to adapt it to new content_find return type.
Apr 16 2019, 3:26 PM
moranegg added a comment to D1416: deposit: Adapt revision dates setting according to new rules.
======================= 112 passed, 1 warnings in 16.33 seconds ========================
Exception ignored in: <function WeakValueDictionary.__init__.<locals>.remove at 0x7ff254794840>
Traceback (most recent call last):
  File "/home/morane/Documents/code/swh-environment/swh-deposit/.tox/py3/lib/python3.5/weakref.py", line 117, in remove
TypeError: 'NoneType' object is not callable
Exception ignored in: <function WeakValueDictionary.__init__.<locals>.remove at 0x7ff254794840>
Traceback (most recent call last):
  File "/home/morane/Documents/code/swh-environment/swh-deposit/.tox/py3/lib/python3.5/weakref.py", line 117, in remove
TypeError: 'NoneType' object is not callable
WARNING [pifpaf.drivers] `psutil.Popen(pid=5224, status='terminated')` is already gone, sending SIGKILL to its process group
WARNING [pifpaf.drivers] `psutil.Popen(pid=5207, status='terminated')` is already gone, sending SIGKILL to its process group
WARNING [pifpaf.drivers] `psutil.Popen(pid=5202, status='terminated')` is already gone, sending SIGKILL to its process group
_______________________________________ summary ________________________________________
  flake8: commands succeeded
  py3: commands succeeded
  congratulations :)
Apr 16 2019, 3:21 PM · SWORD deposit
swh-public-ci added a comment to D1416: deposit: Adapt revision dates setting according to new rules.

Build is green
See https://jenkins.softwareheritage.org/job/DDEP/job/tox/109/ for more details.

Apr 16 2019, 3:15 PM · SWORD deposit
ardumont added inline comments to D1416: deposit: Adapt revision dates setting according to new rules.
Apr 16 2019, 3:13 PM · SWORD deposit
ardumont updated the diff for D1416: deposit: Adapt revision dates setting according to new rules.
  • Refactor: Extract date utils manipulation function into utils module
Apr 16 2019, 3:11 PM · SWORD deposit
douardda added a comment to T1442: Replace Munin graphs with Grafana/Prometheus dashboards.

Indeed it is the object of T1428. That's why I am a bit puzzled the work you have in progress does not simply target T1356. I was expecting some response to this very task in your grafanalib based code, which I did not find. So I was wondering if I missed something, that some data where still in munin only.

Apr 16 2019, 3:04 PM · Sprint 2018 12, System administration
swh-public-ci added a comment to D1416: deposit: Adapt revision dates setting according to new rules.

Build is green
See https://jenkins.softwareheritage.org/job/DDEP/job/tox/108/ for more details.

Apr 16 2019, 2:53 PM · SWORD deposit
ardumont updated the diff for D1416: deposit: Adapt revision dates setting according to new rules.
  • tests: Factorize metadata xml duplication to explicit tested date
Apr 16 2019, 2:50 PM · SWORD deposit
swh-public-ci added a comment to D1416: deposit: Adapt revision dates setting according to new rules.

Build is green
See https://jenkins.softwareheritage.org/job/DDEP/job/tox/107/ for more details.

Apr 16 2019, 2:42 PM · SWORD deposit
ardumont updated the diff for D1416: deposit: Adapt revision dates setting according to new rules.
  • deposit_read: Refactor to clarify intents and docstring
Apr 16 2019, 2:39 PM · SWORD deposit
swh-public-ci added a comment to D1420: Made changes to adapt it to new content_find return type.

Build has FAILED

Apr 16 2019, 2:30 PM
Harbormaster failed remote builds in B5460: Diff 4595 for D1420: Made changes to adapt it to new content_find return type!
Apr 16 2019, 2:30 PM
Herald added a reviewer for D1420: Made changes to adapt it to new content_find return type: Reviewers.
Apr 16 2019, 2:27 PM
faux added a revision to T1349: Storage.content_find should return all matches, not just one.: D1420: Made changes to adapt it to new content_find return type.
Apr 16 2019, 2:27 PM · Easy hack, Storage manager
Harbormaster failed remote builds in B5459: Diff 4594 for D1288: Storage.content_find returns list instead of single value!
Apr 16 2019, 2:23 PM
swh-public-ci added a comment to D1288: Storage.content_find returns list instead of single value.

Build has FAILED

Apr 16 2019, 2:23 PM