Page MenuHomeSoftware Heritage
Feed All Stories

Sep 28 2020

ardumont committed rDDEP36d6b760f4a2: Check updated metadata are correctly stored in the metadata storage (authored by ardumont).
Check updated metadata are correctly stored in the metadata storage
Sep 28 2020, 2:12 PM
ardumont closed D4058: Check updated metadata are correctly stored in the metadata storage.
Sep 28 2020, 2:12 PM
ardumont committed rDDEP92f99745f35b: Add functional metadata checks prior to updating them (authored by ardumont).
Add functional metadata checks prior to updating them
Sep 28 2020, 2:12 PM
ardumont closed D4047: Add functional metadata checks prior to actually update metadata.
Sep 28 2020, 2:12 PM
anlambert accepted D4062: declare the swh-docker-dev job.
Sep 28 2020, 2:11 PM
vsellier retitled D4061: Add documentation to test locally from Add documention test locally to Add documentation to test locally.
Sep 28 2020, 2:11 PM
anlambert added inline comments to D4061: Add documentation to test locally.
Sep 28 2020, 2:11 PM
ardumont accepted D4062: declare the swh-docker-dev job.

\m/

Sep 28 2020, 2:09 PM
ardumont accepted D4061: Add documentation to test locally.
Sep 28 2020, 2:09 PM
vsellier created D4062: declare the swh-docker-dev job.
Sep 28 2020, 2:08 PM
vsellier retitled D4061: Add documentation to test locally from Declare the swh-docker-dev job to Add documention test locally.
Sep 28 2020, 2:08 PM
vsellier updated the diff for D4061: Add documentation to test locally.

Use only one commit per diff

Sep 28 2020, 2:08 PM
ardumont added a comment to D4061: Add documentation to test locally.

Split this into 2 diffs, one for the doc, the other one to actually add the job and we are good ;)

Sep 28 2020, 2:03 PM
vsellier retitled D4061: Add documentation to test locally from - Add documention test locally - Declare the swh-docker-dev job to Declare the swh-docker-dev job.
Sep 28 2020, 2:02 PM
swh-public-ci added a comment to D4060: test_deposit_atom: Fix incomplete tests.

Build is green

Sep 28 2020, 2:00 PM
vsellier created D4061: Add documentation to test locally.
Sep 28 2020, 2:00 PM
vlorentz accepted D4058: Check updated metadata are correctly stored in the metadata storage.
Sep 28 2020, 2:00 PM
vlorentz accepted D4060: test_deposit_atom: Fix incomplete tests.
Sep 28 2020, 1:59 PM
vlorentz accepted D4047: Add functional metadata checks prior to actually update metadata.

please add a module docstring to checks.py

Sep 28 2020, 1:59 PM
ardumont created D4060: test_deposit_atom: Fix incomplete tests.
Sep 28 2020, 1:58 PM
ardumont added inline comments to D4058: Check updated metadata are correctly stored in the metadata storage.
Sep 28 2020, 1:51 PM
vlorentz added inline comments to D4058: Check updated metadata are correctly stored in the metadata storage.
Sep 28 2020, 1:40 PM
vlorentz accepted D4059: deposit_update: Drop the unneeded check on the directory_missing call.
Sep 28 2020, 1:38 PM
swh-public-ci added a comment to D4058: Check updated metadata are correctly stored in the metadata storage.

Build is green

Sep 28 2020, 1:28 PM
ardumont updated the summary of D4058: Check updated metadata are correctly stored in the metadata storage.
Sep 28 2020, 1:25 PM
ardumont updated the diff for D4058: Check updated metadata are correctly stored in the metadata storage.

Add origin to the raw extrinsic metadata stored in the metadata storage

Sep 28 2020, 1:25 PM
swh-public-ci added a comment to D4059: deposit_update: Drop the unneeded check on the directory_missing call.

Build is green

Sep 28 2020, 1:21 PM
ardumont created D4059: deposit_update: Drop the unneeded check on the directory_missing call.
Sep 28 2020, 1:19 PM
ardumont committed rDDEPd1f3f69c9016: Allow deposit metadata update on deposit already completed (authored by ardumont).
Allow deposit metadata update on deposit already completed
Sep 28 2020, 1:16 PM
ardumont closed D4013: Allow deposit metadata update on deposit already complete.
Sep 28 2020, 1:16 PM
swh-public-ci added a comment to D4058: Check updated metadata are correctly stored in the metadata storage.

Build is green

Sep 28 2020, 1:15 PM
vlorentz added a comment to D4058: Check updated metadata are correctly stored in the metadata storage.

You should add an origin context to the RawExtrinsicMetadata object, to be consistent with the deposit loader.

Sep 28 2020, 1:15 PM
vlorentz accepted D4013: Allow deposit metadata update on deposit already complete.

Let's not forget to remove the directory_missing check later

Sep 28 2020, 1:14 PM
swh-public-ci added a comment to D4047: Add functional metadata checks prior to actually update metadata.

Build is green

Sep 28 2020, 1:13 PM
swh-public-ci added a comment to D4013: Allow deposit metadata update on deposit already complete.

Build is green

Sep 28 2020, 1:10 PM
ardumont updated the diff for D4058: Check updated metadata are correctly stored in the metadata storage.

Rebase

Sep 28 2020, 1:08 PM
ardumont updated the diff for D4047: Add functional metadata checks prior to actually update metadata.

Rebase

Sep 28 2020, 1:08 PM
ardumont updated the diff for D4013: Allow deposit metadata update on deposit already complete.

Use the PUT endpoint

Sep 28 2020, 1:07 PM
ardumont committed rDDEP45c7c35e471c: Fix deposit atom test to actually check the response message (authored by ardumont).
Fix deposit atom test to actually check the response message
Sep 28 2020, 1:07 PM
ardumont closed D4057: Fix deposit atom test to actually check the response message.
Sep 28 2020, 1:07 PM
vlorentz accepted D4057: Fix deposit atom test to actually check the response message.
Sep 28 2020, 1:04 PM
ardumont added a comment to D4058: Check updated metadata are correctly stored in the metadata storage.

Note that some checks are commented because they currently fail. Those are
storage issues. The fetcher and authority metadata fields are not properly set
when read from the raw_extrinsic_metadata_get endpoint somehow ¯\_(ツ)_/¯.
That should most probably be the subject for another storage diff.

Sep 28 2020, 12:31 PM
ardumont committed rDDEPb04c457d6568: Fix transition deposit to deposited state with empty post on SE IRI (authored by ardumont).
Fix transition deposit to deposited state with empty post on SE IRI
Sep 28 2020, 12:14 PM
ardumont closed D4052: Fix transition deposit to deposited state with empty post on SE IRI.
Sep 28 2020, 12:14 PM
ardumont added inline comments to D4057: Fix deposit atom test to actually check the response message.
Sep 28 2020, 12:12 PM
vlorentz requested changes to D4057: Fix deposit atom test to actually check the response message.
Sep 28 2020, 12:11 PM
vlorentz accepted D4052: Fix transition deposit to deposited state with empty post on SE IRI.
Sep 28 2020, 12:10 PM
vlorentz requested changes to D4013: Allow deposit metadata update on deposit already complete.
Sep 28 2020, 12:03 PM
vlorentz added a comment to D4013: Allow deposit metadata update on deposit already complete.

as discussed on side-channels, it should be a PUT

Sep 28 2020, 12:03 PM
vlorentz requested changes to D4058: Check updated metadata are correctly stored in the metadata storage.
Sep 28 2020, 12:02 PM
vsellier closed T2644: swh-docker-dev CI build - broken since build #569 - 404 not found for psql apt repo as Resolved.

I have added the `--pull` option in the job configuration so the python image will always be pulled (if needed) during the build :

Sep 28 2020, 10:59 AM · Docker environment, System administration
ardumont added a comment to T2313: Archive git.fsfe.org (Gitea).

The lister is deployed, this forge is not listed though (codeberg.org is).

Sep 28 2020, 10:54 AM · Archive coverage, Lister
tenma closed T2632: swh scanner fail to start when configuration file is missing as Resolved.
Sep 28 2020, 10:14 AM · Code scanner
tenma closed D4046: Fix default config file may be absent in scanner cli.
Sep 28 2020, 10:11 AM
douardda added a comment to T2313: Archive git.fsfe.org (Gitea).

Can this be closed now? What's missing? Adding a listing task?

Sep 28 2020, 9:47 AM · Archive coverage, Lister

Sep 27 2020

ardumont added a comment to D4013: Allow deposit metadata update on deposit already complete.

Like i said in the update diff message, there remains the check on the actual metadata stored (FIXME for now).
I'd like to dedicate to this in another iteration (diff) though.
So the eventual discussion on that can happen there.

Sep 27 2020, 5:12 PM
swh-public-ci added a comment to D4058: Check updated metadata are correctly stored in the metadata storage.

Build is green

Sep 27 2020, 3:42 PM
ardumont updated the summary of D4058: Check updated metadata are correctly stored in the metadata storage.
Sep 27 2020, 3:41 PM
ardumont created D4058: Check updated metadata are correctly stored in the metadata storage.
Sep 27 2020, 3:39 PM
ardumont added a comment to D4013: Allow deposit metadata update on deposit already complete.

@moranegg @vlorentz heads up (for tomorrow), I should have adressed all concerns now ;)

Sep 27 2020, 12:03 PM
swh-public-ci added a comment to D4047: Add functional metadata checks prior to actually update metadata.

Build is green

Sep 27 2020, 11:54 AM
swh-public-ci added a comment to D4013: Allow deposit metadata update on deposit already complete.

Build is green

Sep 27 2020, 11:51 AM
ardumont updated the diff for D4047: Add functional metadata checks prior to actually update metadata.

Rebase on latest D4013

Sep 27 2020, 11:50 AM
ardumont updated the diff for D4013: Allow deposit metadata update on deposit already complete.

Adapt according to remaining unadapted suggestions (and more):

  • Actually add new metadata information during update instead of replacing those (technically also move the code from the PUT to the POST method part)
  • Update docstrings accordingly
  • Rename storage attribute to storage_metadata
  • Drop self.provider setup to actually reference the deposit client as the provider (and adapt the other part already using it)
  • Send raw metadata into the metadata storage (instead of the parsed json one)
Sep 27 2020, 11:48 AM
vsellier claimed T2644: swh-docker-dev CI build - broken since build #569 - 404 not found for psql apt repo.
Sep 27 2020, 11:36 AM · Docker environment, System administration
vsellier added a comment to T2644: swh-docker-dev CI build - broken since build #569 - 404 not found for psql apt repo.

It seems the python 3.7 images on the jenkins master is quite old :

root@thyssen:~# docker images | grep python
python                     3.7                  a4cc999cf2aa        16 months ago       929MB
Sep 27 2020, 11:36 AM · Docker environment, System administration
ardumont added inline comments to D4047: Add functional metadata checks prior to actually update metadata.
Sep 27 2020, 11:30 AM
ardumont updated the summary of D4047: Add functional metadata checks prior to actually update metadata.
Sep 27 2020, 11:30 AM
vlorentz added a comment to T2644: swh-docker-dev CI build - broken since build #569 - 404 not found for psql apt repo.

I'm guessing it's because $VERSION_CODENAME is empty here: https://forge.softwareheritage.org/source/swh-environment/browse/master/docker/Dockerfile$3

Sep 27 2020, 11:17 AM · Docker environment, System administration
zack triaged T2644: swh-docker-dev CI build - broken since build #569 - 404 not found for psql apt repo as High priority.
Sep 27 2020, 11:10 AM · Docker environment, System administration

Sep 26 2020

ardumont committed rDDEPf26ea4492695: test: Update missing scenario on update metadata with multipart (post) (authored by ardumont).
test: Update missing scenario on update metadata with multipart (post)
Sep 26 2020, 7:48 PM
ardumont closed D4049: test: Update missing scenario on update metadata with multipart (post).
Sep 26 2020, 7:48 PM
ardumont committed rDDEP251a0031eada: test: Update missing scenario on update metadata with multipart (put) (authored by ardumont).
test: Update missing scenario on update metadata with multipart (put)
Sep 26 2020, 7:47 PM
ardumont closed D4048: test: Update missing scenario on update metadata with multipart (put).
Sep 26 2020, 7:47 PM
swh-public-ci added a comment to D4047: Add functional metadata checks prior to actually update metadata.

Build is green

Sep 26 2020, 7:25 PM
swh-public-ci added a comment to D4013: Allow deposit metadata update on deposit already complete.

Build is green

Sep 26 2020, 7:22 PM
swh-public-ci added a comment to D4057: Fix deposit atom test to actually check the response message.

Build is green

Sep 26 2020, 7:20 PM
swh-public-ci added a comment to D4052: Fix transition deposit to deposited state with empty post on SE IRI.

Build is green

Sep 26 2020, 7:17 PM
swh-public-ci added a comment to D4049: test: Update missing scenario on update metadata with multipart (post).

Build is green

Sep 26 2020, 7:15 PM
swh-public-ci added a comment to D4048: test: Update missing scenario on update metadata with multipart (put).

Build is green

Sep 26 2020, 7:13 PM
swh-public-ci added a comment to D4047: Add functional metadata checks prior to actually update metadata.

Build is green

Sep 26 2020, 7:10 PM
ardumont updated the diff for D4047: Add functional metadata checks prior to actually update metadata.

Rebase

Sep 26 2020, 7:10 PM
ardumont updated the diff for D4013: Allow deposit metadata update on deposit already complete.

Rebase

Sep 26 2020, 7:10 PM
ardumont updated the diff for D4057: Fix deposit atom test to actually check the response message.

Rebase

Sep 26 2020, 7:09 PM
ardumont updated the diff for D4052: Fix transition deposit to deposited state with empty post on SE IRI.

Rebase

Sep 26 2020, 7:09 PM
ardumont updated the diff for D4049: test: Update missing scenario on update metadata with multipart (post).

Rebase

Sep 26 2020, 7:09 PM
ardumont updated the diff for D4048: test: Update missing scenario on update metadata with multipart (put).

/me *sighs* second round rebase, i forgot to pull --rebase first

Sep 26 2020, 7:08 PM
swh-public-ci added a comment to D4013: Allow deposit metadata update on deposit already complete.

Build is green

Sep 26 2020, 7:07 PM
ardumont updated the diff for D4047: Add functional metadata checks prior to actually update metadata.

Rebase on latest D4013

Sep 26 2020, 7:06 PM
ardumont updated the diff for D4013: Allow deposit metadata update on deposit already complete.

Fix test

Sep 26 2020, 7:05 PM
ardumont planned changes to D4013: Allow deposit metadata update on deposit already complete.
Sep 26 2020, 7:03 PM
ardumont updated the summary of D4047: Add functional metadata checks prior to actually update metadata.
Sep 26 2020, 7:03 PM
ardumont updated the summary of D4047: Add functional metadata checks prior to actually update metadata.
Sep 26 2020, 7:03 PM
ardumont updated the summary of D4052: Fix transition deposit to deposited state with empty post on SE IRI.
Sep 26 2020, 7:02 PM
ardumont updated the summary of D4057: Fix deposit atom test to actually check the response message.
Sep 26 2020, 7:02 PM
ardumont added a comment to D4057: Fix deposit atom test to actually check the response message.

There should be another diff in regards to that test.
At least to add a file similar to the original one (with the xmlns namespace...) which should pass the same way... (heads up, it does not...)
Currently, the check on emptiness fails with that xmlns, why i removed it here (as i'm being lazy right now).
Like i said, I'll adapt in another diff. I don't want to be sidetracked too much right now.

Sep 26 2020, 7:01 PM
Harbormaster failed remote builds in B15611: Diff 14313 for D4013: Allow deposit metadata update on deposit already complete!
Sep 26 2020, 7:00 PM
swh-public-ci added a comment to D4013: Allow deposit metadata update on deposit already complete.

Build has FAILED

Sep 26 2020, 7:00 PM
swh-public-ci added a comment to D4057: Fix deposit atom test to actually check the response message.

Build is green

Sep 26 2020, 6:57 PM
ardumont updated the summary of D4013: Allow deposit metadata update on deposit already complete.
Sep 26 2020, 6:57 PM