Page MenuHomeSoftware Heritage
Feed All Stories

Jan 27 2020

swh-public-ci added a comment to D2582: Web API endpoint /known/.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/509/ for more details.

Jan 27 2020, 5:54 PM
swh-public-ci added a comment to D2582: Web API endpoint /known/.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/508/ for more details.

Jan 27 2020, 5:42 PM
swh-public-ci added a comment to D2566: Add Cassandra backend..

Build was aborted

Jan 27 2020, 5:42 PM
vlorentz committed rDSTO1775eddf61b3: in_memory: Fix content_get_metadata when there is no 'data' key. (authored by vlorentz).
in_memory: Fix content_get_metadata when there is no 'data' key.
Jan 27 2020, 5:38 PM
Harbormaster failed to build B10306: rDSTO0f51e8a08692: Remove cur/db arguments from the in-mem storage. for rDSTO0f51e8a08692: Remove cur/db arguments from the in-mem storage.!
Jan 27 2020, 5:34 PM
swh-public-ci added a comment to D2582: Web API endpoint /known/.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/cypress-diff/507/ for more details.

Jan 27 2020, 5:31 PM
Harbormaster failed to build B10305: rDSTO1cd53c1fbf57: Implement content_update for the in-mem storage. for rDSTO1cd53c1fbf57: Implement content_update for the in-mem storage.!
Jan 27 2020, 5:30 PM
ardumont committed rPCASS7ffce5bcd79c: d/changelog: New release for buster (authored by ardumont).
d/changelog: New release for buster
Jan 27 2020, 5:28 PM
vlorentz committed rDSTO0f51e8a08692: Remove cur/db arguments from the in-mem storage. (authored by vlorentz).
Remove cur/db arguments from the in-mem storage.
Jan 27 2020, 5:27 PM
vlorentz closed D2584: Remove cur/db arguments from the in-mem storage..
Jan 27 2020, 5:27 PM
vlorentz committed rDSTO1cd53c1fbf57: Implement content_update for the in-mem storage. (authored by vlorentz).
Implement content_update for the in-mem storage.
Jan 27 2020, 5:27 PM
vlorentz closed D2583: Implement content_update for the in-mem storage..
Jan 27 2020, 5:27 PM
Harbormaster failed remote builds in B10276: Diff 9215 for D2582: Web API endpoint /known/!
Jan 27 2020, 5:19 PM
swh-public-ci added a comment to D2582: Web API endpoint /known/.

Build was aborted

Jan 27 2020, 5:19 PM
ardumont committed rDDOCdb1725926770: docs: Update swh-docker-dev mentions to swh-environment/docker (authored by ardumont).
docs: Update swh-docker-dev mentions to swh-environment/docker
Jan 27 2020, 5:14 PM
ardumont closed D2590: docs: Update swh-docker-dev mentions to swh-environment/docker.
Jan 27 2020, 5:14 PM
ardumont updated the diff for D2590: docs: Update swh-docker-dev mentions to swh-environment/docker.

Remove unneeded possessive form

Jan 27 2020, 5:14 PM
vlorentz committed rCDFJ4fadf12b9fac: Fix install of Cassandra keys. (authored by vlorentz).
Fix install of Cassandra keys.
Jan 27 2020, 5:12 PM
vlorentz closed D2593: Fix install of Cassandra keys..
Jan 27 2020, 5:12 PM
olasd accepted D2593: Fix install of Cassandra keys..
Jan 27 2020, 5:12 PM
vlorentz created D2593: Fix install of Cassandra keys..
Jan 27 2020, 5:11 PM
olasd accepted D2590: docs: Update swh-docker-dev mentions to swh-environment/docker.
Jan 27 2020, 5:09 PM
moranegg added a comment to T2203: Intrinsic metadata.

publiccode.yml : https://docs.italia.it/italia/developers-italia/publiccodeyml-en/en/master/index.html

Jan 27 2020, 5:04 PM · Intrinsic metadata, Roadmap 2020
swh-public-ci added a comment to D2582: Web API endpoint /known/.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/904/ for more details.

Jan 27 2020, 5:01 PM
DanSeraf updated the diff for D2582: Web API endpoint /known/.
utils.py: function to get a parsed persistent identifier
Jan 27 2020, 4:58 PM
vlorentz renamed T2244: Use swh-model for passing objects instead of dicts from Use swh-model for passing objects instead of JSON/msgpack to Use swh-model for passing objects instead of dicts.
Jan 27 2020, 4:51 PM · Core & foundations
vlorentz added a comment to T1365: Archive coverage metrics in prometheus.

https://grafana.softwareheritage.org/d/3SAW_JEmk/software-heritage-archive-counters

Jan 27 2020, 4:44 PM · Metrics/monitoring, Restricted Project
vlorentz closed T1365: Archive coverage metrics in prometheus, a subtask of T1364: Have production metrics in prometheus or kibana, as Resolved.
Jan 27 2020, 4:44 PM · Metrics/monitoring, Restricted Project
vlorentz closed T1365: Archive coverage metrics in prometheus as Resolved.
Jan 27 2020, 4:44 PM · Metrics/monitoring, Restricted Project
vlorentz updated the task description for T1475: Test more edge cases of metadata indexer mappings.
Jan 27 2020, 4:41 PM · Easy hack, Indexer
vlorentz renamed T1475: Test more edge cases of metadata indexer mappings from Add more tests for edge cases of indexer mappings to Test more edge cases of metadata indexer mappings.
Jan 27 2020, 4:39 PM · Easy hack, Indexer
swh-public-ci added a comment to D2566: Add Cassandra backend..

Build has FAILED

Jan 27 2020, 4:39 PM
vlorentz added a project to T1475: Test more edge cases of metadata indexer mappings: Easy hack.
Jan 27 2020, 4:35 PM · Easy hack, Indexer
vlorentz closed T1575: Kafka clients crash easily in docker-compose environment as Resolved.

Probably doesn't anymore since we moved to confluent-kafka

Jan 27 2020, 4:34 PM · Journal, Docker environment
vlorentz closed T1602: Analyze kakfa storage requirements as Resolved.
Jan 27 2020, 4:32 PM · Sprint 2019 03
vlorentz added a project to T1734: Create a Lister for launchpad.net: Lister.
Jan 27 2020, 4:31 PM · Lister, Archive coverage
vlorentz added a project to T1758: consistently document the configuration option of each module: Easy hack.
Jan 27 2020, 4:30 PM · Easy hack, Documentation
vlorentz closed T1947: Unsupported locale 'en_IN' in arrow lib as Wontfix.
Jan 27 2020, 4:27 PM · Scheduling utilities
vlorentz added a comment to D2582: Web API endpoint /known/.

Should i handle the error in parse_persistent_identifier while creating the pid?

Jan 27 2020, 4:25 PM
vlorentz added a project to T2028: swh-web should use a proper elasticsearch library to do its requests: Easy hack.
Jan 27 2020, 4:21 PM · Easy hack, Web app
vlorentz added a project to T2106: Migrate setuptools metadata from setup.py to setup.cfg: Easy hack.
Jan 27 2020, 4:20 PM · Easy hack, Development environment
ardumont created D2592: webapp0: Switch to cassandra storage.
Jan 27 2020, 4:04 PM
DanSeraf added a comment to D2582: Web API endpoint /known/.
  • returning 400 instead of 500 for invalid pid, type annotation on lookup_missing_hashes

This part is ok now, but cf my earlier comment: "You should add a small wrapper around parse_persistent_identifier to handle the error, that can be shared with resolve_swh_persistent_id."

Jan 27 2020, 4:00 PM
zack added inline comments to D2582: Web API endpoint /known/.
Jan 27 2020, 3:59 PM
ardumont accepted D2583: Implement content_update for the in-mem storage..
Jan 27 2020, 3:56 PM
vlorentz requested changes to D2582: Web API endpoint /known/.
  • returning 400 instead of 500 for invalid pid, type annotation on lookup_missing_hashes
Jan 27 2020, 3:44 PM
swh-public-ci added a comment to D2582: Web API endpoint /known/.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/903/ for more details.

Jan 27 2020, 3:18 PM
DanSeraf updated the diff for D2582: Web API endpoint /known/.
fix /known/ test expected values
Jan 27 2020, 3:15 PM
moranegg committed rMSLDa5fc3375ae7c: Update massive adoption slide with survey (authored by moranegg).
Update massive adoption slide with survey
Jan 27 2020, 3:15 PM
Harbormaster failed remote builds in B10297: Diff 9237 for D2582: Web API endpoint /known/!
Jan 27 2020, 3:10 PM
swh-public-ci added a comment to D2582: Web API endpoint /known/.

Build has FAILED

Jan 27 2020, 3:10 PM
DanSeraf updated the diff for D2582: Web API endpoint /known/.
minor changes, rebase
Jan 27 2020, 3:07 PM
zack requested changes to D2582: Web API endpoint /known/.
Jan 27 2020, 2:52 PM
swh-public-ci added a comment to D2582: Web API endpoint /known/.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/901/ for more details.

Jan 27 2020, 2:35 PM
DanSeraf updated the diff for D2582: Web API endpoint /known/.
  • returning 400 instead of 500 for invalid pid, type annotation on lookup_missing_hashes
  • rebase
Jan 27 2020, 2:32 PM
swh-public-ci added a comment to D2566: Add Cassandra backend..

Build has FAILED

Jan 27 2020, 1:59 PM
vlorentz requested changes to D2579: package.loader: Skip artifacts with no intrinsic metadata.

Could you add tests?

Jan 27 2020, 1:58 PM
vlorentz committed rCDFJ5622e45c2caf: Install Cassandra in base-buster image. (authored by vlorentz).
Install Cassandra in base-buster image.
Jan 27 2020, 1:48 PM
vlorentz closed D2591: Install Cassandra in base-buster image..
Jan 27 2020, 1:48 PM
vlorentz created P588 docker elasticsearch logs.
Jan 27 2020, 1:46 PM
moranegg committed rMSLD586d3e7637b0: Update SWH-ID diagram (authored by moranegg).
Update SWH-ID diagram
Jan 27 2020, 1:42 PM
swh-public-ci added a comment to D2586: Make db_transaction* remove db/cur from the signature..

Build is green
See https://jenkins.softwareheritage.org/job/DCORE/job/tox/397/ for more details.

Jan 27 2020, 1:38 PM
swh-public-ci added a comment to D2585: Add tests for db_transaction and db_transaction_generator..

Build is green
See https://jenkins.softwareheritage.org/job/DCORE/job/tox/396/ for more details.

Jan 27 2020, 1:36 PM
vlorentz updated the diff for D2586: Make db_transaction* remove db/cur from the signature..

rebase

Jan 27 2020, 1:35 PM
vlorentz updated the diff for D2585: Add tests for db_transaction and db_transaction_generator..

add missing dep

Jan 27 2020, 1:35 PM
ardumont accepted D2591: Install Cassandra in base-buster image..
Jan 27 2020, 1:34 PM
Harbormaster failed remote builds in B10293: Diff 9232 for D2585: Add tests for db_transaction and db_transaction_generator.!
Jan 27 2020, 1:30 PM
swh-public-ci added a comment to D2585: Add tests for db_transaction and db_transaction_generator..

Build has FAILED

Jan 27 2020, 1:30 PM
vlorentz updated the diff for D2585: Add tests for db_transaction and db_transaction_generator..

remove commit

Jan 27 2020, 1:28 PM
vlorentz created D2591: Install Cassandra in base-buster image..
Jan 27 2020, 1:26 PM
vlorentz accepted D2590: docs: Update swh-docker-dev mentions to swh-environment/docker.
Jan 27 2020, 1:22 PM
vlorentz added inline comments to D2587: Move Storage documentation and endpoint paths to a new StorageInterface class.
Jan 27 2020, 1:19 PM
vlorentz added a comment to D2586: Make db_transaction* remove db/cur from the signature..

As it's in core though, I'm unsure about the impacts so if someone else could review as well, that'd be neat.

Jan 27 2020, 1:18 PM
vlorentz added inline comments to D2583: Implement content_update for the in-mem storage..
Jan 27 2020, 1:15 PM
olasd closed T2034: Unbreak journal clients as Resolved.

I don't think I've actually seen this specific symptom in prod again, and if so only on workers that were hung up on something else already. We can reopen it if we notice it again.

Jan 27 2020, 1:10 PM · Journal
moranegg committed rMSLDb1b4528577cb: Update diagram in Pidapalooza slides (authored by moranegg).
Update diagram in Pidapalooza slides
Jan 27 2020, 12:40 PM
ardumont updated the diff for D2590: docs: Update swh-docker-dev mentions to swh-environment/docker.

Adapt according to review

Jan 27 2020, 12:34 PM
ardumont added inline comments to D2590: docs: Update swh-docker-dev mentions to swh-environment/docker.
Jan 27 2020, 12:31 PM
olasd requested changes to D2590: docs: Update swh-docker-dev mentions to swh-environment/docker.
Jan 27 2020, 12:29 PM
vlorentz requested changes to D2582: Web API endpoint /known/.

You shouldn't remove the handling of ValidationError exceptions; because it raises an error 500 Internal Server Error, which should only be returned if there is a bug in the server code.

Jan 27 2020, 12:27 PM
ardumont added a comment to T2183: Switch webapp0 to use swh-search instead of postgresql search..

It's deployed btw.

Jan 27 2020, 12:01 PM · Archive search, Storage manager
ardumont created D2590: docs: Update swh-docker-dev mentions to swh-environment/docker.
Jan 27 2020, 11:55 AM
swh-public-ci added a comment to D2582: Web API endpoint /known/.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/900/ for more details.

Jan 27 2020, 11:54 AM
DanSeraf updated the diff for D2582: Web API endpoint /known/.
  • /known/ endpoint tests
  • test_service: testing missing hashes lookup with present pids
Jan 27 2020, 11:51 AM
ardumont updated the summary of D2566: Add Cassandra backend..
Jan 27 2020, 11:41 AM
ardumont added a comment to D2585: Add tests for db_transaction and db_transaction_generator..

looks like D2585 and D2586 are entangled together.

Jan 27 2020, 11:12 AM
ardumont accepted D2586: Make db_transaction* remove db/cur from the signature..

I agree with the reasoning in the description.

Jan 27 2020, 11:11 AM
ardumont added inline comments to D2587: Move Storage documentation and endpoint paths to a new StorageInterface class.
Jan 27 2020, 11:10 AM
ardumont updated subscribers of D2583: Implement content_update for the in-mem storage..
Jan 27 2020, 10:00 AM
ardumont accepted D2587: Move Storage documentation and endpoint paths to a new StorageInterface class.
Jan 27 2020, 9:47 AM
ardumont added a comment to D2587: Move Storage documentation and endpoint paths to a new StorageInterface class.

I started the review and asked for docstring updates.
I was gonna ask to add types as well.
But i stopped.

Jan 27 2020, 9:45 AM
ardumont accepted D2588: Rename in_memory.Storage to in_memory.InMemoryStorage..
Jan 27 2020, 9:40 AM
ardumont accepted D2584: Remove cur/db arguments from the in-mem storage..
Jan 27 2020, 9:00 AM

Jan 26 2020

zack added a comment to T2251: Update publication page.

thanks Morane!

Jan 26 2020, 5:20 PM · Website
moranegg committed rMSLDe1a7a9eb0ab2: Add references slide and other fixes (authored by moranegg).
Add references slide and other fixes
Jan 26 2020, 3:59 AM
moranegg triaged T2251: Update publication page as Normal priority.
Jan 26 2020, 2:05 AM · Website

Jan 24 2020

Harbormaster failed remote builds in B10288: Diff 9227 for D2582: Web API endpoint /known/!
Jan 24 2020, 6:52 PM
swh-public-ci added a comment to D2582: Web API endpoint /known/.

Build has FAILED

Jan 24 2020, 6:52 PM
DanSeraf updated the diff for D2582: Web API endpoint /known/.
  • known moved to identifiers view, parse pids before grouping them
  • test present pids
Jan 24 2020, 6:49 PM
swh-public-ci added a comment to D2589: Add return type to get_storage..

Build has FAILED

Jan 24 2020, 6:14 PM