Page MenuHomeSoftware Heritage
Feed All Stories

Apr 8 2020

olasd created D2979: Allow overriding broker settings in the kafka server fixture.
Apr 8 2020, 1:36 PM
Harbormaster failed to build B11674: rDSTObe954f2fde39: buffer: filter out duplicate objects. for rDSTObe954f2fde39: buffer: filter out duplicate objects.!
Apr 8 2020, 1:01 PM
vlorentz closed D2978: buffer: filter out duplicate objects..

be954f2fde39cefa80baeda01381a29582772684

Apr 8 2020, 1:01 PM
vlorentz committed rDSTObe954f2fde39: buffer: filter out duplicate objects. (authored by vlorentz).
buffer: filter out duplicate objects.
Apr 8 2020, 1:01 PM
ardumont added inline comments to D2974: Added LaunchpadLister.
Apr 8 2020, 12:42 PM
anlambert committed rSPSITE49ed9869b1b6: webapp: Add keycloak configuration for production and staging (authored by anlambert).
webapp: Add keycloak configuration for production and staging
Apr 8 2020, 12:34 PM
anlambert closed D2975: webapp: Add keycloak configuration for production and staging.
Apr 8 2020, 12:34 PM
ardumont accepted D2975: webapp: Add keycloak configuration for production and staging.

lgtm

Apr 8 2020, 12:31 PM
ardumont added a comment to D2974: Added LaunchpadLister.

Also, add yourself to the CONTRIBUTORS file (at the top of the repository ;)
(I think it's alphabetic order on the firstname there.)

Apr 8 2020, 12:31 PM
anlambert updated the diff for D2975: webapp: Add keycloak configuration for production and staging.

Update: Address @ardumont comment.

Apr 8 2020, 12:26 PM
anlambert added inline comments to D2975: webapp: Add keycloak configuration for production and staging.
Apr 8 2020, 12:24 PM
ardumont added a comment to D2974: Added LaunchpadLister.

Remains mostly nitpicks to fix (type, indentation, docstring updates...)

Apr 8 2020, 12:24 PM
anlambert committed rDWAPPS2251dea545d8: tox: Remove django 1.11 build (authored by anlambert).
tox: Remove django 1.11 build
Apr 8 2020, 12:23 PM
anlambert closed D2976: tox: Remove django 1.11 build.
Apr 8 2020, 12:23 PM
ardumont accepted D2978: buffer: filter out duplicate objects..
Apr 8 2020, 12:18 PM
Harbormaster failed remote builds in B11668: Diff 10581 for D2972: test: update storage tests to (future) swh.journal 0.0.30!
Apr 8 2020, 12:12 PM
swh-public-ci added a comment to D2972: test: update storage tests to (future) swh.journal 0.0.30.

Build has FAILED

Apr 8 2020, 12:12 PM
Harbormaster failed remote builds in B11667: Diff 10580 for D2971: Make Storage.origin_visit_update() add an OriginVisit model entity in the journal!
Apr 8 2020, 12:11 PM
swh-public-ci added a comment to D2971: Make Storage.origin_visit_update() add an OriginVisit model entity in the journal.

Build has FAILED

Apr 8 2020, 12:11 PM
Harbormaster failed remote builds in B11666: Diff 10579 for D2970: Make swh/storage/storage.py flake8 compliant!
Apr 8 2020, 12:11 PM
swh-public-ci added a comment to D2970: Make swh/storage/storage.py flake8 compliant.

Build has FAILED

Apr 8 2020, 12:11 PM
Harbormaster failed remote builds in B11665: Diff 10578 for D2969: Add a setup.cfg file to configure flake8 for black compatibility!
Apr 8 2020, 12:10 PM
swh-public-ci added a comment to D2969: Add a setup.cfg file to configure flake8 for black compatibility.

Build has FAILED

Apr 8 2020, 12:10 PM
swh-public-ci added a comment to D2978: buffer: filter out duplicate objects..

Build is green

Apr 8 2020, 12:09 PM
Harbormaster failed to build B11671: rDSTOc51139ef9622: Make Storage.origin_visit_update() add an OriginVisit model entity in the… for rDSTOc51139ef9622: Make Storage.origin_visit_update() add an OriginVisit model entity in the…!
Apr 8 2020, 12:08 PM
Harbormaster failed to build B11670: rDSTObf48cfe98b33: Make swh/storage/storage.py flake8 compliant for rDSTObf48cfe98b33: Make swh/storage/storage.py flake8 compliant!
Apr 8 2020, 12:08 PM
douardda committed rDSTOc51139ef9622: Make Storage.origin_visit_update() add an OriginVisit model entity in the… (authored by douardda).
Make Storage.origin_visit_update() add an OriginVisit model entity in the…
Apr 8 2020, 12:08 PM
douardda closed D2971: Make Storage.origin_visit_update() add an OriginVisit model entity in the journal.
Apr 8 2020, 12:08 PM
Harbormaster failed to build B11669: rDSTOfbb51aa0cfdd: Add a setup.cfg file to configure flake8 for black compatibility for rDSTOfbb51aa0cfdd: Add a setup.cfg file to configure flake8 for black compatibility!
Apr 8 2020, 12:07 PM
douardda committed rDSTObf48cfe98b33: Make swh/storage/storage.py flake8 compliant (authored by douardda).
Make swh/storage/storage.py flake8 compliant
Apr 8 2020, 12:06 PM
douardda committed rDSTOfbb51aa0cfdd: Add a setup.cfg file to configure flake8 for black compatibility (authored by douardda).
Add a setup.cfg file to configure flake8 for black compatibility
Apr 8 2020, 12:06 PM
douardda closed D2970: Make swh/storage/storage.py flake8 compliant.
Apr 8 2020, 12:06 PM
douardda closed D2969: Add a setup.cfg file to configure flake8 for black compatibility.
Apr 8 2020, 12:06 PM
douardda updated the diff for D2972: test: update storage tests to (future) swh.journal 0.0.30.

rebase

Apr 8 2020, 12:06 PM
douardda updated the diff for D2971: Make Storage.origin_visit_update() add an OriginVisit model entity in the journal.

rebase

Apr 8 2020, 12:05 PM
douardda updated the diff for D2970: Make swh/storage/storage.py flake8 compliant.

rebase

Apr 8 2020, 12:04 PM
douardda updated the diff for D2969: Add a setup.cfg file to configure flake8 for black compatibility.

rebase

Apr 8 2020, 12:04 PM
vlorentz created D2978: buffer: filter out duplicate objects..
Apr 8 2020, 12:03 PM
DanSeraf committed rDMOD5d6883b26594: from_disk: path parameter to dir_filter functions (authored by DanSeraf).
from_disk: path parameter to dir_filter functions
Apr 8 2020, 11:58 AM
DanSeraf closed D2960: swh-model: exclude path support for the function from_disk in Directory.
Apr 8 2020, 11:58 AM
DanSeraf added a comment to D2960: swh-model: exclude path support for the function from_disk in Directory.

Looks good to me.

Don't you want to implement your callable here (the one which actually exclude stuff)?

Because now, the diff title is no longer in sync with the code ;)

Or do you plan to add it in swh-scanner?

Apr 8 2020, 11:58 AM
ardumont accepted D2960: swh-model: exclude path support for the function from_disk in Directory.
Apr 8 2020, 11:44 AM
swh-public-ci added a comment to D2966: storage.buffer: Add a new clear_buffers operation for the buffer proxy.

Build is green

Apr 8 2020, 11:39 AM
ardumont added a comment to D2960: swh-model: exclude path support for the function from_disk in Directory.

Looks good to me.

Apr 8 2020, 11:36 AM
swh-public-ci added a comment to D2960: swh-model: exclude path support for the function from_disk in Directory.

Build is green

Apr 8 2020, 11:35 AM
ardumont updated the diff for D2966: storage.buffer: Add a new clear_buffers operation for the buffer proxy.
  • storage*: Add clear_buffers operation for proxy storages (including interface, rpc/server)
  • test_filter: Extract the filter storage into a fixture
Apr 8 2020, 11:34 AM
DanSeraf updated the diff for D2960: swh-model: exclude path support for the function from_disk in Directory.

requested changes

Apr 8 2020, 11:33 AM
ardumont added a comment to D2966: storage.buffer: Add a new clear_buffers operation for the buffer proxy.

You really need to add the clear_buffers method to interface.py as well. The method needs to be available on the rpc client/server, because we're recursively calling it on all backends.

Apr 8 2020, 11:25 AM
ardumont added a comment to D2966: storage.buffer: Add a new clear_buffers operation for the buffer proxy.

The get_storage_with_filter_config refactoring could have been a new storage fixture (local to this module) instead

Apr 8 2020, 11:16 AM
olasd requested changes to D2966: storage.buffer: Add a new clear_buffers operation for the buffer proxy.

The get_storage_with_filter_config refactoring could have been a new storage fixture (local to this module) instead (and it should be in a separate commit).

Apr 8 2020, 11:15 AM
ardumont updated subscribers of D2973: package.loader: Insert consistently data to storage, clear buffer proxy state in case of error when skipping artifact.
Apr 8 2020, 11:08 AM
ardumont added a comment to T2332: Analyze hash collisions.

An interesting experiment, disabling the proxy buffer storage in the loader nixguix configuration.
And the number of hashcollision dropped to 0 (no new event for that loader since yesterday around 6pm our time).

Apr 8 2020, 10:58 AM · Object storage, Storage manager
vlorentz added a revision to T2332: Analyze hash collisions: D2977: Prevent erroneous HashCollisions by using the same ctime for all rows..
Apr 8 2020, 10:53 AM · Object storage, Storage manager
vlorentz added a task to D2977: Prevent erroneous HashCollisions by using the same ctime for all rows.: T2332: Analyze hash collisions.
Apr 8 2020, 10:53 AM
vlorentz committed rDSTO8e8577e8d491: Prevent erroneous HashCollisions by using the same ctime for all rows. (authored by vlorentz).
Prevent erroneous HashCollisions by using the same ctime for all rows.
Apr 8 2020, 10:44 AM
vlorentz closed D2977: Prevent erroneous HashCollisions by using the same ctime for all rows..
Apr 8 2020, 10:44 AM
swh-public-ci added a comment to D2977: Prevent erroneous HashCollisions by using the same ctime for all rows..

Build is green

Apr 8 2020, 10:38 AM
ardumont updated the summary of D2977: Prevent erroneous HashCollisions by using the same ctime for all rows..
Apr 8 2020, 10:36 AM
olasd accepted D2977: Prevent erroneous HashCollisions by using the same ctime for all rows..
Apr 8 2020, 10:36 AM
vlorentz created D2977: Prevent erroneous HashCollisions by using the same ctime for all rows..
Apr 8 2020, 10:32 AM
ardumont requested changes to D2960: swh-model: exclude path support for the function from_disk in Directory.

Getting there, see my previous comments ;)

Apr 8 2020, 10:26 AM
ardumont added inline comments to D2975: webapp: Add keycloak configuration for production and staging.
Apr 8 2020, 10:04 AM
ardumont added inline comments to D2960: swh-model: exclude path support for the function from_disk in Directory.
Apr 8 2020, 9:58 AM
ardumont added a comment to D2960: swh-model: exclude path support for the function from_disk in Directory.

I did not reply as val did (and i agreed with him ;)

Apr 8 2020, 9:56 AM
Harbormaster failed remote builds in B11658: Diff 10571 for D2960: swh-model: exclude path support for the function from_disk in Directory!
Apr 8 2020, 9:48 AM
swh-public-ci added a comment to D2960: swh-model: exclude path support for the function from_disk in Directory.

Build has FAILED

Apr 8 2020, 9:48 AM
DanSeraf updated the diff for D2960: swh-model: exclude path support for the function from_disk in Directory.

requested changes

Apr 8 2020, 9:47 AM
ardumont accepted D2976: tox: Remove django 1.11 build.
Apr 8 2020, 9:40 AM

Apr 7 2020

swh-public-ci added a comment to D2976: tox: Remove django 1.11 build.

Build is green

Apr 7 2020, 7:03 PM
anlambert created D2976: tox: Remove django 1.11 build.
Apr 7 2020, 6:55 PM
anlambert closed T2303: Upgrade to Django 2.x in production as Resolved.

Closing this as Django has been successfully upgraded to version 2.2 in production.

Apr 7 2020, 6:43 PM · System administration, Web app
anlambert created D2975: webapp: Add keycloak configuration for production and staging.
Apr 7 2020, 6:41 PM
ardumont updated the summary of D2973: package.loader: Insert consistently data to storage, clear buffer proxy state in case of error when skipping artifact.
Apr 7 2020, 6:22 PM
olasd committed rSPGUNI53aca02c1d81: Always restart crashed gunicorns after 10 seconds (authored by olasd).
Always restart crashed gunicorns after 10 seconds
Apr 7 2020, 6:20 PM
ardumont added a comment to D2974: Added LaunchpadLister.
cd swh-lister
git checkout master
git pull
arc patch D2974
tox -r -- -x -s
Apr 7 2020, 5:48 PM
ardumont added inline comments to D2973: package.loader: Insert consistently data to storage, clear buffer proxy state in case of error when skipping artifact.
Apr 7 2020, 5:38 PM
ardumont added a comment to D2974: Added LaunchpadLister.

@ardumont It seems the build error did not go away

Apr 7 2020, 5:38 PM
legau added a comment to D2974: Added LaunchpadLister.

@ardumont It seems the build error did not go away

Apr 7 2020, 5:37 PM
ardumont added inline comments to D2973: package.loader: Insert consistently data to storage, clear buffer proxy state in case of error when skipping artifact.
Apr 7 2020, 5:36 PM
vlorentz added inline comments to D2973: package.loader: Insert consistently data to storage, clear buffer proxy state in case of error when skipping artifact.
Apr 7 2020, 5:35 PM
legau abandoned D2799: Added LaunchpadLister.
Apr 7 2020, 5:34 PM
Harbormaster failed remote builds in B11655: Diff 10568 for D2974: Added LaunchpadLister!
Apr 7 2020, 5:34 PM
swh-public-ci added a comment to D2974: Added LaunchpadLister.

Build has FAILED

Apr 7 2020, 5:34 PM
Harbormaster failed remote builds in B11654: Diff 10567 for D2973: package.loader: Insert consistently data to storage, clear buffer proxy state in case of error when skipping artifact!
Apr 7 2020, 5:34 PM
swh-public-ci added a comment to D2973: package.loader: Insert consistently data to storage, clear buffer proxy state in case of error when skipping artifact.

Build has FAILED

Apr 7 2020, 5:34 PM
legau created D2974: Added LaunchpadLister.
Apr 7 2020, 5:34 PM
ardumont created D2973: package.loader: Insert consistently data to storage, clear buffer proxy state in case of error when skipping artifact.
Apr 7 2020, 5:33 PM
anlambert committed rDENV5bdffaf03067: docker/conftest: Increase wait-for-it timeouts (authored by anlambert).
docker/conftest: Increase wait-for-it timeouts
Apr 7 2020, 5:28 PM
anlambert closed D2967: docker/conftest: Increase wait-for-it timeouts.
Apr 7 2020, 5:28 PM
anlambert updated the diff for D2967: docker/conftest: Increase wait-for-it timeouts.

Update: use pytest fixture instead of private module variable

Apr 7 2020, 5:25 PM
anlambert added a comment to D2967: docker/conftest: Increase wait-for-it timeouts.

why calling it _wfi_timeout (with the leading _)?

Also, in the glorious pytest way, this could be implemented as a fixture!

But yeah.

Apr 7 2020, 4:53 PM
douardda accepted D2967: docker/conftest: Increase wait-for-it timeouts.

why calling it _wfi_timeout (with the leading _)?

Apr 7 2020, 4:51 PM
vlorentz added a comment to D2968: Fix the inmemory journal writer.

Could you improve the first line of commit message before merging? (eg. Remove useless copy)

Apr 7 2020, 4:49 PM
Harbormaster failed remote builds in B11652: Diff 10564 for D2972: test: update storage tests to (future) swh.journal 0.0.30!
Apr 7 2020, 4:47 PM
swh-public-ci added a comment to D2972: test: update storage tests to (future) swh.journal 0.0.30.

Build has FAILED

Apr 7 2020, 4:47 PM
Harbormaster failed remote builds in B11651: Diff 10563 for D2971: Make Storage.origin_visit_update() add an OriginVisit model entity in the journal!
Apr 7 2020, 4:46 PM
swh-public-ci added a comment to D2971: Make Storage.origin_visit_update() add an OriginVisit model entity in the journal.

Build has FAILED

Apr 7 2020, 4:46 PM
anlambert accepted D2971: Make Storage.origin_visit_update() add an OriginVisit model entity in the journal.
Apr 7 2020, 4:46 PM
douardda created D2972: test: update storage tests to (future) swh.journal 0.0.30.
Apr 7 2020, 4:46 PM
Harbormaster failed remote builds in B11650: Diff 10562 for D2970: Make swh/storage/storage.py flake8 compliant!
Apr 7 2020, 4:46 PM