Page MenuHomeSoftware Heritage
Feed Advanced Search

Apr 15 2022

douardda updated the diff for D7531: Regularly check for EOF in Client.process() while waiting for messages.

typo (thx ardumont)

Apr 15 2022, 12:41 PM
douardda added inline comments to D7531: Regularly check for EOF in Client.process() while waiting for messages.
Apr 15 2022, 12:38 PM
douardda requested review of D7589: Add a --margin option to the `swh dataset graph export` command.
Apr 15 2022, 12:31 PM
douardda requested review of D7588: Add a --type option to the `swh dataset graph export` command.
Apr 15 2022, 12:31 PM

Apr 14 2022

douardda added a comment to D7580: sentry: always override init settings with the environment variables.

lgtm, maybe add a comment in test_cli.py to justify the presence of forked over there

Apr 14 2022, 3:35 PM
douardda accepted D7570: Add support for disabling logging integration in sentry.
Apr 14 2022, 12:19 PM
douardda committed rDSTOf136559bbdd7: User logger everywhere in tenacious.py (authored by douardda).
User logger everywhere in tenacious.py
Apr 14 2022, 12:14 PM

Apr 13 2022

douardda requested review of D7570: Add support for disabling logging integration in sentry.
Apr 13 2022, 4:23 PM

Apr 12 2022

douardda accepted D7558: journalprocessor: save final offsets to a text file.
Apr 12 2022, 5:04 PM
douardda claimed T3087: Implement support for takedown notices (infra, admin tools, workflow).
Apr 12 2022, 12:34 PM · Roadmap 2022, meta-task, Roadmap 2021, Web app
douardda accepted D7552: origin_get_with_statuses: Fix case when fetched visits list is empty.
Apr 12 2022, 12:26 PM
douardda accepted D7517: Add support for multipart/mixed + better fallback for multipart/*.

lgtm

Apr 12 2022, 12:24 PM

Apr 11 2022

douardda closed D7536: Add a SWH_MODEL_OBJECT_TYPES map {cls.objet_type: cls} in model.py.
Apr 11 2022, 12:32 PM
douardda committed rDMOD85f36f84347a: Add a SWH_MODEL_OBJECT_TYPES map {cls.objet_type: cls} in model.py (authored by douardda).
Add a SWH_MODEL_OBJECT_TYPES map {cls.objet_type: cls} in model.py
Apr 11 2022, 12:32 PM
douardda updated the diff for D7536: Add a SWH_MODEL_OBJECT_TYPES map {cls.objet_type: cls} in model.py.

rebase

Apr 11 2022, 12:29 PM
douardda updated the diff for D7536: Add a SWH_MODEL_OBJECT_TYPES map {cls.objet_type: cls} in model.py.

Add type annotation for SWH_MODEL_OBJECT_TYPES

Apr 11 2022, 10:25 AM
douardda added inline comments to D7536: Add a SWH_MODEL_OBJECT_TYPES map {cls.objet_type: cls} in model.py.
Apr 11 2022, 10:17 AM

Apr 8 2022

douardda added inline comments to D7536: Add a SWH_MODEL_OBJECT_TYPES map {cls.objet_type: cls} in model.py.
Apr 8 2022, 5:11 PM
douardda added inline comments to D7536: Add a SWH_MODEL_OBJECT_TYPES map {cls.objet_type: cls} in model.py.
Apr 8 2022, 5:08 PM
douardda added inline comments to D7536: Add a SWH_MODEL_OBJECT_TYPES map {cls.objet_type: cls} in model.py.
Apr 8 2022, 5:06 PM
douardda requested review of D7531: Regularly check for EOF in Client.process() while waiting for messages.
Apr 8 2022, 4:57 PM
douardda requested review of D7536: Add a SWH_MODEL_OBJECT_TYPES map {cls.objet_type: cls} in model.py.
Apr 8 2022, 12:48 PM
douardda requested review of D7520: Add a ORC file loading function.
Apr 8 2022, 12:15 PM
douardda closed D7519: Reduce cli's loading time by moving import statements in commands.
Apr 8 2022, 11:48 AM
douardda committed rDDATASET18325cc8e78e: Reduce cli's loading time by moving import statements in commands (authored by douardda).
Reduce cli's loading time by moving import statements in commands
Apr 8 2022, 11:48 AM
douardda added inline comments to D7519: Reduce cli's loading time by moving import statements in commands.
Apr 8 2022, 11:36 AM
douardda accepted D7518: Add support for recursive multipart messages.

LGTM

Apr 8 2022, 11:34 AM

Apr 7 2022

douardda requested review of D7519: Reduce cli's loading time by moving import statements in commands.
Apr 7 2022, 12:05 PM
douardda closed D7465: Add support for blob in content export.
Apr 7 2022, 12:02 PM
douardda committed rDDATASET9d97f0c0826d: Add support for blob in content export (authored by douardda).
Add support for blob in content export
Apr 7 2022, 12:02 PM
douardda updated the diff for D7465: Add support for blob in content export.

rebase

Apr 7 2022, 11:24 AM

Apr 6 2022

douardda accepted D7506: Add support for HAL-ID as identifier.
Apr 6 2022, 3:23 PM
douardda accepted D7433: Add Fixer class, which re-loads corrupt objects from origins.

As I said, I'm not completely enthusiastic by this code (especially the lack of flexibility and versatility to provision for other origin sources than git) but it's a nice start at least.
Thanks

Apr 6 2022, 3:02 PM
douardda closed D7493: docker: make yarn run in the local copy of the swh-web source code.
Apr 6 2022, 2:33 PM
douardda committed rDENVb4ddd00b694a: docker: make yarn run in the local copy of the swh-web source code (authored by douardda).
docker: make yarn run in the local copy of the swh-web source code
Apr 6 2022, 2:33 PM
douardda closed D7039: Update the debian local package building section.
Apr 6 2022, 2:24 PM
douardda committed rDDOC94533948188e: Update the debian local package building section (authored by douardda).
Update the debian local package building section
Apr 6 2022, 2:24 PM
douardda updated the diff for D7039: Update the debian local package building section.

rebase

Apr 6 2022, 2:24 PM
douardda updated the diff for D7493: docker: make yarn run in the local copy of the swh-web source code.

adapt code according to anlambert's suggestion

Apr 6 2022, 1:43 PM
douardda added a comment to D7433: Add Fixer class, which re-loads corrupt objects from origins.

I think I'd prefer not to have 3 versions of almost the same function (corrupt_object_(get|grab_xxx) ).

Apr 6 2022, 10:51 AM

Apr 5 2022

douardda added a comment to D7506: Add support for HAL-ID as identifier.

lgtm but I'm not sure to understand what's at stake here.

Apr 5 2022, 5:01 PM
douardda updated the diff for D7039: Update the debian local package building section.

specify aptitude is to be used only when needed (sic)

Apr 5 2022, 4:57 PM
douardda added a comment to D7039: Update the debian local package building section.

Ping, what do we do with this diff?

Apr 5 2022, 4:55 PM
douardda added a comment to D7039: Update the debian local package building section.
Apr 5 2022, 4:54 PM
douardda updated the diff for D7258: Update the mirror operation docker manual.

typos (thx ardumont) + add a warning about db migration

Apr 5 2022, 4:25 PM
douardda added a comment to D7465: Add support for blob in content export.

LGTM, but we should eventually replace self.objstorage.get() with self.objstorage.get_batch(), it is much faster with the Azure backend (and hopefully more backends in the future, inc. Winery!)

Apr 5 2022, 4:11 PM

Apr 4 2022

douardda closed D7492: docker: add librdkakfa-dev so confluent-kafka in pip installable on non-x86 platforms.
Apr 4 2022, 5:06 PM
douardda committed rDENV995e90739c3d: docker: add librdkakfa-dev so confluent-kafka in pip installable on non-x86… (authored by douardda).
docker: add librdkakfa-dev so confluent-kafka in pip installable on non-x86…
Apr 4 2022, 5:06 PM
douardda updated the diff for D7493: docker: make yarn run in the local copy of the swh-web source code.

fix indentation and do only call 'yarn build-dev'

Apr 4 2022, 5:06 PM
douardda closed D7473: Make postgresql's Storage client options configurable from config.
Apr 4 2022, 4:59 PM
douardda committed rDSTOc6dc5cd3b58a: Make postgresql's Storage client options configurable from config (authored by douardda).
Make postgresql's Storage client options configurable from config
Apr 4 2022, 4:59 PM
douardda updated the diff for D7473: Make postgresql's Storage client options configurable from config.

better docstring (thx olasd)

Apr 4 2022, 4:44 PM
douardda updated the diff for D7473: Make postgresql's Storage client options configurable from config.

docstring

Apr 4 2022, 4:08 PM
douardda added a comment to D7473: Make postgresql's Storage client options configurable from config.

Please add a docstring; it's unclear this is used by swh-core otherwise

Apr 4 2022, 3:41 PM
douardda closed D7472: Make db_transaction's client_options configurable at run time.
Apr 4 2022, 3:38 PM
douardda committed rDCORE2a2615357b07: Make db_transaction's client_options configurable at run time (authored by douardda).
Make db_transaction's client_options configurable at run time
Apr 4 2022, 3:38 PM
douardda added a comment to D7472: Make db_transaction's client_options configurable at run time.
In D7472#195801, @olasd wrote:

The duplication between db_transaction and db_transaction_generator looks a bit silly, but I'm not sure there's much to do about it, so this lgtm, thanks.

Apr 4 2022, 3:37 PM
douardda added inline comments to D7493: docker: make yarn run in the local copy of the swh-web source code.
Apr 4 2022, 3:33 PM
douardda requested review of D7493: docker: make yarn run in the local copy of the swh-web source code.
Apr 4 2022, 2:06 PM
douardda updated the diff for D7492: docker: add librdkakfa-dev so confluent-kafka in pip installable on non-x86 platforms.

wrong head...

Apr 4 2022, 2:06 PM
douardda requested review of D7492: docker: add librdkakfa-dev so confluent-kafka in pip installable on non-x86 platforms.
Apr 4 2022, 2:05 PM

Apr 1 2022

douardda accepted D7483: Ensure that tests run with the C.UTF-8 locale.
Apr 1 2022, 11:36 AM
douardda published D7483: Ensure that tests run with the C.UTF-8 locale for review.
Apr 1 2022, 11:36 AM

Mar 31 2022

douardda accepted D7478: Clean up `model` removing old unused logic.
Mar 31 2022, 3:54 PM
douardda updated the diff for D7465: Add support for blob in content export.

rebase

Mar 31 2022, 2:46 PM
douardda closed D7476: Keep each ORC table in a dedicated directory.
Mar 31 2022, 2:45 PM
douardda committed rDDATASET76eba6595bac: Keep each ORC table in a dedicated directory (authored by douardda).
Keep each ORC table in a dedicated directory
Mar 31 2022, 2:45 PM
douardda requested review of D7476: Keep each ORC table in a dedicated directory.
Mar 31 2022, 12:32 PM

Mar 30 2022

douardda updated the diff for D7472: Make db_transaction's client_options configurable at run time.

doubleticks

Mar 30 2022, 4:15 PM
douardda updated the diff for D7472: Make db_transaction's client_options configurable at run time.

Merge all 4 added tests in a single paramtrized one

Mar 30 2022, 3:56 PM
douardda added inline comments to D7472: Make db_transaction's client_options configurable at run time.
Mar 30 2022, 3:31 PM
douardda requested review of D7473: Make postgresql's Storage client options configurable from config.
Mar 30 2022, 3:00 PM
douardda requested review of D7472: Make db_transaction's client_options configurable at run time.
Mar 30 2022, 2:38 PM
douardda created P1322 (An Untitled Masterwork).
Mar 30 2022, 1:45 PM
douardda closed D7463: Delay the unsubscribe to the end of handle_messages.
Mar 30 2022, 11:34 AM
douardda closed D7464: Improve the progress reporting look and feel.
Mar 30 2022, 11:34 AM
douardda committed rDDATASET58b9f92e1d4d: Improve the progress reporting look and feel (authored by douardda).
Improve the progress reporting look and feel
Mar 30 2022, 11:34 AM
douardda committed rDDATASETbe3c5da2bb44: Delay the unsubscribe to the end of handle_messages (authored by douardda).
Delay the unsubscribe to the end of handle_messages
Mar 30 2022, 11:34 AM
douardda closed D7461: Add support for limited row numbers in ORC files.
Mar 30 2022, 11:34 AM
douardda closed D7462: Move exporter config entries in dedicated sections.
Mar 30 2022, 11:34 AM
douardda committed rDDATASET3df08fd71759: Add support for limited row numbers in ORC files (authored by douardda).
Add support for limited row numbers in ORC files
Mar 30 2022, 11:34 AM
douardda committed rDDATASETe01daba4d601: Move exporter config entries in dedicated sections (authored by douardda).
Move exporter config entries in dedicated sections
Mar 30 2022, 11:34 AM
douardda updated the diff for D7465: Add support for blob in content export.

rebase

Mar 30 2022, 11:23 AM
douardda updated the diff for D7464: Improve the progress reporting look and feel.

rebase

Mar 30 2022, 11:22 AM
douardda updated the diff for D7463: Delay the unsubscribe to the end of handle_messages.

Add a comment

Mar 30 2022, 11:22 AM
douardda updated the diff for D7465: Add support for blob in content export.

rebase

Mar 30 2022, 11:09 AM
douardda updated the diff for D7464: Improve the progress reporting look and feel.

rebase

Mar 30 2022, 11:09 AM
douardda updated the diff for D7463: Delay the unsubscribe to the end of handle_messages.

rebase

Mar 30 2022, 11:09 AM
douardda updated the diff for D7462: Move exporter config entries in dedicated sections.

rebase

Mar 30 2022, 11:09 AM
douardda updated the diff for D7461: Add support for limited row numbers in ORC files.

make it py37 compliant

Mar 30 2022, 11:08 AM
douardda updated the diff for D7465: Add support for blob in content export.

rebase

Mar 30 2022, 10:45 AM
douardda updated the diff for D7464: Improve the progress reporting look and feel.

rebase

Mar 30 2022, 10:44 AM
douardda updated the diff for D7463: Delay the unsubscribe to the end of handle_messages.

rebase

Mar 30 2022, 10:44 AM
douardda updated the diff for D7462: Move exporter config entries in dedicated sections.

rebase

Mar 30 2022, 10:44 AM
douardda updated the diff for D7461: Add support for limited row numbers in ORC files.

check for max_rows validity at ORCExporter class instanciation + copyrights

Mar 30 2022, 10:43 AM
douardda added inline comments to D7461: Add support for limited row numbers in ORC files.
Mar 30 2022, 10:18 AM
douardda requested review of D7465: Add support for blob in content export.
Mar 30 2022, 10:04 AM
douardda added inline comments to D7460: docker/build_image.sh: Add script to build/ push to docker registry.
Mar 30 2022, 9:50 AM

Mar 29 2022

douardda added inline comments to D7461: Add support for limited row numbers in ORC files.
Mar 29 2022, 6:22 PM
douardda requested review of D7464: Improve the progress reporting look and feel.
Mar 29 2022, 5:47 PM