Page MenuHomeSoftware Heritage
Feed Advanced Search

Mar 8 2019

douardda updated the diff for D1221: readme: integrate the docker-based development setup from the main's doc.

typo (missing t in url)

Mar 8 2019, 12:05 PM
douardda added inline comments to D1221: readme: integrate the docker-based development setup from the main's doc.
Mar 8 2019, 12:03 PM
douardda requested changes to D1196: Add an 'swh-scheduler task schedule_origins' command to run a task on all origins..
Mar 8 2019, 11:52 AM
douardda requested changes to D1184: swh.deposit.cli: Transform create_user into a main cli (+ docs updates).

I'm globally OK with this, but please add support for a --config-file cmd line option in parallel with the environment variable support, as other swh-xxx commands do.

Mar 8 2019, 11:36 AM
douardda committed rDDOC9542708dd478: Refactor the getting started guides (authored by douardda).
Refactor the getting started guides
Mar 8 2019, 11:17 AM
douardda closed D1220: Refactor the getting started guides.
Mar 8 2019, 11:17 AM
douardda committed rDSCHb0aea8c0fd05: tests: enforce LC_ALL since test_cli depends on it (authored by douardda).
tests: enforce LC_ALL since test_cli depends on it
Mar 8 2019, 10:45 AM
douardda closed D1223: tests: enforce LC_ALL since test_cli depends on it.
Mar 8 2019, 10:45 AM
douardda accepted D1230: [docker-compose] swh-web: Install pandoc used to render README.rst file to HTML.
Mar 8 2019, 10:42 AM
douardda accepted D1229: [docker-compose] gunicorn: Use same timeout value as in production.
Mar 8 2019, 10:41 AM
douardda accepted D1228: [docker-compose] Ensure Kafka server is up before starting related services.
Mar 8 2019, 10:40 AM
douardda accepted D1227: [docker-compose] Ensure RabbitMQ server is up before starting related services.

nice one, thanks!

Mar 8 2019, 10:39 AM
douardda accepted D1225: [docker-compose] conf/objstorage: Use a higher value for client_max_size.

I'm ok with this, but whichever client_max_size we set, we might encounter this traceback. The question is then: how should we handle this the proper way? But it's another story,

Mar 8 2019, 10:37 AM
douardda accepted D1224: objstorage: Reinstate a good default value for client_max_size.
Mar 8 2019, 10:34 AM
douardda updated the diff for D1221: readme: integrate the docker-based development setup from the main's doc.

rebased and fix some of zack's comments

Mar 8 2019, 10:28 AM
douardda updated the summary of D1221: readme: integrate the docker-based development setup from the main's doc.
Mar 8 2019, 10:22 AM
douardda added inline comments to D1221: readme: integrate the docker-based development setup from the main's doc.
Mar 8 2019, 10:18 AM
douardda updated the diff for D1220: Refactor the getting started guides.

remove the paragraph on disabling the postgresql service

Mar 8 2019, 9:43 AM

Mar 6 2019

Herald added a reviewer for D1223: tests: enforce LC_ALL since test_cli depends on it: Reviewers.
Mar 6 2019, 12:46 PM

Mar 5 2019

douardda updated the diff for D1220: Refactor the getting started guides.

fixes from olasd's comments

Mar 5 2019, 4:02 PM
douardda created D1221: readme: integrate the docker-based development setup from the main's doc.
Mar 5 2019, 3:33 PM
douardda created D1220: Refactor the getting started guides.
Mar 5 2019, 3:24 PM

Mar 4 2019

douardda committed rDLDG791bd65fc5b9: tasks: give a proper function name to the task that loads a git repo from a zip (authored by douardda).
tasks: give a proper function name to the task that loads a git repo from a zip
Mar 4 2019, 3:08 PM
douardda committed rDLDG0415610ead1e: tasks: rename load_disk_git_repository task's function into load_git_from_dir (authored by douardda).
tasks: rename load_disk_git_repository task's function into load_git_from_dir
Mar 4 2019, 3:08 PM
douardda closed D1219: settings: clear the password validators in development settings.
Mar 4 2019, 1:53 PM
douardda committed rDWAPPSb7719e3987c1: settings: clear the password validators in development settings (authored by douardda).
settings: clear the password validators in development settings
Mar 4 2019, 1:53 PM
douardda committed rCDFD7b3213293ca1: conf: make swh-web always serve assets (authored by douardda).
conf: make swh-web always serve assets
Mar 4 2019, 1:53 PM
douardda committed rCDFD0b031ba074df: web: remove djangosettings.py file and use swh.web.settings.development (authored by douardda).
web: remove djangosettings.py file and use swh.web.settings.development
Mar 4 2019, 1:53 PM
douardda committed rCDFD0c89cd3350bb: dockerfile: ensure swh-web's version of pip is up to date (authored by douardda).
dockerfile: ensure swh-web's version of pip is up to date
Mar 4 2019, 1:53 PM
douardda committed rCDFDaac4cff40773: grafana: replace the tasks dashboard with olasd's task-processing one (authored by douardda).
grafana: replace the tasks dashboard with olasd's task-processing one
Mar 4 2019, 1:50 PM
douardda closed D1191: grafana: replace the tasks dashboard with olasd's task-processing one.
Mar 4 2019, 1:50 PM
douardda updated the diff for D1191: grafana: replace the tasks dashboard with olasd's task-processing one.

rebased

Mar 4 2019, 1:50 PM
douardda created D1219: settings: clear the password validators in development settings.
Mar 4 2019, 1:42 PM
douardda created P366 swh-web with development settings.
Mar 4 2019, 1:29 PM
douardda closed D1190: celery: fix the celery config to ensure the default 'celery' queue exists.

closed by 40d32b4eb78f6fa3ee0848e028437c0598ea11e9

Mar 4 2019, 11:20 AM
douardda closed D1189: celery: do not set the default value for celery's result_backend.

closed by 55f7e1f66a4d974a9aa158653837d4be9a615232

Mar 4 2019, 11:20 AM

Feb 25 2019

douardda committed rDOBJSc17337598d9a: Move bin/ scripts in cli.py and document a bit how to use the objstorage. (authored by douardda).
Move bin/ scripts in cli.py and document a bit how to use the objstorage.
Feb 25 2019, 6:39 PM
douardda closed D1181: Move bin/ scripts in cli.py and document a bit how to use the objstorage..
Feb 25 2019, 6:39 PM
douardda updated the diff for D1181: Move bin/ scripts in cli.py and document a bit how to use the objstorage..

cli: properly set the app's debug flag according the log-level option

Feb 25 2019, 6:37 PM
douardda added inline comments to D1181: Move bin/ scripts in cli.py and document a bit how to use the objstorage..
Feb 25 2019, 6:32 PM
Herald added a reviewer for D1191: grafana: replace the tasks dashboard with olasd's task-processing one: Reviewers.
Feb 25 2019, 5:57 PM
douardda closed D1177: Fix celery workers conf files: explicit list of queues is now required.
Feb 25 2019, 5:56 PM
douardda committed rCDFD1f10d774be7b: Fix celery workers conf files: explicit list of queues is now required (authored by douardda).
Fix celery workers conf files: explicit list of queues is now required
Feb 25 2019, 5:56 PM
douardda committed rCDFD777d5be54ec0: Fix nginx revert proxy for rabbitmq (authored by douardda).
Fix nginx revert proxy for rabbitmq
Feb 25 2019, 5:56 PM
douardda closed D1176: Fix nginx revert proxy for rabbitmq.
Feb 25 2019, 5:56 PM
douardda committed rCDFD48038bdb707a: Upgrade to pg 11 (authored by douardda).
Upgrade to pg 11
Feb 25 2019, 5:56 PM
douardda closed D1175: Upgrade to pg 11.
Feb 25 2019, 5:56 PM
douardda committed rCDFD3a7e4df76175: Activate a 10s heartbeat for celery workers (authored by douardda).
Activate a 10s heartbeat for celery workers
Feb 25 2019, 5:56 PM
douardda closed D1174: Activate a 10s heartbeat for celery workers.
Feb 25 2019, 5:56 PM
douardda updated the diff for D1177: Fix celery workers conf files: explicit list of queues is now required.

rebased

Feb 25 2019, 5:55 PM
douardda updated the diff for D1176: Fix nginx revert proxy for rabbitmq.

rebased

Feb 25 2019, 5:55 PM
douardda updated the diff for D1175: Upgrade to pg 11.

rebased

Feb 25 2019, 5:55 PM
douardda updated the diff for D1174: Activate a 10s heartbeat for celery workers.

rebased

Feb 25 2019, 5:55 PM
douardda committed rDSCH40d32b4eb78f: celery: fix the celery config to ensure the default 'celery' queue exists (authored by douardda).
celery: fix the celery config to ensure the default 'celery' queue exists
Feb 25 2019, 2:26 PM
douardda committed rDSCH55f7e1f66a4d: celery: do not set the default value for celery's result_backend (authored by douardda).
celery: do not set the default value for celery's result_backend
Feb 25 2019, 2:26 PM
douardda closed T1539: Investigate strange behavior seen in the rabbitmq admin web page as Resolved by committing rDSCH55f7e1f66a4d: celery: do not set the default value for celery's result_backend.
Feb 25 2019, 2:26 PM · Scheduling utilities
douardda committed rDSCH5b08922a5e43: cli: kill a useless statement (authored by douardda).
cli: kill a useless statement
Feb 25 2019, 2:26 PM
douardda closed D1187: cli: kill a useless statement.
Feb 25 2019, 2:26 PM
douardda abandoned D1188: tox: pifpaf is not needed any more.

obviously this is false, pifpaf is still needed here. Sorry.

Feb 25 2019, 2:09 PM
Herald added a reviewer for D1190: celery: fix the celery config to ensure the default 'celery' queue exists: Reviewers.
Feb 25 2019, 2:05 PM
Herald added a reviewer for D1189: celery: do not set the default value for celery's result_backend: Reviewers.
Feb 25 2019, 2:05 PM
douardda added a revision to T1539: Investigate strange behavior seen in the rabbitmq admin web page: D1189: celery: do not set the default value for celery's result_backend.
Feb 25 2019, 2:05 PM · Scheduling utilities
Herald added a reviewer for D1188: tox: pifpaf is not needed any more: Reviewers.
Feb 25 2019, 2:04 PM
Herald added a reviewer for D1187: cli: kill a useless statement: Reviewers.
Feb 25 2019, 2:03 PM
douardda added a comment to T1539: Investigate strange behavior seen in the rabbitmq admin web page.

I think these queues are result queues, since we have the result backend activated by default now, and its default value is rpc:// i.e. also amqp.

Feb 25 2019, 1:25 PM · Scheduling utilities
douardda triaged T1541: journal publisher may crash on HTTP timeouts as High priority.
Feb 25 2019, 12:06 PM · Journal

Feb 23 2019

douardda committed R65:b4d298b2194f: docs: fix the index to use a proper sphinx ref to the routing table (authored by douardda).
docs: fix the index to use a proper sphinx ref to the routing table
Feb 23 2019, 1:57 AM
douardda committed R65:da8a25a9f3f8: doc: update index to match new swh-doc format (authored by douardda).
doc: update index to match new swh-doc format
Feb 23 2019, 1:57 AM
douardda committed R65:4e36feab0b66: setup: prepare for pypi upload (authored by douardda).
setup: prepare for pypi upload
Feb 23 2019, 1:57 AM
douardda committed R65:41abf2455eff: add build/ and dist/ to gitignore (authored by douardda).
add build/ and dist/ to gitignore
Feb 23 2019, 1:57 AM

Feb 22 2019

Herald added a reviewer for D1181: Move bin/ scripts in cli.py and document a bit how to use the objstorage.: Reviewers.
Feb 22 2019, 5:39 PM
douardda added a comment to D1176: Fix nginx revert proxy for rabbitmq.

What does this do?

Feb 22 2019, 3:23 PM
douardda accepted D1165: Add a CLI tool to reindex origins based on mapping used..
Feb 22 2019, 11:27 AM

Feb 21 2019

Herald added a reviewer for D1177: Fix celery workers conf files: explicit list of queues is now required: Reviewers.
Feb 21 2019, 6:32 PM
Herald added a reviewer for D1176: Fix nginx revert proxy for rabbitmq: Reviewers.
Feb 21 2019, 6:31 PM
Herald added a reviewer for D1175: Upgrade to pg 11: Reviewers.
Feb 21 2019, 6:31 PM
Herald added a reviewer for D1174: Activate a 10s heartbeat for celery workers: Reviewers.
Feb 21 2019, 6:31 PM
douardda accepted D1166: Double the timeout of snapshot_get_latest..
Feb 21 2019, 11:21 AM
douardda added a comment to T1442: Replace Munin graphs with Grafana/Prometheus dashboards.

Since the current status of munin's pg monitoring is inconsistent (eg. awure's dbreplica0 has not pg curve), let's get rid of all munin's pg monitors so there is less confusion, and is one step towards T1356.

Feb 21 2019, 11:06 AM · Sprint 2018 12, System administration
douardda requested changes to D1165: Add a CLI tool to reindex origins based on mapping used..

Please add an entry_point in the setup.py for this cli.

Feb 21 2019, 10:42 AM
douardda committed rDCIDX76e9796fa475: config: use already loaded swh config, if any, when instantiating an Indexer (authored by douardda).
config: use already loaded swh config, if any, when instantiating an Indexer
Feb 21 2019, 10:17 AM
douardda closed D1148: config: use already loaded swh config, if any, when instantiating an Indexer.
Feb 21 2019, 10:17 AM
douardda updated the diff for D1148: config: use already loaded swh config, if any, when instantiating an Indexer.

rebased

Feb 21 2019, 10:17 AM

Feb 20 2019

douardda accepted D1161: deposit: Use explicit configuration.
Feb 20 2019, 6:02 PM · SWORD deposit, Puppet recipes
douardda added a comment to D398: [WIP] "packing" object storage design documentation.

No idea whether if it's of some interest for our subject, but we may also have a look at openio

Feb 20 2019, 1:09 PM · Object storage
douardda committed rDWAPPS7aa941d7b2dd: Update vault service config for vault 0.0.23 (authored by douardda).
Update vault service config for vault 0.0.23
Feb 20 2019, 12:02 PM
douardda closed D1114: Update vault service config for vault 0.0.22.
Feb 20 2019, 12:02 PM
douardda updated the diff for D1114: Update vault service config for vault 0.0.22.

rebased and fix requirements-swh.txt

Feb 20 2019, 12:02 PM
douardda committed rDSCH058f12001ebf: cli: add a --runs option to 'swh-scheduler task list' tool (authored by douardda).
cli: add a --runs option to 'swh-scheduler task list' tool
Feb 20 2019, 11:31 AM
douardda closed D1146: cli: add a --runs option to 'swh-scheduler task list' tool.
Feb 20 2019, 11:31 AM
douardda committed rDSCH7f951d393737: Keep the main loaded configuration as swh.scheduler.CONFIG (authored by douardda).
Keep the main loaded configuration as swh.scheduler.CONFIG
Feb 20 2019, 10:24 AM
douardda closed D1147: Keep the main loaded configuration as swh.scheduler.CONFIG.
Feb 20 2019, 10:24 AM
douardda committed rDSCH73bca41c6de3: celery_backend: remove debug statements and reduce info level verbosity (authored by douardda).
celery_backend: remove debug statements and reduce info level verbosity
Feb 20 2019, 10:24 AM
douardda closed D1145: celery_backend: remove debug statements and reduce info level verbosity.
Feb 20 2019, 10:24 AM
douardda updated the diff for D1146: cli: add a --runs option to 'swh-scheduler task list' tool.

rebase

Feb 20 2019, 10:24 AM
douardda updated the diff for D1147: Keep the main loaded configuration as swh.scheduler.CONFIG.

rebased

Feb 20 2019, 10:23 AM
douardda updated the diff for D1147: Keep the main loaded configuration as swh.scheduler.CONFIG.

rebased

Feb 20 2019, 10:22 AM
douardda updated the diff for D1146: cli: add a --runs option to 'swh-scheduler task list' tool.

add tests... and consequently fix the client part of the new backend method

Feb 20 2019, 10:22 AM
douardda updated the diff for D1145: celery_backend: remove debug statements and reduce info level verbosity.

rebased

Feb 20 2019, 10:21 AM

Feb 19 2019

douardda added a comment to D398: [WIP] "packing" object storage design documentation.

One interesting project we can look at is seaweedfs which implements ideas, presented in this paper, of fb's Haystack.

Feb 19 2019, 5:43 PM · Object storage