Details
- Reviewers
- None
- Group Reviewers
Reviewers - Maniphest Tasks
- T3280: Separate save code now request updates from the save code now ui
docker-compose up, do some save code now requests and amazingly check the status gets
updated when browsing through pagination:
swh-doco exec swh-web-cron /bin/bash + cd /home/tony/work/inria/repo/swh/swh-environment/docker + docker-compose -f docker-compose.yml -f docker-compose.override.yml exec swh-web-cron /bin/bash root@8b20f39d055c:/# tail -f /var/log/refresh-statuses.log + SWH_CONFIG_FILENAME=/web.yml + /srv/softwareheritage/venv/bin/django-admin refresh_savecodenow_statuses /srv/softwareheritage/venv/lib/python3.7/site-packages/jose/backends/cryptography_backend.py:18: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead from cryptography.utils import int_from_bytes, int_to_bytes [27/Apr/2021 14:58:04] [DEBUG] urllib3.connectionpool._new_conn:231 - Starting new HTTP connection (1): swh-scheduler:5008 [27/Apr/2021 14:58:04] [DEBUG] urllib3.connectionpool._make_request:461 - http://swh-scheduler:5008 "POST /task/get HTTP/1.1" 200 497 [27/Apr/2021 14:58:04] [DEBUG] urllib3.connectionpool._make_request:461 - http://swh-scheduler:5008 "POST /task_run/get HTTP/1.1" 200 281 [27/Apr/2021 14:58:04] [DEBUG] urllib3.connectionpool._new_conn:231 - Starting new HTTP connection (1): swh-storage:5002 [27/Apr/2021 14:58:04] [DEBUG] urllib3.connectionpool._make_request:461 - http://swh-storage:5002 "POST /origin/get HTTP/1.1" 200 3 Successfully updated 2 save request(s).
Diff Detail
- Repository
- rDENV Development environment
- Branch
- master
- Lint
No Linters Available - Unit
No Unit Test Coverage - Build Status
Buildable 21055 Build 32676: arc lint + arc unit
Event Timeline
| docker/docker-compose.yml | ||
|---|---|---|
| 242 | adding the cron as mountpoint here installs the file but no cron is running so no. | |
Finally made the damn thing work...
swh-doco exec swh-web-cron /bin/bash + cd /home/tony/work/inria/repo/swh/swh-environment/docker + docker-compose -f docker-compose.yml -f docker-compose.override.yml exec swh-web-cron /bin/bash root@8b20f39d055c:/# tail -f /var/log/refresh-statuses.log + SWH_CONFIG_FILENAME=/web.yml + /srv/softwareheritage/venv/bin/django-admin refresh_savecodenow_statuses /srv/softwareheritage/venv/lib/python3.7/site-packages/jose/backends/cryptography_backend.py:18: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead from cryptography.utils import int_from_bytes, int_to_bytes [27/Apr/2021 14:58:04] [DEBUG] urllib3.connectionpool._new_conn:231 - Starting new HTTP connection (1): swh-scheduler:5008 [27/Apr/2021 14:58:04] [DEBUG] urllib3.connectionpool._make_request:461 - http://swh-scheduler:5008 "POST /task/get HTTP/1.1" 200 497 [27/Apr/2021 14:58:04] [DEBUG] urllib3.connectionpool._make_request:461 - http://swh-scheduler:5008 "POST /task_run/get HTTP/1.1" 200 281 [27/Apr/2021 14:58:04] [DEBUG] urllib3.connectionpool._new_conn:231 - Starting new HTTP connection (1): swh-storage:5002 [27/Apr/2021 14:58:04] [DEBUG] urllib3.connectionpool._make_request:461 - http://swh-storage:5002 "POST /origin/get HTTP/1.1" 200 3 Successfully updated 2 save request(s).
I think this would be simpler to deploy the cron in the same docker-compose service as the webapp.
The cron command could be set directly in the swh-web entrypoint.
You will surely need to modify cron.allow file for that.
| docker/docker-compose.yml | ||
|---|---|---|
| 249 | Adding environment variable in the container does not transmit the environment variables required for the cron to actually work so not an option either... | |
| docker/services/swh-web/refresh-savecodenow-statuses | ||
| 6 | thus the duplication here. Note also that this is installed through the Dockerfile.cron during the image build. | |
I think this would be simpler to deploy the cron in the same docker-compose service as
the webapp. The cron command could be set directly in the swh-web entrypoint.
I tried that and it did not work [1] The cron did not run at all.
You will surely need to modify cron.allow file for that.
do you mean adding the swh user to it ^
(i did not try although that'd mean modifying the main Dockerfile, ain't it?)
Yes, this way you should be able to use crontab in the swh-web entrypoint. There is also crontab -u that could be used.
Yes, this way you should be able to use crontab in the swh-web entrypoint. There
is also crontab -u that could be used.
I'll try now that i'm sure it can work in some way ;)
So, i can't find a way to make your suggestion work properly.
See D5639 with the latest tryouts (changes out of this diff)...
Instead of adding a new docker-compose service, how about adding a background bash process that will call the refresh_savecodenow_statuses django command periodically ?
This can be setup in the swh-web service entrypoint the following way:
diff --git a/docker/services/swh-web/entrypoint.sh b/docker/services/swh-web/entrypoint.sh index 2d0ef86..9fb843f 100755 --- a/docker/services/swh-web/entrypoint.sh +++ b/docker/services/swh-web/entrypoint.sh @@ -46,6 +46,16 @@ case "$1" in echo "Creating Django admin user" echo "$create_admin_script" | python3 -m swh.web.manage shell + ( + while true + do + (date && django-admin refresh_savecodenow_statuses \ + --settings=${DJANGO_SETTINGS_MODULE} 2>&1) >> /tmp/refresh-statuses.log + sleep 5 + done + ) & + disown + echo "starting the swh-web server" exec gunicorn --bind 0.0.0.0:5004 \ --threads 2 \