Page MenuHomeSoftware Heritage

Docker environmentFolder
ActivePublic

Members

  • This project does not have any members.
  • View All

Watchers

  • This project does not have any watchers.
  • View All

Recent Activity

Jul 29 2021

vlorentz closed T1936: integrate swh-graph into the docker environment as Resolved.
Jul 29 2021, 3:51 PM · Docker environment, Graph service

Jul 28 2021

vlorentz added revisions to T1936: integrate swh-graph into the docker environment: D6042: graph: don't run export+compression unless needed, and add documentation, D6038: journalprocessor: Fix deserialize_message raising EOFError on the last message of each assignment.
Jul 28 2021, 4:44 PM · Docker environment, Graph service
ardumont added a comment to T1936: integrate swh-graph into the docker environment.

Plus that documents it a bit in an automatic manner, so *thumbs up*.

Jul 28 2021, 10:37 AM · Docker environment, Graph service

Jul 27 2021

vlorentz added a comment to T1936: integrate swh-graph into the docker environment.

I decided to make the swh-graph container create the compressed graph itself before starting. That's the easiest way to use it AND to implement it IMO.

Jul 27 2021, 5:39 PM · Docker environment, Graph service
vlorentz claimed T1936: integrate swh-graph into the docker environment.
Jul 27 2021, 5:37 PM · Docker environment, Graph service
vlorentz added a revision to T1936: integrate swh-graph into the docker environment: D6028: journalprocessor: Fix freeze on empty offset ranges..
Jul 27 2021, 5:37 PM · Docker environment, Graph service
vlorentz added revisions to T1936: integrate swh-graph into the docker environment: D6029: Add swh-graph to the docker environment, D6030: server: Define make_app_from_configfile so it can be ran by gunicorn.
Jul 27 2021, 5:37 PM · Docker environment, Graph service

May 19 2021

anlambert closed T3326: docker tests on Jenkins: error while removing network as Resolved.

Issue is now solved, closing this.

May 19 2021, 11:19 AM · Continuous Integration, Docker environment
ardumont added a comment to T3326: docker tests on Jenkins: error while removing network.

awesome, thanks :)

May 19 2021, 9:22 AM · Continuous Integration, Docker environment

May 18 2021

vsellier added a comment to T3326: docker tests on Jenkins: error while removing network.

and the build is green ;)

May 18 2021, 7:07 PM · Continuous Integration, Docker environment
anlambert added a comment to T3326: docker tests on Jenkins: error while removing network.
16:50:52 ============================= test session starts ==============================
16:50:52 platform linux -- Python 3.7.3, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
16:50:52 cachedir: .tox/py3/.pytest_cache
16:50:52 rootdir: /var/lib/jenkins/workspace/swh-docker-dev/docker
16:50:52 plugins: testinfra-6.3.0, testinfra-6.0.0
16:50:52 collected 7 items
16:50:52 
16:50:52 tests/test_deposit.py .....                                              [ 71%]
16:53:13 tests/test_git_loader.py .                                               [ 85%]
May 18 2021, 6:57 PM · Continuous Integration, Docker environment
vsellier added a comment to T3326: docker tests on Jenkins: error while removing network.

thanks for having investigated that

May 18 2021, 6:49 PM · Continuous Integration, Docker environment
anlambert added a comment to T3326: docker tests on Jenkins: error while removing network.

So the docker_default network did not get removed since April 20, 2021, see docker network inspect docker_default output below:

15:07:16 [
15:07:16     {
15:07:16         "Name": "docker_default",
15:07:16         "Id": "a0145eb2f5b3f055bfafce7a88289346c4668f3a19f2544122267272cd96f7b3",
15:07:16         "Created": "2021-04-20T02:57:51.141593622Z",
15:07:16         "Scope": "local",
15:07:16         "Driver": "bridge",
15:07:16         "EnableIPv6": false,
15:07:16         "IPAM": {
15:07:16             "Driver": "default",
15:07:16             "Options": null,
15:07:16             "Config": [
15:07:16                 {
15:07:16                     "Subnet": "172.28.0.0/16",
15:07:16                     "Gateway": "172.28.0.1"
15:07:16                 }
15:07:16             ]
15:07:16         },
15:07:16         "Internal": false,
15:07:16         "Attachable": true,
15:07:16         "Ingress": false,
15:07:16         "ConfigFrom": {
15:07:16             "Network": ""
15:07:16         },
15:07:16         "ConfigOnly": false,
15:07:16         "Containers": {},
15:07:16         "Options": {},
15:07:16         "Labels": {
15:07:16             "com.docker.compose.network": "default",
15:07:16             "com.docker.compose.project": "docker",
15:07:16             "com.docker.compose.version": "1.29.1"
15:07:16         }
15:07:16     }
15:07:16 ]
May 18 2021, 5:18 PM · Continuous Integration, Docker environment

May 12 2021

anlambert triaged T3326: docker tests on Jenkins: error while removing network as Normal priority.
May 12 2021, 1:46 PM · Continuous Integration, Docker environment
anlambert created T3326: docker tests on Jenkins: error while removing network.
May 12 2021, 1:46 PM · Continuous Integration, Docker environment

May 4 2021

KShivendu added a comment to T3304: Kafka throws flush timeout error.

If you face this issue, try restarting the containers using docker-compose down and docker-compose up.

May 4 2021, 4:00 PM · Docker environment, Journal
vlorentz triaged T3304: Kafka throws flush timeout error as High priority.
May 4 2021, 12:24 PM · Docker environment, Journal
vlorentz edited projects for T3304: Kafka throws flush timeout error, added: Docker environment; removed Core Loader.
May 4 2021, 12:24 PM · Docker environment, Journal

Apr 8 2021

anlambert closed T3181: docker-dev: fix vault test which fails the build as Resolved by committing rDENV80e43a5fbe72: docker/tests: Fix symlink handling in test_vault_directory.
Apr 8 2021, 4:28 PM · Vault, Development environment, Docker environment
anlambert added a revision to T3181: docker-dev: fix vault test which fails the build: D5455: docker/tests: Fix symlink handling in test_vault_directory.
Apr 8 2021, 2:49 PM · Vault, Development environment, Docker environment
anlambert added a comment to T3181: docker-dev: fix vault test which fails the build.

So either the vault is fine and the test needs to be improved.
Or the vault builds the tarball without taking into account the symlink case.

Apr 8 2021, 12:08 PM · Vault, Development environment, Docker environment

Mar 29 2021

ardumont added projects to T3181: docker-dev: fix vault test which fails the build: Docker environment, Development environment, Vault.
Mar 29 2021, 9:46 AM · Vault, Development environment, Docker environment

Mar 15 2021

vlorentz lowered the priority of T2854: docker: Service Startup Error from High to Normal.

@rendong951 Sorry, but we can't reproduce the issue on our end.

Mar 15 2021, 1:14 PM · Docker environment, Development environment

Feb 18 2021

vlorentz renamed T3059: CODEMETA.json files do not get indexed from CODEMETA.json files do not get indexed to CODEMETA.json files do not get indexed in the docker environment.
Feb 18 2021, 12:46 PM · Easy hack, Indexer

Dec 31 2020

rendong951 added a comment to T2854: docker: Service Startup Error.

I also try command fllowing your reply,the results are the same.

Dec 31 2020, 8:46 AM · Docker environment, Development environment

Dec 29 2020

vlorentz added a comment to T2854: docker: Service Startup Error.

It's not a DNS issue. The error is Could not resolve hostname https:. It shouldn't try to resolve https:, it's a scheme, not a domain name.

Dec 29 2020, 12:45 PM · Docker environment, Development environment
rendong951 added a comment to T2854: docker: Service Startup Error.

Thank you very much for your reply.
I followed your command.[1]

Dec 29 2020, 8:16 AM · Docker environment, Development environment

Dec 18 2020

ardumont added a comment to T2854: docker: Service Startup Error.

Please wrap your stacktrace, excerpt of code, etc... with triple backquote before and after.
So this becomes more readable.
Thanks in advance.

Dec 18 2020, 9:54 AM · Docker environment, Development environment
rendong951 added a comment to T2854: docker: Service Startup Error.

Thank you very much for your reply.
And I follow your guidance,now,all the services are started,but when I save code in my Software Heritage platform, the status always failed, I checked log :
swh-loader_1 | [2020-12-18 02:49:13,495: ERROR/ForkPoolWorker-1] Loading failure, updating to partial status
swh-loader_1 | Traceback (most recent call last):
swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/dulwich/client.py", line 912, in fetch_pack
swh-loader_1 | refs, server_capabilities = read_pkt_refs(proto)
swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/dulwich/client.py", line 215, in read_pkt_refs
swh-loader_1 | for pkt in proto.read_pkt_seq():
swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/dulwich/protocol.py", line 277, in read_pkt_seq
swh-loader_1 | pkt = self.read_pkt_line()
swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/dulwich/protocol.py", line 223, in read_pkt_line
swh-loader_1 | raise HangupException()
swh-loader_1 | dulwich.errors.HangupException: The remote server unexpectedly closed the connection.
swh-loader_1 |
swh-loader_1 | During handling of the above exception, another exception occurred:
swh-loader_1 |
swh-loader_1 | Traceback (most recent call last):
swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/loader/core/loader.py", line 318, in load
swh-loader_1 | more_data_to_fetch = self.fetch_data()
swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/loader/git/loader.py", line 239, in fetch_data
swh-loader_1 | self.origin.url, self.base_snapshot, do_progress
swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/loader/git/loader.py", line 174, in fetch_pack_from_origin
swh-loader_1 | progress=do_activity,
swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/dulwich/client.py", line 914, in fetch_pack
swh-loader_1 | raise _remote_error_from_stderr(stderr)
swh-loader_1 | dulwich.errors.HangupException: ssh: Could not resolve hostname https: Temporary failure in name resolution

Dec 18 2020, 4:02 AM · Docker environment, Development environment

Dec 9 2020

ardumont added a revision to T2854: docker: Service Startup Error: D4708: swh-indexer: Fix configuration to add the tools configuration entry.
Dec 9 2020, 7:46 PM · Docker environment, Development environment
ardumont renamed T2854: docker: Service Startup Error from Service Startup Error to docker: Service Startup Error.
Dec 9 2020, 6:40 PM · Docker environment, Development environment
ardumont added a project to T2854: docker: Service Startup Error: Docker environment.
Dec 9 2020, 6:40 PM · Docker environment, Development environment

Sep 28 2020

vsellier closed T2644: swh-docker-dev CI build - broken since build #569 - 404 not found for psql apt repo as Resolved.

I have added the `--pull` option in the job configuration so the python image will always be pulled (if needed) during the build :

Sep 28 2020, 10:59 AM · Docker environment, System administration

Sep 27 2020

vsellier claimed T2644: swh-docker-dev CI build - broken since build #569 - 404 not found for psql apt repo.
Sep 27 2020, 11:36 AM · Docker environment, System administration
vsellier added a comment to T2644: swh-docker-dev CI build - broken since build #569 - 404 not found for psql apt repo.

It seems the python 3.7 images on the jenkins master is quite old :

root@thyssen:~# docker images | grep python
python                     3.7                  a4cc999cf2aa        16 months ago       929MB
Sep 27 2020, 11:36 AM · Docker environment, System administration
vlorentz added a comment to T2644: swh-docker-dev CI build - broken since build #569 - 404 not found for psql apt repo.

I'm guessing it's because $VERSION_CODENAME is empty here: https://forge.softwareheritage.org/source/swh-environment/browse/master/docker/Dockerfile$3

Sep 27 2020, 11:17 AM · Docker environment, System administration
zack triaged T2644: swh-docker-dev CI build - broken since build #569 - 404 not found for psql apt repo as High priority.
Sep 27 2020, 11:10 AM · Docker environment, System administration

Sep 17 2020

ardumont added a comment to T2607: git loader OOM when loading the linux kernel repo.

possibly related to T2373.

Sep 17 2020, 8:51 AM · Git loader

Sep 16 2020

zack updated the task description for T2607: git loader OOM when loading the linux kernel repo.
Sep 16 2020, 8:28 PM · Git loader
zack triaged T2607: git loader OOM when loading the linux kernel repo as Normal priority.
Sep 16 2020, 8:26 PM · Git loader

May 16 2020

ardumont added a comment to D3158: (fix ci?) docker/conf/deposit: Add missing storage entry.

It worked \m/ (that was it).

May 16 2020, 5:19 PM · Docker environment, SWORD deposit
ardumont added a comment to D3158: (fix ci?) docker/conf/deposit: Add missing storage entry.

Hmph, should have changed the commit title ¯\_(ツ)_/¯

May 16 2020, 5:09 PM · Docker environment, SWORD deposit
ardumont closed D3158: (fix ci?) docker/conf/deposit: Add missing storage entry.
May 16 2020, 5:07 PM · Docker environment, SWORD deposit
ardumont updated the summary of D3158: (fix ci?) docker/conf/deposit: Add missing storage entry.
May 16 2020, 5:06 PM · Docker environment, SWORD deposit

May 6 2020

vlorentz updated the task description for T2392: swh-scheduler-listener container crashes when running for the first time.
May 6 2020, 11:23 AM · Docker environment, Scheduling utilities
vlorentz triaged T2392: swh-scheduler-listener container crashes when running for the first time as Normal priority.
May 6 2020, 11:23 AM · Docker environment, Scheduling utilities

Jan 27 2020

vlorentz closed T1575: Kafka clients crash easily in docker-compose environment as Resolved.

Probably doesn't anymore since we moved to confluent-kafka

Jan 27 2020, 4:34 PM · Journal, Docker environment

Dec 11 2019

vlorentz closed T2142: Document how to use Sentry with the docker dev environment as Resolved.
Dec 11 2019, 3:42 PM · Docker environment, Sprint 2019/12 (Monitor and Conquer), Metrics/monitoring
vlorentz moved T2142: Document how to use Sentry with the docker dev environment from in progress to deployed on the Sprint 2019/12 (Monitor and Conquer) board.
Dec 11 2019, 3:40 PM · Docker environment, Sprint 2019/12 (Monitor and Conquer), Metrics/monitoring
vlorentz added a comment to T2142: Document how to use Sentry with the docker dev environment.

Resolved by D2424.

Dec 11 2019, 3:39 PM · Docker environment, Sprint 2019/12 (Monitor and Conquer), Metrics/monitoring