Page MenuHomeSoftware Heritage

Fix all icinga checks on staging webapp
Open, NormalPublic

Description

There remains currently 2 checks [1] [2] which returns 404 on icinga.

It's happening because we recently started back from scratch the staging archive (then migrated on new storage1/db1)
We now need to find back all the origins to ingest in that archive so those get green again.

We could also make the checks more dynamic somehow.

[1] https://icinga.softwareheritage.org/dashboard#!/monitoring/service/show?host=webapp.internal.staging.swh.network&service=swh%20webapp%20revision%20log%20end%20to%20end

[2] https://icinga.softwareheritage.org/monitoring/list/services?service_problem=1&sort=service_severity&dir=desc#!/monitoring/service/show?host=webapp.internal.staging.swh.network&service=swh%20webapp%20snapshot%20end%20to%20end

Event Timeline

ardumont triaged this task as Normal priority.Tue, Nov 10, 4:40 PM
ardumont created this task.
ardumont added a comment.EditedThu, Nov 12, 4:59 PM

i recall (now) one of those repositories is the parmap one, we got it on uffizi:

root@uffizi:/srv/storage/space/mirrors/gitorious.org/mnt/repositories/parmap/parmap.git# git log | head -5
commit b9b0ecd1e2f9db10335383651f8317ed8cec8296
Author: Roberto Di Cosmo <github@dicosmo.org>
Date:   Tue Jul 17 14:21:14 2012 +0200

    Moved to github

root@uffizi:/srv/storage/space/mirrors/gitorious.org/mnt/repositories/parmap/parmap.git# grep "parmap/parmap.git " /srv/storage/space/mirrors/gitorious.org/full_mapping.txt
https://gitorious.org/parmap/parmap.git parmap/parmap.git

Injected "manually" [1]:

$ ipython3
In [1]: from swh.loader.git.from_disk import GitLoaderFromDisk

In [2]: loader = GitLoaderFromDisk("https://gitorious.org/parmap/parmap.git", directory="/home/swhworker/repos/parmap.git")

In [3]: loader.load()
Out[3]: {'status': 'eventful'}

[1] swh loader run only deals with the remote repository

[2] The swh.loader.git.from_disk cli is broken

swhworker@worker0:~$ SWH_CONFIG_FILENAME=/etc/softwareheritage/loader_git.yml python3 -m swh.loader.git.from_disk --origin-url https://gitorious.org/parmap/parmap.git --git-directory $HOME/repos/parmap.git/
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3/dist-packages/swh/loader/git/from_disk.py", line 414, in <module>
    main()
  File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/swh/loader/git/from_disk.py", line 412, in main
    return GitLoaderFromDisk().load(origin_url, git_directory, visit_date)
TypeError: __init__() missing 1 required positional argument: 'url'

(not that it is used often ¯\_(ツ)_/¯)

One more check done.
One last check to go.

ardumont updated the task description. (Show Details)Thu, Nov 12, 5:49 PM

for the last check, here is its origin:

softwareheritage=> select url from origin o inner join origin_visit ov on ov.origin=o.id inner join origin_visit_status ovs on ov.origin=ovs.origin and ov.visit=ovs.visit where ov.type='deb' and ovs.snapshot='\xbaebc2109e4a2ec22a1129a38596
47e191d04df4';
             url
-----------------------------
 deb://Debian/packages/linux
 deb://Debian/packages/linux
 deb://Debian/packages/linux
 deb://Debian/packages/linux
 deb://Debian/packages/linux
 deb://Debian/packages/linux
(6 rows)