for example:
select last_visit_status, count(ovs.url) from origin_visit_stats ovs join listed_origins lo on lo.url = ovs.url and lo.visit_type = ovs.visit_type where lister_id='8d1ebad5-a2dc-482c-9b9b-188665274377' group by last_visit_status;
Just an inline remark but otherwise it's ok.
Update according the review suggestions
migrated to gitlab MR https://gitlab.softwareheritage.org/infra/swh-sysadmin-provisioning/-/merge_requests/88
Task moved on gitlab: https://gitlab.softwareheritage.org/infra/sysadm-environment/-/issues/4653
task will be handled in gitlab: https://gitlab.softwareheritage.org/infra/sysadm-environment/-/issues/4648
thanks, I will try to restart the loading with the patch to see how it behaves
The loading finally failed:
INFO:swh.loader.cvs.loader.CvsLoader:Load origin 'ssh://anoncvs@anoncvs.NetBSD.org:/cvsroot/src' with type 'cvs' DEBUG:swh.loader.cvs.loader.CvsLoader:lister_not provided, skipping extrinsic origin metadata DEBUG:swh.loader.cvs.loader.CvsLoader:prepare; origin_url=ssh://anoncvs@anoncvs.NetBSD.org:/cvsroot/src scheme=ssh path=/cvsroot/src Warning: Permanently added 'anoncvs.netbsd.org,199.233.217.198' (RSA) to the list of known hosts. DEBUG:swh.loader.cvs.loader.CvsLoader:Fetching CVS rlog from anoncvs.netbsd.org:/cvsroot/src
I've launched the loader on the /src subdirectory, if I follow the cvs loader documentation: https://forge.softwareheritage.org/source/swh-loader-cvs/browse/master/docs/ ,
it should also be launched on the other subdirectories:
The loader got killed after it starts to consume a lot of memory...
Warning: Permanently added 'anoncvs.netbsd.org,199.233.217.198' (RSA) to the list of known hosts. DEBUG:swh.loader.cvs.loader.CvsLoader:Fetching CVS rlog from anoncvs.netbsd.org:/cvsroot/src Killed swh@loader-cvs-manual:~$
The first try with a oneshot task failed, the containe was killed:
swhscheduler@scheduler0:~$ swh scheduler --url http://scheduler0.internal.staging.swh.network:5008/ task add load-cvs -p oneshot ssh://anoncvs@anoncvs.NetBSD.org:/cvsroot/src Created 1 tasks
DNS entry gitlab.softwareheritage.org declared in the dns
name: euwest-gitlab-production internal_ip: 192.168.200.4 public_ip: 20.50.42.230
I've tested with the ips closed to the apache ones (88.99.95.218 and 88.99.95.220) and we can reach them, so we are probably blocked somewhere in the apache's infra
Apparently we're blocked somewhere at hertzner:
% tcptraceroute downloads.apache.org Selected device vlan1300, address XXX, port 49995 for outgoing packets Tracing the path to downloads.apache.org (88.99.95.219) on TCP port 80 (http), 30 hops max 1 xxx 2 xxx 3 * * * 4 xe1-1-10-paris1-rtr-131.noc.renater.fr (193.51.177.106) 1.101 ms 1.038 ms 1.018 ms 5 renater-ias-geant-gw.par.fr.geant.net (83.97.89.9) 1.470 ms 1.343 ms 1.259 ms 6 ae5.mx1.gen.ch.geant.net (62.40.98.182) 18.107 ms 18.143 ms 17.893 ms 7 ae2.mx1.fra.de.geant.net (62.40.98.180) 17.804 ms 27.069 ms 18.987 ms 8 * * * 9 core24.fsn1.hetzner.com (213.239.252.42) 20.140 ms 20.348 ms 19.926 ms 10 ex9k1.dc1.fsn1.hetzner.com (213.239.245.234) 19.754 ms 19.677 ms 20.425 ms 11 * * * ...
After the operator was updated to 0.12.2, it was possible to upgrade kubernetes to 1.22.
It was done by changing the version in the azure portal:
Looks like the gitlab operator is now compatible with kubernetes 1.22.
https://docs.gitlab.com/operator/installation.html#kubernetes
unfortunately it can't work without T4604 as the store is configured to use the letencrypt certificate:
rebase
@KShivendu Any news regarding these profiling ?