Smoke test it with remote and local (archived from BitBucket) repositories.
loader mercurial version v2.1
Description
Description
Status | Assigned | Task | ||
---|---|---|---|---|
Migrated | gitlab-migration | T3338 Load the archived bitbucket mercurial repositories | ||
Migrated | gitlab-migration | T3337 Smoke test ingestion of bitbucket repositories with latest loader mercurial | ||
Migrated | gitlab-migration | T3336 Deploy swh.loader.mercurial 2.1 in staging |
Event Timeline
Comment Actions
Smoke test it with remote repositories.
Well, our check end to end loading the hgview [1] repository is now green again and no longer flip-floping so that takes care of it.
Some other remote repositories were tested as well without any issues [2]
[1] https://foss.heptapod.net/mercurial/hgview
[2] P1107
Smoke test it with local bitbucket repositories
that's next.
Comment Actions
Smoke test it with local bitbucket repositories
that's next.
It's working as expected [1].
This is ready to be deployed in production.
Here is their visit status result and analysis next to visit status that matters.
12:46:22 softwareheritage@belvedere:5432=> select o.url, ovs.date, ovs.status, ovs.snapshot from origin o inner join origin_visit_status ovs on o.id=ovs.origin where o.url in ('https://bitbucket.org/sunithacharan/pocketbook1', 'https://bit bucket.org/alienonland/ownershipandriskweb', 'https://bitbucket.org/alienonland/ownershipandriskweb/wiki', 'https://bitbucket.org/rarchk/jsmn', 'https://bitbucket.org/rarchk/jsmn/wiki', 'https://bitbucket.org/k4nagatsuki/cw_wsn1example-hg' , 'https://bitbucket.org/csimpson/pythononrails', 'https://bitbucket.org/tindelli/polytope') and status='full' order by origin desc,visit desc; +------------------------------------------------------------+-------------------------------+--------+--------------------------------------------+ | url | date | status | snapshot | +------------------------------------------------------------+-------------------------------+--------+--------------------------------------------+ | https://bitbucket.org/k4nagatsuki/cw_wsn1example-hg | 2021-07-30 10:30:36.532165+00 | full | (null) | 2nd run uneventful | https://bitbucket.org/k4nagatsuki/cw_wsn1example-hg | 2021-07-30 10:29:41.131538+00 | full | \x4e85afea09148d757966425d12026f99c40cd181 | 1st run evenfful | https://bitbucket.org/rarchk/jsmn/wiki | 2021-07-30 10:30:30.356632+00 | full | (null) | 2nd run uneventful | https://bitbucket.org/rarchk/jsmn/wiki | 2021-07-30 10:29:32.817241+00 | full | \x9731b94489737bc6a459a591dedcbeb4e2adde1d | 1st run evenfful | https://bitbucket.org/alienonland/ownershipandriskweb/wiki | 2021-07-30 10:30:26.201182+00 | full | (null) | 2nd run uneventful | https://bitbucket.org/alienonland/ownershipandriskweb/wiki | 2021-07-30 10:29:27.681315+00 | full | \x564df4553f129c77c7f683931929907266f9f072 | 2nd run uneventful | https://bitbucket.org/tindelli/polytope | 2021-07-30 10:43:12.058411+00 | full | (null) | 3rd run uneventful (i had another run independently from the paste, overlapping range) | https://bitbucket.org/tindelli/polytope | 2021-07-30 10:30:41.082185+00 | full | (null) | 2nd run uneventful | https://bitbucket.org/tindelli/polytope | 2021-07-30 10:30:07.399909+00 | full | \x4cfe3bcc2a05672715ca91b2634eaf7e923a5e71 | 1st run uneventful | https://bitbucket.org/tindelli/polytope | 2019-09-23 16:33:01.485429+00 | full | \x5d36d39afc5ff7685c8e3ad69229eb3a4add9e27 | previous snapshot prior to the new release so snapshot was different because of the "default" extid version 0 | https://bitbucket.org/tindelli/polytope | 2019-09-19 13:29:25.313042+00 | full | \x5d36d39afc5ff7685c8e3ad69229eb3a4add9e27 | ... | https://bitbucket.org/tindelli/polytope | 2019-09-11 13:28:52.070991+00 | full | \x5d36d39afc5ff7685c8e3ad69229eb3a4add9e27 | | https://bitbucket.org/tindelli/polytope | 2019-08-25 11:10:14.362887+00 | full | \x5d36d39afc5ff7685c8e3ad69229eb3a4add9e27 | | https://bitbucket.org/tindelli/polytope | 2019-07-06 03:26:54.404194+00 | full | \x5d36d39afc5ff7685c8e3ad69229eb3a4add9e27 | | https://bitbucket.org/rarchk/jsmn | 2021-07-30 10:30:28.378383+00 | full | (null) | | https://bitbucket.org/rarchk/jsmn | 2021-07-30 10:29:30.613391+00 | full | \xc07b7f6b2c84a59b2ed38f867831cec466db02cb | | https://bitbucket.org/rarchk/jsmn | 2019-12-10 11:18:00.14157+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-12-08 19:09:42.092199+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-12-07 05:46:00.746191+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-12-05 10:56:01.93435+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-12-02 20:46:13.740019+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-12-01 08:25:17.093066+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-29 18:12:02.422935+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-28 00:27:55.639441+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-26 06:37:10.417357+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-24 18:23:50.054938+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-23 08:01:28.024955+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-21 15:57:13.062139+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-19 21:53:58.539205+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-18 03:55:22.896125+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-16 16:12:55.214658+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-15 00:06:22.283524+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-13 04:10:10.904844+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-11 11:37:13.496159+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-09 22:42:06.663271+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-08 06:31:49.302642+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-06 13:37:37.053864+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-04 21:04:16.742105+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-11-01 21:49:58.214468+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-29 16:40:36.82471+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-17 07:24:10.446816+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-15 00:00:33.547458+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-12 07:08:42.233412+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-09 22:56:17.834132+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-07 21:31:22.227515+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-05 14:08:13.437975+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-10-03 06:17:19.175246+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-09-30 09:57:31.611388+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-09-26 00:01:59.286519+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-09-08 22:29:07.337007+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-08-23 12:34:55.080055+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/rarchk/jsmn | 2019-07-01 12:53:13.721234+00 | full | \x61320da4a3980cca57f93a5106b70126078680e1 | | https://bitbucket.org/sunithacharan/pocketbook1 | 2021-07-30 10:30:21.770151+00 | full | (null) | empty repository [2] | https://bitbucket.org/sunithacharan/pocketbook1 | 2021-07-30 10:29:22.310871+00 | full | (null) | | https://bitbucket.org/csimpson/pythononrails | 2021-07-30 10:30:38.998849+00 | full | (null) | same pattern as before... | https://bitbucket.org/csimpson/pythononrails | 2021-07-30 10:29:44.559665+00 | full | \x042f695a1768bff19b8dfdd497ad4f6504e0fc6f | | https://bitbucket.org/csimpson/pythononrails | 2019-12-10 17:42:53.270323+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-12-08 23:35:46.74614+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-12-07 10:55:28.856216+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-12-05 17:08:36.001726+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-12-03 02:07:46.069676+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-12-01 12:46:48.708118+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-29 23:23:17.158027+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-28 05:46:14.40683+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-26 12:20:18.439113+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-24 22:17:57.685404+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-23 12:39:16.899187+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-21 21:10:34.829363+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-20 02:42:45.542426+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-18 08:05:14.350113+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-16 20:41:21.411309+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-15 04:40:16.795394+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-13 10:13:04.192558+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-11 16:21:47.134495+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-10 02:49:53.025858+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-08 11:29:23.697376+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-06 18:48:27.53713+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-05 01:43:06.324477+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-11-02 06:24:22.09214+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-30 01:47:24.573198+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-17 17:33:33.105511+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-15 06:26:47.392683+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-12 13:37:10.587124+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-10 04:59:49.034164+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-08 03:51:15.326945+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-05 22:06:30.09897+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-03 23:20:14.156245+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-10-01 16:11:35.54927+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-09-29 01:41:35.957731+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-09-24 23:01:59.883073+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-09-07 09:03:17.9129+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-08-13 02:03:07.679798+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/csimpson/pythononrails | 2019-06-30 15:38:58.06108+00 | full | \x34446e03d09755b3e48569a575f0eec7fe149abe | | https://bitbucket.org/alienonland/ownershipandriskweb | 2021-07-30 10:30:24.049421+00 | full | (null) | | https://bitbucket.org/alienonland/ownershipandriskweb | 2021-07-30 10:29:25.438277+00 | full | \x4f429edb601ea9aa1ab1d48aab4e961c3eab159e | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-10-12 20:16:01.14027+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-10-10 10:40:15.296953+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-10-08 08:12:09.836132+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-10-06 03:57:51.121655+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-10-04 04:05:41.913737+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-10-01 19:54:23.079177+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-30 04:12:31.91622+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-28 17:51:05.256742+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-27 15:44:13.862242+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-26 03:56:42.959891+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-24 00:32:53.056976+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-22 20:49:03.454986+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-20 19:37:44.966056+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-09-08 15:52:01.564134+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-08-31 01:27:53.877789+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-07-30 01:52:57.936951+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | | https://bitbucket.org/alienonland/ownershipandriskweb | 2019-06-28 01:51:50.188403+00 | full | \x661d74b97dfd5519d8840e3f1fdd42978db10a98 | +------------------------------------------------------------+-------------------------------+--------+--------------------------------------------+ (112 rows)
[1] P1108
[2] Emtpy repository
ardumont@worker16:/srv/storage/space/mirrors/boatbucket$ base_dir=/srv/storage/space/mirrors/boatbucket ardumont@worker16:/srv/storage/space/mirrors/boatbucket$ grep sunithacharan/pocketbook1 mapping-to-repos.txt refugees/data/f6/f6f2cb2b-7bb5-482a-9974-c86fc28a1ef8/main-repo.mercurial https://bitbucket.org/sunithacharan/pocketbook1 ardumont@worker16:/srv/storage/space/mirrors/boatbucket$ repo=$(grep sunithacharan/pocketbook1 mapping-to-repos.txt | awk '{print $1}') ardumont@worker16:/tmp% hg clone $base_dir/$repo not trusting file /srv/storage/space/mirrors/boatbucket/refugees/data/f6/f6f2cb2b-7bb5-482a-9974-c86fc28a1ef8/main-repo.mercurial/.hg/hgrc from untrusted user 1002, group aeviso destination directory: main-repo.mercurial no changes found updating to branch default 0 files updated, 0 files merged, 0 files removed, 0 files unresolved swhworker@worker0:/tmp$ cd main-repo.mercurial/ swhworker@worker0:/tmp/main-repo.mercurial$ hg log -> nothing to show here