Page MenuHomeSoftware Heritage

Extract List of all IPOL deposits in csv
Closed, ResolvedPublic

Description

Export from the deposit storage list of deposits with SWHID for IPOL

Event Timeline

moranegg triaged this task as Normal priority.Apr 2 2021, 2:44 PM
moranegg created this task.

Raw extract data [1] of both deposit with status done and those joined as pastes:

  • deposit-ipol.done.csv: P999
  • deposit-ipol.not-done.csv: P1000

[1]

$ psql service=swh-deposit
psql (12.6)
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.

softwareheritage-deposit=> \conninfo
You are connected to database "softwareheritage-deposit" as user "guest" on host "belvedere.internal.softwareheritage.org" (address "192.168.100.210") at port "5432".
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
softwareheritage-deposit=> \copy (select * from deposit where client_id=7 and status='done' order by id asc) to '/tmp/deposit-ipol.not-done.csv' with csv;
COPY 115
softwareheritage-deposit=> \copy (select * from deposit where client_id=7 and status!='done' order by id asc) to '/tmp/deposit-ipol.not-done.csv' with csv;
COPY 68

Sample extract:

551,2020-04-20 23:11:20.74476+00,2020-04-20 23:11:20.731795+00,87afca27-ebbb-408a-845d-5277b6a92f49,swh:1:dir:a10423592dd061a00f7d34e4a3c102ba00c3d2ab,done,7,6,,,swh:1:dir:a10423592dd061a00f7d34e4a3c102ba00c3d2ab;origin=https://www.ipol.im/87afca27-ebbb-408a-845d-5277b6a92f49;visit=swh:1:snp:98b5853bb907a1d931d46521c26f753ddf4de4d3;anchor=swh:1:rev:86e6c7e3ad0699601f73f343344a65f65b6f76ce;path=/,325543487,325543502,https://doi.org/10.5201/87afca27-ebbb-408a-845d-5277b6a92f49
595,2020-05-06 22:48:30.487078+00,2020-05-06 22:48:30.447662+00,ipol.2018.236,swh:1:dir:a10423592dd061a00f7d34e4a3c102ba00c3d2ab,done,7,6,,,swh:1:dir:a10423592dd061a00f7d34e4a3c102ba00c3d2ab;origin=https://doi.org/10.5201/ipol.2018.236;visit=swh:1:snp:206991563b219d18cdfc998d0e35716ba142c12e;anchor=swh:1:rev:baabf12542cd6df80af9594de8922b5281f07272;path=/,333184241,333184251,https://doi.org/10.5201/ipol.2018.236
596,2020-05-06 22:50:06.271071+00,2020-05-06 22:50:06.267703+00,ipol.2018.236,swh:1:dir:a10423592dd061a00f7d34e4a3c102ba00c3d2ab,done,7,6,,,swh:1:dir:a10423592dd061a00f7d34e4a3c102ba00c3d2ab;origin=https://doi.org/10.5201/ipol.2018.236;visit=swh:1:snp:3ceae6836c8fa31e09d66133934dbc2c68ac7860;anchor=swh:1:rev:17ba798378f39d47e0525ac436856976262ac733;path=/,333184242,333184252,https://doi.org/10.5201/ipol.2018.236