Page MenuHomeSoftware Heritage
Paste Active Pastes
  • From 1e8c3d694e2775afd01d80031765f919603eef10 Mon Sep 17 00:00:00 2001
    From: Antoine Pietri <antoine.pietri1@gmail.com>
    Date: Thu, 30 Jun 2022 15:47:06 +0200
    Subject: [PATCH] docs: fix relative include of swhgraph.proto
    ...
    • Thu, Jun 30, 3:48 PM
    • 33 Lines
    • Diff
  • grep -v ".zip" guix-sources.json | grep -v ".jar\|.7z\|.gem\|.gz\|.tgz\|.tar.\|.tbz\|.tgz\|output\|integrity\|urls\|inferred\|url\|{\|}\|\]\|\[\|git_ref\|"git"\|hg\|no-origin\|false\|svn_revision\|name\|"svn"\|crates"
    "https://common-lisp.net/project/asdf/archives/asdf-3.3.5.lisp"
    "https://linux-libre.fsfla.org/pub/linux-libre/releases/5.17.14-gnu/deblob-check"
    "https://linux-libre.fsfla.org/pub/linux-libre/releases/5.17.14-gnu/deblob-5.17"
    "http://mccarrolllab.com/download/1276/"
    ...
    • Thu, Jun 30, 3:19 PM
    • 139 Lines
  • swh_graph-replayer.11.vlqkq2mupe8t@shsrvc2 | ERROR:swh.storage.replay:Object has id a03d79c816c7ffebe3e8d90d7d395482075b086f, but it should be c8c1462ff14300cc081475b0d97da679eab5a8ff: Release(name=b'debian/0.26.1-0ubuntu2.2', message=b'tagging package python-glance-store version debian/0.26.1-0ubuntu2.2\n', target=hash_to_bytes('e4c28d0c01ea1274e8e9b28f113572c36211d538'), target_type=ObjectType.REVISION, synthetic=False, author=Person(fullname=b'Corey Bryant <corey.bryant@canonical.com>', name=b'Corey Bryant', email=b'corey.bryant@canonical.com'), date=TimestampWithTimezone(timestamp=Timestamp(seconds=1560178555, microseconds=0), offset_bytes=b'-0400'), metadata=None, id=hash_to_bytes('a03d79c816c7ffebe3e8d90d7d395482075b086f'), raw_manifest=None)
    • Thu, Jun 30, 1:57 PM
    • 1 Line
  • privileged: true
    object_types:
    - content
    - skipped_content
    - directory
    ...
    • Wed, Jun 29, 5:32 PM
    • 20 Lines
  • {
    "license": [
    "LGPL-2.1-only",
    "GPL-3.0-or-later"
    ]
    ...
    • Tue, Jun 28, 4:26 PM
    • 12 Lines
    • JSON
  • terraform apply
    module.worker18.proxmox_vm_qemu.node: Refreshing state... [id=uffizi/qemu/141]
    module.kelvingrove.proxmox_vm_qemu.node: Refreshing state... [id=branly/qemu/123]
    module.provenance-client01.proxmox_vm_qemu.node: Refreshing state... [id=uffizi/qemu/126]
    module.counters1.proxmox_vm_qemu.node: Refreshing state... [id=branly/qemu/139]
    ...
    • Mon, Jun 27, 4:37 PM
    • 3,788 Lines
  • $ terraform apply
    module.webapp1.proxmox_vm_qemu.node: Refreshing state... [id=hypervisor3/qemu/125]
    module.counters1.proxmox_vm_qemu.node: Refreshing state... [id=branly/qemu/139]
    module.search1.proxmox_vm_qemu.node: Refreshing state... [id=branly/qemu/136]
    module.worker17.proxmox_vm_qemu.node: Refreshing state... [id=uffizi/qemu/143]
    ...
    • Mon, Jun 27, 3:38 PM
    • 139 Lines
  • root@getty:~# /usr/local/sbin/create_kafka_users_rocquencourt.sh snyk-prod-01
    Creating user snyk-prod-01, with unprivileged access to consumer group prefix snyk-prod-01-
    Password for user snyk-prod-01:
    Setting user credentials
    Warning: --zookeeper is deprecated and will be removed in a future version of Kafka.
    ...
    • Mon, Jun 27, 2:14 PM
    • 15 Lines
  • scheduler:
    cls: remote
    url: http://swh-scheduler:5008/
    vault:
    ...
    • Tue, Jun 21, 2:31 PM
    • 9 Lines
  • anlambert@carnavalet:/tmp$ cvs -z3 -d:pserver:anonymous@a.cvs.sourceforge.net:/cvsroot/javanms co -P jnms-area-service
    cvs checkout: Updating jnms-area-service
    U jnms-area-service/.classpath
    U jnms-area-service/.packaging
    U jnms-area-service/.project
    ...
    • Mon, Jun 20, 3:18 PM
    • 234 Lines
  • (defun create-user (user)
    (when-unmapped-with-update (:user (forgerie-core:user-username user))
    (let*
    ((avatar (forgerie-core:user-avatar user))
    (avatar
    ...
    • Sat, Jun 18, 4:08 PM
    • 75 Lines
  • def normalize_description(self, description):
    r"""Try to re-decode ``description`` as UTF-16, as this is a somewhat common
    mistake that causes issues in the database because of null bytes in JSON.
    >>> NpmMapping().normalize_description("foo bar")
    ...
    • Fri, Jun 17, 6:05 PM
    • 71 Lines
    • Python
  • Metadata on Social Code Repositories
    ====================================
    MeSoCoRe is a vocabulary which complements ontologies like schema.org/CodeMeta
    and DOAP in describing software projects. While the latter are meant to describe
    ...
    • Thu, Jun 16, 6:00 PM
    • 77 Lines
  • 17:08 $ swh counters -C ~/.config/swh/counters.yml journal-client
    {'brokers': ['kafka1.internal.softwareheritage.org:9094', 'kafka2.internal.softwareheritage.org:9094', 'kafka3.internal.softwareheritage.org:9094', 'kafka4.internal.softwareheritage.org:9094'], 'prefix': 'swh.journal.objects', 'group_id': 'anlambert.origin_counts.dev4', 'object_types': ['origin'], 'batch_size': 1000, 'sasl.mechanism': 'SCRAM-SHA-512', 'security.protocol': 'SASL_SSL', 'sasl.username': 'mirror-test-ro', 'sasl.password': '**********', 'stop_after_objects': None}
    Traceback (most recent call last):
    File "/home/anlambert/.virtualenvs/swh/bin/swh", line 33, in <module>
    sys.exit(load_entry_point('swh.core', 'console_scripts', 'swh')())
    ...
    • Thu, Jun 16, 5:11 PM
    • 26 Lines
  • (swh) anlambert@carnavalet:/tmp$ bzr branch lp:~jelmer/bzr-git/dulwich
    Branched 236 revisions.
    (swh) anlambert@carnavalet:/tmp$ cd dulwich/
    (swh) anlambert@carnavalet:/tmp/dulwich$ brz tags
    bzr-svn-0.1 ?
    ...
    • Tue, Jun 14, 3:11 PM
    • 26 Lines
  • from laptop ok (yavin4):
    ```
    kubectl exec -ti -n gitlab-system deployment/gitlab-toolbox -- /srv/gitlab/bin/rails console
    Defaulted container "toolbox" out of: toolbox, certificates (init), configure (init)
    ...
    • Mon, Jun 13, 3:42 PM
    • 51 Lines
  • morane@hplaptopft0:~/Documents/work/swh/presentations/slides$ git push
    Username for 'https://forge.softwareheritage.org': moranegg
    Password for 'https://moranegg@forge.softwareheritage.org':
    Enumerating objects: 9, done.
    Counting objects: 100% (9/9), done.
    ...
    • Wed, Jun 8, 11:12 PM
    • 13 Lines
  • journal_client:
    cls: kafka
    brokers:
    - broker1.journal.staging.swh.network:9093
    group_id: {group_id}
    ...
    • Tue, Jun 7, 1:29 PM
    • 30 Lines
  • {
    "tar.gz": 124760,
    "tar.xz": 23499,
    "tar.bz2": 10652,
    "git": 6847,
    ...
    • Mon, Jun 6, 8:34 PM
    • 56 Lines
    • JSON
  • 16:04:03 swh-scheduler@db1:5432=> with maven_lister_ids as (
    select id from listers where name='maven'
    ) select now(), visit_type, url
    from
    listed_origins lo1
    ...
    • Fri, Jun 3, 4:12 PM
    • 1,910 Lines
  • swh/web/tests/api/views/test_metadata.py:81:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    swh/web/tests/conftest.py:1106: in inner
    item.ihook.pytest_runtest_call(item=item)
    ../../../.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_hooks.py:265: in __call__
    ...
    • Fri, Jun 3, 11:25 AM
    • 72 Lines
  • 11:00 $ pytest -sv --hypothesis-verbosity=verbose swh/web/tests/api/views/test_metadata.py::test_api_raw_extrinsic_metadata_scroll
    ================================================================================================================================== test session starts ==================================================================================================================================
    platform linux -- Python 3.9.2, pytest-7.1.2, pluggy-1.0.0 -- /home/anlambert/.virtualenvs/swh/bin/python
    cachedir: .pytest_cache
    hypothesis profile 'swh-web-fast-with-verbose-verbosity' -> deadline=None, max_examples=5, verbosity=Verbosity.verbose, database=DirectoryBasedExampleDatabase('/home/anlambert/swh/swh-environment/swh-web/.hypothesis/examples'), suppress_health_check=[HealthCheck.too_slow, HealthCheck.filter_too_much, HealthCheck.function_scoped_fixture]
    ...
    • Fri, Jun 3, 11:02 AM
    • 102 Lines
  • 09:52:37 softwareheritage-scheduler@belvedere:5432=> select * from task_type where type like 'load-%' or type like 'list-%';
    +------------------------------+--------------------------------------------------------------+-----------------------------------------------------------+------------------+--------------+--------------+----------------+------------------+-------------+-------------+
    | type | description | backend_name | default_interval | min_interval | max_interval | backoff_factor | max_queue_length | num_retries | retry_delay |
    +------------------------------+--------------------------------------------------------------+-----------------------------------------------------------+------------------+--------------+--------------+----------------+------------------+-------------+-------------+
    | load-hg | Update an origin of type mercurial | swh.loader.mercurial.tasks.LoadMercurial | 64 days | 1 day | 64 days | 2 | 5000 | 3 | (null) |
    ...
    • Fri, Jun 3, 9:55 AM
    • 55 Lines
  • 09:49:06 swh-scheduler@db1:5432=> select * from task_type where type like 'load-%';
    +-----------------------+--------------------------------------------------------------+----------------------------------------------------+------------------+--------------+--------------+----------------+------------------+-------------+-------------+
    | type | description | backend_name | default_interval | min_interval | max_interval | backoff_factor | max_queue_length | num_retries | retry_delay |
    +-----------------------+--------------------------------------------------------------+----------------------------------------------------+------------------+--------------+--------------+----------------+------------------+-------------+-------------+
    | load-svn-from-archive | Loading svn repositories from svn dump | swh.loader.svn.tasks.MountAndLoadSvnRepository | 1 day | 1 day | 1 day | 1 | 1000 | (null) | (null) |
    ...
    • Fri, Jun 3, 9:52 AM
    • 26 Lines
  • $ docker-compose exec swh-idx-storage swh db --config-file /indexer_storage.yml upgrade indexer_storage
    ERROR:swh.core.db.db_utils:Failed to connect to `postgresql:///?service=swh-indexers`
    Traceback (most recent call last):
    File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/core/db/db_utils.py", line 241, in swh_db_module
    db = connect_to_conninfo(db_or_conninfo)
    ...
    • Thu, Jun 2, 3:06 PM
    • 44 Lines
  • psql service=admin-staging-swh-scheduler
    Pager usage is off.
    Null display is "(null)".
    Timing is on.
    Border style is 2.
    ...
    • Thu, Jun 2, 2:06 PM
    • 776 Lines
  • ```
    swhworker@worker0:~$ dpkg -l python3-swh.indexer | grep ii
    ii python3-swh.indexer 1.1.0-1~swh1~bpo10+1 all Software Heritage Content Indexerswhworker@worker0:~$ echo $SWH_CONFIG_FILENAME
    /etc/softwareheritage/indexer/journal_client.yml
    swhworker@worker0:~$ /usr/bin/swh indexer --config-file $SWH_CONFIG_FILENAME journal-client indexer '*'
    ...
    • Jun 1 2022, 4:43 PM
    • 19 Lines
  • 10:26:05 swh-scheduler@db1:5432=> with maven_lister_ids as (
    swh-scheduler(> select id from listers where name='maven'
    swh-scheduler(> ) select now(), visit_type, url
    swh-scheduler-> from
    swh-scheduler-> listed_origins lo1
    ...
    • Jun 1 2022, 10:46 AM
    • 28,051 Lines
  • ---
    layout: post
    title: "Software Heritage"
    description: "About Software Heritage, what we stand for, and where we are headed"
    date:
    ...
    • May 26 2022, 11:58 AM
    • 52 Lines
  • diff /tmp/tox-swh-web.now.txt /tmp/tox-swh-web-last-green.txt
    0a1,3
    > aiohttp==3.8.1
    > aiohttp-utils==3.1.1
    > aiosignal==1.2.0
    ...
    • May 25 2022, 5:33 PM
    • 123 Lines
  • [19/May/2022 08:29:58] [DEBUG] Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/rest_framework/views.py", line 483, in dispatch
    self.initial(request, *args, **kwargs)
    File "/usr/lib/python3/dist-packages/rest_framework/views.py", line 402, in initial
    self.check_throttles(request)
    ...
    • May 19 2022, 11:18 AM
    • 12 Lines
  • anlambert@carnavalet:/tmp$ python3
    Python 3.9.2 (default, Feb 28 2021, 17:03:44)
    [GCC 10.2.1 20210110] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import dulwich
    ...
    • May 17 2022, 1:43 PM
    • 56 Lines
  • [
    // Default query template (Don't run)
    {
    "query": {
    "bool": {
    ...
    • May 16 2022, 8:52 PM
    • 183 Lines
    • JSON
  • swh-scheduler=> select url from listed_origins where lister_id = 'ab79de00-a2d3-44ac-90a8-f53c35c5d893' and visit_type = 'git';
    -------------------------------------------------------------------------------------------------------------------------------------
    ${basedir}
    ${base.url}.git
    ${custom.gitprivate.url}simplerest.git
    ...
    • May 12 2022, 6:05 PM
    • 72 Lines
  • 11:53:48 swh-scheduler@db1:5432=> select * from task where type='list-maven-full';
    +-[ RECORD 1 ]-----+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | id | 31171944 |
    | type | list-maven-full |
    | arguments | {"args": [], "kwargs": {"url": "https://repo1.maven.org/maven2/", "index_url": "https://maven-exporter.internal.staging.swh.network/export-maven-central.fld"}} |
    ...
    • May 12 2022, 11:54 AM
    • 23 Lines
  • % join <( cat 2021-03-23/compressed/graph.edges.stats.txt | sort ) <( cat 2022-04-25/compressed/graph.edges.stats.txt | sort ) | while read t a b; do echo $t $(( 1. * b / a )); done
    dir:cnt 1.6619934106831602
    dir:dir 1.5117835202334333
    dir:rev 2.5607389960164033
    ori:snp 1.1175533201104195
    ...
    • May 8 2022, 12:32 PM
    • 22 Lines
  • seirl@vab ../softwareheritage/ssd/graph/2022-04-25 % cat compressed/graph.nodes.count.txt
    25340003875
    seirl@vab ../softwareheritage/ssd/graph/2022-04-25 % cat compressed/graph.nodes.stats.txt
    cnt 12213798094
    dir 10173900390
    ...
    • May 8 2022, 11:15 AM
    • 26 Lines
  • ~/dati/src/rust/coreutils/src/uu/sort   main  rg fork
    ✘  ~/dati/src/rust/coreutils/src/uu/sort   main  rg spawn
    src/check.rs
    41: thread::spawn({
    ...
    • May 6 2022, 4:50 PM
    • 12 Lines
  • select directory_id, from_utf8(name), type, target, perms from directory_entry where from_utf8(name) in ('.', '..', '') or strpos(from_utf8(name), '/') > 0
    "directory_id","","type","target","perms"
    "5002df6cc84b3245e9554f1b4ef286876cbb0856",".","dir","3dc4aa1834136cbff6f4451c677f8c18f28c56f2","16384"
    ...
    • May 4 2022, 5:01 PM
    • 545 Lines
  • seirl@vab ../softwareheritage/ssd/graph/2021-03-23 % du -shc --apparent-size compressed/* | sort -h :(
    9 compressed/graph.persons.count.txt
    11 compressed/graph.labels.count.txt
    12 compressed/graph.nodes.count.txt
    13 compressed/graph.edges.count.txt
    ...
    • Apr 29 2022, 3:11 PM
    • 49 Lines
  • softwareheritage=> select count(*) from (select url as child, (select count(*) from origin_visit where origin=origin.id) as child_visits, encode(metadata, 'escape')::json->'parent'->'full_name' as parent, (select count(*) from origin_visit, origin as o where origin_visit.origin=o.id and o.url= concat('https://github.com/'::text, (encode(metadata, 'escape')::json->'parent'->>'full_name')::text)) as parent_visits from raw_extrinsic_metadata, origin where digest(url, 'sha1')=decode(substring(target, 11), 'hex') and target>'swh:1:ori:' and target < 'swh:1:orz' and format='application/vnd.github.v3+json' and encode(metadata, 'escape') not like '%\\3%' and encode(metadata, 'escape') not like '%\\\\%' and encode(metadata, 'escape')::json->'parent'->'full_name' is not null) as t where t.child_visits=1 and t.parent_visits>0;
    ^[[A^[[A
    count
    -------
    10642
    ...
    • Apr 29 2022, 1:48 PM
    • 12 Lines
  • diff --git a/swh/loader/package/maven/loader.py b/swh/loader/package/maven/loader.py
    index d664e67..e2d9d05 100644
    --- a/swh/loader/package/maven/loader.py
    +++ b/swh/loader/package/maven/loader.py
    @@ -1,8 +1,10 @@
    ...
    • Apr 28 2022, 5:52 PM
    • 39 Lines
  • (swh) $ pip freeze | grep swh.core
    -e git+ssh://git@forge.softwareheritage.org/diffusion/DCORE/swh-core.git@d374b6002955bf3591c84335b6bcc3521ff78061#egg=swh.core
    (swh) $ pip show swh.core | grep Version:
    Version: 1.0.1.dev3+gd374b60
    • Apr 28 2022, 2:09 PM
    • 4 Lines
  • swhworker@worker3:~$ url=https://github.com/progval/Limnoria ; /usr/bin/time swh loader run git $url lister_name=github lister_instance_name=github
    WARNING:swh.core.cli:Could not load subcommand graph: DistributionNotFound(Requirement.parse('py4j'), None)
    INFO:swh.loader.git.loader.GitLoader:Load origin 'https://github.com/progval/Limnoria' with type 'git'
    INFO:swh.loader.git.loader:Listed 819 refs for repo https://github.com/progval/Limnoria
    {'status': 'uneventful'} for origin 'https://github.com/progval/Limnoria'
    ...
    • Apr 28 2022, 1:34 PM
    • 28 Lines
  • listed origins:
    ```
    10:24:59 swh@db1:5432=> select now(), count(*) from origin where url like 'https://repo1.maven.org/maven2/%'
    swh-> ;
    +------------------------------+--------+
    ...
    • Apr 28 2022, 10:31 AM
    • 31 Lines
  • $ cat example.py
    import contextlib
    @contextlib.contextmanager
    def f():
    ...
    • Apr 26 2022, 11:54 AM
    • 21 Lines
  • #!/usr/bin/env bash
    # set -x
    export KUBECONFIG=staging-workers.yaml
    ...
    • Apr 26 2022, 11:28 AM
    • 26 Lines
  • ERROR:swh.dataset.journalprocessor:Exporter ORCExporter: error while exporting the object: {'origin': 'https://www.npmjs.com/package/@reactionaries/hookd', 'visit': 170, 'date': datetime.datetime(2020, 9, 15, 18, 11, 37, 416264
    , tzinfo=datetime.timezone.utc), 'status': 'full', 'snapshot': b'\x15\xa3i\x0c\x05\x86PT\x19\\_~k7\x1cY$V\xdb\xe5', 'metadata': None}
    Traceback (most recent call last):
    File "/home/seirl/swh-environment/swh-dataset/swh/dataset/journalprocessor.py", line 467, in process_message
    [message.key()],
    ...
    • Apr 25 2022, 2:23 PM
    • 31 Lines
  • swh  tony  yavin4  ~  work  …  isos  libvirt  debian   master  1+  %  sha512sum -c swh-debian-zfs-11.3-amd64-20220422-1444.qcow2.sha512
    swh-debian-zfs-11.3-amd64-20220422-1444.qcow2: OK
    swh  tony  yavin4  ~  work  …  isos  libvirt  debian   master  2+  %  ls -lah
    total 841M
    drwxr-xr-x 2 tony tony 16 Apr 22 17:00 .
    ...
    • Apr 22 2022, 5:06 PM
    • 132 Lines
  • swh  tony  yavin4  ~  work  …  isos  libvirt  debian   master  %  ls -lah
    total 576M
    drwxr-xr-x 2 tony tony 8 Apr 22 16:08 .
    drwxr-xr-x 3 tony tony 3 Mar 12 19:25 ..
    lrwxrwxrwx 1 tony tony 203 May 18 2021 swh-debian-10.6-amd64-20201012-1352.qcow2 -> ../../../.git/annex/objects/6M/K7/SHA256E-s540454212--96a6280841959cf9be85ee5d07afa7cf53d31ced56369cb66c536bd1b61f2d7c/SHA256E-s540454212--96a6280841959cf9be85ee5d07afa7cf53d31ced56369cb66c536bd1b61f2d7c
    ...
    • Apr 22 2022, 4:13 PM
    • 442 Lines
  • packer build debian_bullseye.qemu.json
    qemu: output will be in this color.
    ==> qemu: Retrieving ISO
    ==> qemu: Trying https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-11.3.0-amd64-netinst.iso
    ...
    • Apr 22 2022, 3:54 PM
    • 686 Lines
  • 14:06 <+moranegg> olasd: (or any other git annex expert) I have done a git sync to add one pdf and seems lot of stuff has happened during this sync, Is there a way to check that this is normal and that I didn't break anything?
    14:08 <+olasd> there's a bunch of pdfs that are in the git repo instead of the annex (events/2021-11-30-swh5years/community-meetup/) but that commit was in january
    14:11 <+moranegg> interesting, I had in January other issues
    14:12 <+olasd> I managed a git annex get of the pdf you've added just fine
    14:15 <+moranegg> great, I hope I haven't deleted anything with the sync
    ...
    • Apr 22 2022, 2:41 PM
    • 16 Lines
  • if has_add_forge_now_permission(request.user):
    for item in page.object_list:
    history = AddForgeNowRequestHistory.objects.filter(request=item ).filter(actor_role = "MODERATOR").order_by(
    "id"
    )
    ...
    • Apr 21 2022, 9:39 PM
    • 18 Lines
  • packer build debian_bullseye_zfs.json
    proxmox: output will be in this color.
    ==> proxmox: Retrieving ISO
    ==> proxmox: Trying https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-11.3.0-amd64-netinst.iso
    ...
    • Apr 21 2022, 4:06 PM
    • 1,671 Lines
  • packer build debian_bullseye.json
    proxmox: output will be in this color.
    ==> proxmox: Retrieving ISO
    ==> proxmox: Trying https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-11.3.0-amd64-netinst.iso
    ...
    • Apr 21 2022, 3:30 PM
    • 1,189 Lines
  • root@elastic-worker0:~# puppet agent --test
    Info: Using configured environment 'staging'
    Info: Retrieving pluginfacts
    Info: Retrieving plugin
    Info: Retrieving locales
    ...
    • Apr 20 2022, 11:54 AM
    • 47 Lines
  • softwareheritage-scheduler=> select url, last_scheduled, last_visit from origin_visit_stats where url = 'http://svn.code.sf.net/p/csreadgen/code/';
    url | last_scheduled | last_visit
    ------------------------------------------+-------------------------------+-------------------------------
    http://svn.code.sf.net/p/csreadgen/code/ | 2022-04-13 09:42:21.829297+00 | 2022-04-13 09:42:26.456121+00
    (1 row)
    • Apr 13 2022, 5:27 PM
    • 5 Lines
  • Traceback (most recent call last):
    File "/home/seirl/swh-environment/.venv/bin/swh", line 33, in <module>
    sys.exit(load_entry_point('swh.core', 'console_scripts', 'swh')())
    File "/home/seirl/swh-environment/swh-core/swh/core/cli/__init__.py", line 185, in main
    return swh(auto_envvar_prefix="SWH")
    ...
    • Apr 12 2022, 5:33 PM
    • 49 Lines
  • journal:
    brokers:
    - kafka1.internal.softwareheritage.org:9094
    - kafka2.internal.softwareheritage.org:9094
    - kafka3.internal.softwareheritage.org:9094
    ...
    • Apr 12 2022, 5:27 PM
    • 11 Lines
  • softwareheritage=> SELECT count(*) FROM origin_visit ov INNER JOIN origin o ON o.id = ov.origin WHERE o.url = 'https://subversion.renater.fr/anonscm/svn/panda';
    count
    --------
    132706
    (1 row)
    • Apr 12 2022, 12:02 PM
    • 5 Lines
  • class SupportsObjectType(Protocol):
    object_type: str
    _L: List[SupportsObjectType] = [
    Person,
    ...
    • Apr 11 2022, 10:29 AM
    • 29 Lines
    • Python
  • Traceback (most recent call last):
    File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
    File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
    ...
    • Apr 8 2022, 5:37 PM
    • 26 Lines
  • 13:42 $ swh search -C ~/.config/swh/search.yml rpc-serve
    * Serving Flask app 'swh.search.api.server' (lazy loading)
    * Environment: production
    WARNING: This is a development server. Do not use it in a production deployment.
    Use a production WSGI server instead.
    ...
    • Apr 7 2022, 1:45 PM
    • 35 Lines
  • P1333 query 3
    query test {
    origin(url: "https://github.com/python/cpython") {
    url
    id
    visits(first: 2, after: "Mg==") {
    ...
    • Apr 6 2022, 6:01 PM
    • 50 Lines
  • P1332 query 2
    query rels {
    release(SWHId: "d28343dc3ad53a411ae3685e7d6a7866c8c22d6b") {
    id
    date
    name
    ...
    • Apr 6 2022, 5:58 PM
    • 15 Lines
  • P1331 query1
    query snapshots {
    snapshot(SWHId: "6f1e3067acba64891af07680367a38cc1b9cb57a") {
    branches(first: 10) {
    nodes {
    name
    ...
    • Apr 6 2022, 5:57 PM
    • 31 Lines
  • #!/usr/bin/env bash
    set -xe
    DBPORT=5432
    ...
    • Apr 6 2022, 4:36 PM
    • 15 Lines
  • $ sudo pidof dnsmasq
    240165 240164 240049 240048
    $ ls /var/lib/libvirt
    boot dnsmasq images qemu sanlock
    $ ls /var/lib/libvirt/dnsmasq
    ...
    • Apr 4 2022, 5:08 PM
    • 25 Lines
  • version: '2'
    services:
    ...
    ...
    • Apr 4 2022, 1:30 PM
    • 8 Lines
  • diff --git java/src/main/java/org/softwareheritage/graph/compress/LabelMapBuilder.java java/src/main/java/org/softwareheritage/graph/compress/LabelMapBuilder.java
    index 600870e..0825266 100644
    --- java/src/main/java/org/softwareheritage/graph/compress/LabelMapBuilder.java
    +++ java/src/main/java/org/softwareheritage/graph/compress/LabelMapBuilder.java
    @@ -143,7 +143,7 @@ public class LabelMapBuilder {
    ...
    • Apr 1 2022, 4:45 PM
    • 104 Lines
    • Diff
  • query testquery {
    origin(url: "https://github.com/python/cpython") {
    url
    id
    visits(first: 2, after: "Mg==") {
    ...
    • Apr 1 2022, 3:36 PM
    • 45 Lines
  • (swh) parallels@ubuntu-linux-20-04-desktop:~/Desktop/swh/swh-environment/docker$ sudo docker-compose restart
    Restarting docker_swh-web-cron_1 ...
    Restarting docker_swh-vault-worker_1 ...
    Restarting docker_swh-indexer-journal-client_1 ...
    Restarting docker_swh-lister_1 ...
    ...
    • Apr 1 2022, 10:02 AM
    • 304 Lines
  • diff --git a/swh/web/admin/deposit.py b/swh/web/admin/deposit.py
    index ddbfe304..1d653f36 100644
    --- a/swh/web/admin/deposit.py
    +++ b/swh/web/admin/deposit.py
    @@ -3,21 +3,18 @@
    ...
    • Mar 30 2022, 6:04 PM
    • 132 Lines
  • diff --git a/swh/deposit/api/private/deposit_list.py b/swh/deposit/api/private/deposit_list.py
    index 4bb40e31..e8c07e31 100644
    --- a/swh/deposit/api/private/deposit_list.py
    +++ b/swh/deposit/api/private/deposit_list.py
    @@ -3,10 +3,20 @@
    ...
    • Mar 30 2022, 6:03 PM
    • 162 Lines
  • # swh-core
    diff --git a/swh/core/db/common.py b/swh/core/db/common.py
    index 65e6d75..6f89058 100644
    --- a/swh/core/db/common.py
    +++ b/swh/core/db/common.py
    ...
    • Mar 30 2022, 1:45 PM
    • 76 Lines
  • make test-frontend-ui-cmd
    rm -f swh-web-test.sqlite3
    django-admin migrate --settings=swh.web.settings.tests
    Operations to perform:
    Apply all migrations: admin, auth, contenttypes, sessions, swh_web_add_forge_now, swh_web_auth, swh_web_common
    ...
    • Mar 29 2022, 5:50 PM
    • 49 Lines
  • ERROR: Command errored out with exit status 1:
    command: /home/parallels/.virtualenvs/swh/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-0_w3ghla/confluent-kafka/setup.py'"'"'; __file__='"'"'/tmp/pip-install-0_w3ghla/confluent-kafka/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-tx1t5lrk
    cwd: /tmp/pip-install-0_w3ghla/confluent-kafka/
    Complete output (116 lines):
    running bdist_wheel
    ...
    • Mar 28 2022, 10:14 PM
    • 122 Lines
  • codemeta:identifier does not allow propertyvalues (source: https://github.com/codemeta/codemeta/blob/master/codemeta.jsonld#L36 ; `"@type": "@id"` means the value must be a URI, but propertyvalues are not URIs), so we need to replace:
    <codemeta:identifier>
    <codemeta:type>schema:PropertyValue</codemeta:type>
    <schema:propertyID>HAL-ID</schema:propertyID>
    ...
    • Mar 28 2022, 4:26 PM
    • 15 Lines
  • 11:14 <KShivendu> That's great! Now try fixing some easy hacks and write a draft for your proposal and share it with us :)
    12:59 <KShivendu> Hey vsellier can you please tell me about how we have deployed the elasticsearch cluster. for example: what kind of a machine(s) is it, and where is it hosted?
    13:00 <KShivendu> also, did we face any challenge in this because of the scale? if yes, please briefly describe them :)
    13:25 <+ardumont> KShivendu: where services are exposed is described in https://docs.softwareheritage.org/sysadm/network-architecture/service-urls.html
    13:25 <+ardumont> ah he is not here
    ...
    • Mar 28 2022, 1:53 PM
    • 11 Lines
  • describe('Browse requests list tests', function() {
    beforeEach(function() {
    this.addForgeNowUrl = this.Urls.forge_add();
    this.listAddForgeRequestsUrl = this.Urls.add_forge_request_list_datatables();
    });
    ...
    • Mar 25 2022, 3:02 PM
    • 48 Lines
  • softwareheritage=> SELECT DISTINCT ON (ov.visit) ov.visit, ov.type, ov.date, ovs.date as status_date, ovs.status
    FROM origin_visit ov
    INNER JOIN origin o ON o.id = ov.origin
    INNER JOIN origin_visit_status ovs USING (origin, visit)
    WHERE o.url = 'https://github.com/python/cpython'
    ...
    • Mar 25 2022, 2:27 PM
    • 78 Lines
  • diff --git a/CHANGELOG.rst b/CHANGELOG.rst
    index e417bcd..198150f 100644
    --- a/CHANGELOG.rst
    +++ b/CHANGELOG.rst
    @@ -57,7 +57,7 @@ Added
    ...
    • Mar 22 2022, 4:43 PM
    • 148 Lines
    • Diff
  • new file cypress/fixtures/add-forge-now-create-request-fail.json
    @@ -0,0 +1,8 @@
    +{
    + "status": 400,
    + "ok": false,
    ...
    • Mar 18 2022, 3:13 PM
    • 43 Lines
  • make test-frontend-ui-cmd
    rm -f swh-web-test.sqlite3
    django-admin migrate --settings=swh.web.settings.tests -v0 2>/dev/null
    cat swh/web/tests/create_test_admin.py | django-admin shell --settings=swh.web.settings.tests
    cat swh/web/tests/create_test_users.py | django-admin shell --settings=swh.web.settings.tests
    ...
    • Mar 17 2022, 11:08 AM
    • 62 Lines
  • swh/web/tests/create_test_users.py:
    ```
    from swh.web.auth.utils import SWH_AMBASSADOR_PERMISSION, SWH_MODERATOR_PERMISSION
    ...
    • Mar 15 2022, 11:11 AM
    • 106 Lines
  • softwareheritage=> explain select ov.* from origin_visit ov INNER JOIN origin o ON o.id = ov.origin WHERE o.url = 'svn://scm.gforge.inria.fr/svnroot/fpg-ipc/' ORDER BY ov.visit DESC LIMIT 1;
    QUERY PLAN
    ----------------------------------------------------------------------------------------------------------------
    Limit (cost=633.57..633.57 rows=1 width=28)
    -> Sort (cost=633.57..633.59 rows=7 width=28)
    ...
    • Mar 11 2022, 11:53 AM
    • 29 Lines
  • [alias]
    checkout-from-staging = "!f() { git fetch -nf https://forge.softwareheritage.org/source/staging.git phabricator/diff/$1:diff/$1 && git checkout diff/$1; }; f"
    • Mar 8 2022, 2:33 PM
    • 2 Lines
  • query test {
    origin(url: "https://github.com/python/cpython") {
    id
    url
    visits(first: 2) {
    ...
    • Mar 8 2022, 9:16 AM
    • 46 Lines
  • softwareheritage=> \copy (select encode(metadata, 'escape') from raw_extrinsic_metadata where target='swh:1:dir:5d6f2287a9e2e08aaad3f31f833c82d50bcf3c88') to stdout
    <
    ?
    x
    m
    ...
    • Mar 2 2022, 2:48 PM
    • 71 Lines
  • diff --git a/swh/objstorage/backends/azure.py b/swh/objstorage/backends/azure.py
    index be7aeaa..80d8fba 100644
    --- a/swh/objstorage/backends/azure.py
    +++ b/swh/objstorage/backends/azure.py
    @@ -78,12 +78,23 @@ def get_container_url(
    ...
    • Feb 28 2022, 5:06 PM
    • 28 Lines
    • Diff
  • ✘ dev@desktop5  ~/swh-environment/swh-deposit   validate-metadata ●  pip3 show xmlschema
    Name: xmlschema
    Version: 1.9.2
    Summary: An XML Schema validator and decoder
    Home-page: https://github.com/sissaschool/xmlschema
    ...
    • Feb 24 2022, 6:35 PM
    • 20 Lines
  • ### Strucure of answer with short answer
    <details>
    <summary> use this template for dropdown </summary>
    <br>
    ...
    • Feb 23 2022, 5:01 PM
    • 7 Lines
  • class SwhGraphTinkerpopProvider extends WebGraphTinkerpopProvider {
    @Override
    public ImmutableGraph load() {
    var graph = SwhBidirectionalImmutableGraph.loadLabelled(...);
    graph.loadCommitAuthors(LoadMethod.MAPPED);
    ...
    • Feb 23 2022, 4:32 PM
    • 19 Lines
    • Java
  • Error: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install python3-swh.web' returned 100: Reading package lists...
    Building dependency tree...
    Reading state information...
    Some packages could not be installed. This may mean that you have
    requested an impossible situation or if you are using the unstable
    ...
    • Feb 23 2022, 3:29 PM
    • 24 Lines
  • $ echo "world" > "<string>"
    $ python3 -c "hello"
    Traceback (most recent call last):
    File "<string>", line 1, in <module>
    world
    ...
    • Feb 23 2022, 11:59 AM
    • 6 Lines
  • query myquery {
    origin(url: "https://github.com/python/cpython") {
    url
    id
    visits(first: 2) {
    ...
    • Feb 23 2022, 10:51 AM
    • 27 Lines
  • swh  tony  yavin4  ~  work  …  sysadm-provisioning  azure  terraform   master  2⬆  8⚑  %  terraform apply
    azurerm_resource_group.euwest-vault: Refreshing state... [id=/subscriptions/49b7f681-8efc-4689-8524-870fc0c1db09/resourceGroups/euwest-vault]
    azurerm_resource_group.euwest-cassandra: Refreshing state... [id=/subscriptions/49b7f681-8efc-4689-8524-870fc0c1db09/resourceGroups/euwest-cassandra]
    azurerm_storage_account.vault-storage: Refreshing state... [id=/subscriptions/49b7f681-8efc-4689-8524-870fc0c1db09/resourceGroups/euwest-vault/providers/Microsoft.Storage/storageAccounts/swhvaultstorage]
    azurerm_network_interface.storage-interface["storage01"]: Refreshing state... [id=/subscriptions/49b7f681-8efc-4689-8524-870fc0c1db09/resourceGroups/euwest-servers/providers/Microsoft.Network/networkInterfaces/storage01-interface]
    ...
    • Feb 18 2022, 4:33 PM
    • 1,094 Lines
  • ipython
    Python 3.9.2 (default, Feb 28 2021, 17:03:44)
    Type 'copyright', 'credits' or 'license' for more information
    IPython 7.27.0 -- An enhanced Interactive Python. Type '?' for help.
    ...
    • Feb 16 2022, 4:14 PM
    • 235 Lines
  • [2022-02-15 13:42:59,440: INFO/MainProcess] Task swh.loader.cvs.tasks.LoadCvsRepository[9ea64ca6-744b-46b0-a5dd-754ad1a7ae99] received
    swh-loader_1 | [2022-02-15 13:43:02,110: INFO/ForkPoolWorker-1] Load origin 'pserver://anonymous:anonymous@a.cvs.sourceforge.net/cvsroot/mx4j/mx4j' with type 'cvs'
    swh-loader_1 | Request: BEGIN AUTH REQUEST
    swh-loader_1 | /cvsroot/mx4j
    swh-loader_1 | anonymous
    ...
    • Feb 15 2022, 2:46 PM
    • 22 Lines
  • $ psql service=staging-swh-deposit -c '\x' -c 'select now(), * from deposit order by id desc limit 5;'
    ```
    +-------------------------------+-----+-------------------------------+-------------------------------+------------------------------------------+----------------------------------------------------+--------+-----------+---------------+-----------+---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+--------------+---------------------------------------------------------------------------+
    | now | id | reception_date | complete_date | external_id | swhid | status | client_id | collection_id | parent_id | status_detail | swhid_context | check_task_id | load_task_id | origin_url |
    ...
    • Feb 15 2022, 10:33 AM
    • 16 Lines