Page MenuHomeSoftware Heritage
Paste Active Pastes
  • # Copyright (C) 2015-2020 The Software Heritage developers
    # See the AUTHORS file at the top-level directory of this distribution
    # License: GNU General Public License version 3, or any later version
    # See top-level LICENSE file for more information
    ...
    • Tue, Nov 24, 2:50 PM
    • 456 Lines
    • Python
  • (jupyterhub) ddouard@desktop6:~/src/repo2docker$ repo2docker --config cfg.json --image-name test:pystuff --user-name swh --user-id 1000 swh:1:rev:94dca98c006b80309704c717b5d83dff3c1fa3a0
    Picked Swhid content provider.
    Fetching revision 94dca98c006b80309704c717b5d83dff3c1fa3a0 from https://archive.softwareheritage.org/api/1/revision/94dca98c006b80309704c717b5d83dff3c1fa3a0/
    Fetching directory f557a108b5878c7d7a46e94f538d6bb65379e62e from https://archive.softwareheritage.org/api/1/directory/f557a108b5878c7d7a46e94f538d6bb65379e62e/
    Fetching /tmp/repo2docker4onz44zc/.travis.yml from https://archive.softwareheritage.org/api/1/content/sha1_git:eb1b383f0a3ee7a983df05604a5b454448c28d2c/raw/
    ...
    • Tue, Nov 24, 2:18 PM
    • 152 Lines
  • (python:.venv) ↪ tox -e py3 -- -v
    GLOB sdist-make: /home/haltode/work/swh/swh-environment/swh-fuse/setup.py
    py3 inst-nodeps: /home/haltode/work/swh/swh-environment/swh-fuse/.tox/.tmp/package/1/swh.fuse-0.9.2.dev3+gbb4e535.zip
    py3 installed: aiosqlite==0.16.0,async-generator==1.10,attrs==20.3.0,attrs-strict==0.2.0,certifi==2020.11.8,chardet==3.0.4,click==7.1.2,coverage==5.3,Deprecated==1.2.10,docutils==0.16,hypothesis==5.41.2,idna==2.10,iniconfig==1.1.1,iso8601==0.1.13,lockfile==0.12.2,outcome==1.1.0,packaging==20.4,pluggy==0.13.1,psutil==5.7.3,py==1.9.0,pyfuse3==3.1.1,pyparsing==2.4.7,pytest==6.1.2,pytest-cov==2.10.1,python-daemon==2.2.4,python-dateutil==2.8.1,PyYAML==5.3.1,requests==2.25.0,requests-mock==1.8.0,sentry-sdk==0.19.3,six==1.15.0,sniffio==1.2.0,sortedcontainers==2.3.0,swh.core==0.8.0,swh.fuse @ file:///home/haltode/work/swh/swh-environment/swh-fuse/.tox/.tmp/package/1/swh.fuse-0.9.2.dev3%2Bgbb4e535.zip,swh.model==0.9.0,swh.web.client==0.2.2,toml==0.10.2,trio==0.17.0,typing-extensions==3.7.4.3,urllib3==1.26.2,wrapt==1.12.1
    py3 run-test-pre: PYTHONHASHSEED='3708312725'
    ...
    • Tue, Nov 24, 1:39 PM
    • 69 Lines
  • diff --git a/swh/core/db/tests/test_db.py b/swh/core/db/tests/test_db.py
    index bb7f80b..6b7529c 100644
    --- a/swh/core/db/tests/test_db.py
    +++ b/swh/core/db/tests/test_db.py
    @@ -7,11 +7,8 @@ from dataclasses import dataclass
    ...
    • Mon, Nov 23, 6:07 PM
    • 147 Lines
  • Nov 20 11:30:14 saam python3[2819769]: 2020-11-20 11:30:14 [2819769] root:ERROR __init__() got an unexpected keyword argument 'id'
    Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/swh/core/api/serializers.py", line 272, in msgpack_loads
    strict_map_key=False,
    File "msgpack/_unpacker.pyx", line 161, in msgpack._unpacker.unpackb
    ...
    • Fri, Nov 20, 12:33 PM
    • 52 Lines
  • $ curl -X PUT "localhost:9200/origin/"
    {"acknowledged":true,"shards_acknowledged":true,"index":"origin"}
    $ curl -X PUT "localhost:9200/origin/_mapping" -H 'Content-Type: application/json' -d'
    ...
    • Fri, Nov 20, 12:25 PM
    • 511 Lines
  • $ curl -X PUT "localhost:9200/origin/"
    {"acknowledged":true,"shards_acknowledged":true,"index":"origin"}
    $ curl -X PUT "localhost:9200/origin/_mapping" -H 'Content-Type: application/json' -d'
    ...
    • Fri, Nov 20, 12:24 PM
    • 432 Lines
  • root@worker01:~# SWH_CONFIG_FILENAME=/etc/softwareheritage/loader_npm.yml swh loader run npm https://www.npmjs.com/vue
    Traceback (most recent call last):
    File "/usr/bin/swh", line 11, in <module>
    load_entry_point('swh.core==0.8.0', 'console_scripts', 'swh')()
    File "/usr/lib/python3/dist-packages/swh/core/cli/__init__.py", line 135, in main
    ...
    • Fri, Nov 20, 12:09 PM
    • 27 Lines
  • (swh) zack@noziere:~/swh-environment/swhfs/archive/swh:1:rev:9d76c0b163675505d1a901e5fe5249a2c55609bc$ ls -f history/by-date/2009/12/
    31 22 30 21 19 18 17 15 16 14 11 10 09 08 07 06 05 01 04 03 02
    (swh) zack@noziere:~/swh-environment/swhfs/archive/swh:1:rev:9d76c0b163675505d1a901e5fe5249a2c55609bc$ ls -f history/by-date/2009/12/31
    (swh) zack@noziere:~/swh-environment/swhfs/archive/swh:1:rev:9d76c0b163675505d1a901e5fe5249a2c55609bc$ ls -f history/by-date/2009/12/31
    (swh) zack@noziere:~/swh-environment/swhfs/archive/swh:1:rev:9d76c0b163675505d1a901e5fe5249a2c55609bc$ ls -f history/by-date/2009/12/01
    ...
    • Thu, Nov 19, 11:30 AM
    • 6 Lines
  • Hello,
    We recently opened the staging archive [1] and deposit [2] to the world.
    Note that it will be mostly used to preview new features and allow third
    ...
    • Tue, Nov 17, 10:16 AM
    • 32 Lines
  • id | reception_date | complete_date | external_id | swhid | status | client_id | collection_id | parent_id | status_detail | swhid_context | check_task_id | load_task_id
    ----+-------------------------------+-------------------------------+--------------------------------------+----------------------------------------------------+---------+-----------+---------------+-----------+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+--------------
    21 |...
    • Mon, Nov 16, 3:41 PM
    • 5 Lines
  • n.n.n / 2020-11-13
    ==================
    * cache: replace asizeof() call with simpler heuristic
    ...
    • Fri, Nov 13, 5:19 PM
    • 11 Lines
  • $ cat .config/swh/global.yml
    swh:
    fuse:
    web-api:
    url: "https://archive.internal.softwareheritage.org/api/1/"
    • Thu, Nov 12, 1:16 PM
    • 5 Lines
  • root@pergamon:~# puppet agent --test
    Info: Using configured environment 'production'
    Info: Retrieving pluginfacts
    Info: Retrieving plugin
    Info: Retrieving locales
    ...
    • Mon, Nov 9, 12:14 PM
    • 17 Lines
  • (cleaned up)
    22:58 <swhbot> icinga PROBLEM: service swh deposit on rp0.internal.staging.swh.network is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 395 bytes in 0.149 second response time
    23:55 <swhbot> icinga PROBLEM: service swh webapp revision log end to end on webapp.internal.staging.swh.network is CRITICAL: HTTP CRITICAL: HTTP/1.1 404 Not Found - pattern not found - 10146 bytes in 0.010 second response time
    09:43 <swhbot> icinga PROBLEM: service journalbeat on rp0.internal.staging.swh.network is WARNING: JOURNAL LAG WARNING - monotonic_lag is 1363s (outside range 0:1200)
    ...
    • Sat, Nov 7, 4:56 PM
    • 14 Lines
  • ______________________________________________________________________________________ test_api_endpoints_have_cors_header _______________________________________________________________________________________
    client = <django.test.client.Client object at 0x7f0a94071a20>
    @given(directory(), revision())
    ...
    • Fri, Nov 6, 4:13 PM
    • 58 Lines
  •  ddouard  (e) swh   master ✚ 1 ⚑ 1  …/s/s/docs  make
    sphinx-apidoc --ext-viewcode --separate -o apidoc .. . ../setup.py ../swh/*/tests ../swh/*/tests/* ../swh/*/*/tests/* ../swh/*/*/*/tests/* ../swh/*/migrations ../swh/*/migrations/* ../swh/*/*/migrations/* ../swh/*/*/*/migrations/* ../bin/* ../utils/* ../node_modules/* ../swh/*/settings/*
    Creating file apidoc/conftest.rst.
    Creating file apidoc/swh.rst.
    Creating file apidoc/swh.deposit.rst.
    ...
    • Fri, Nov 6, 10:02 AM
    • 95 Lines
  •  ddouard  (e) swh   master ✚ 1 ⚑ 1  …/s/s/docs  make
    sphinx-build -b html . _build/html
    Running Sphinx v3.3.0
    loading translations [en]... done
    WARNING: logo file '_static/software-heritage-logo-title-motto-vertical-white.png' does not exist
    ...
    • Fri, Nov 6, 10:00 AM
    • 159 Lines
  • diff buster-preseed.cfg buster-preseed.vagrant.cfg
    296c296
    < grub-pc grub-pc/install_devices multiselect /dev/vda
    ---
    > grub-pc grub-pc/install_devices multiselect /dev/disk/by-id/ata-VBOX_HARDDISK_VBbc5508de-d37f68c1
    ...
    • Thu, Nov 5, 4:50 PM
    • 23 Lines
  • @startuml
    title Webapp and Deposit roles with and without reverse proxy
    note top of role_swh_rp_webapp : production webapp0 (azure)
    note top of role_swh_rp_webapps : production moma (rocq)
    ...
    • Thu, Nov 5, 1:54 PM
    • 25 Lines
  • @startuml
    title Webapp and Deposit roles with and without reverse proxy
    role_swh_base_api *-- profile_swh_webapp
    note top of role_swh_base_api: - production webapp0 (azure)\n- staging webapp (rocq)
    ...
    • Thu, Nov 5, 1:53 PM
    • 19 Lines
  • (swh) ✘  ~/swh-environment/swhfs/archive  time (lf | wc -l)
    519
    ( ls --color=tty -f | wc -l; ) 0,01s user 0,00s system 4% cpu 0,362 total
    (swh) ~/swh-environment/swhfs/archive  time (lf | wc -l)
    519
    ...
    • Thu, Nov 5, 10:28 AM
    • 12 Lines
  • @startuml
    title Staging Reverse Proxy
    participant User
    box "Firewall" #Orange
    ...
    • Wed, Nov 4, 3:39 PM
    • 51 Lines
  • @startuml
    participant User
    box "webserver: moma (webapp, deposit), webapp.staging, deposit.staging..." #LightBlue
    participant FrontendHitch
    participant Varnish
    ...
    • Wed, Nov 4, 2:02 PM
    • 41 Lines
  • Selecting previously unselected package linux-image-5.9.0-fusebuf-amd64-unsigned.
    (Reading database ... 478590 files and directories currently installed.)
    Preparing to unpack linux-image-5.9.0-fusebuf-amd64-unsigned_5.9.1-2_amd64.deb ...
    Unpacking linux-image-5.9.0-fusebuf-amd64-unsigned (5.9.1-2) ...
    Setting up linux-image-5.9.0-fusebuf-amd64-unsigned (5.9.1-2) ...
    ...
    • Wed, Nov 4, 10:25 AM
    • 56 Lines
  • apt-get source linux
    cd linux-*/
    $EDITOR debian/config/defines
    # change "debug-info:" line to "false"
    ...
    • Tue, Nov 3, 8:08 PM
    • 14 Lines
  • softwareheritage-scheduler=> \x
    softwareheritage-scheduler=> select * from task where type='index-revision-metadata' limit 10;
    -[ RECORD 1 ]----+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------$
    id | 114244463
    type | index-revision-metadata
    ...
    • Tue, Nov 3, 2:44 PM
    • 66 Lines
  • -[ RECORD 1 ]----+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------$
    id | 114244463
    type | index-revision-metadata
    arguments | {"args": [], "kwargs": {"ids": ["2ade866be83193ef461c8f661bbbaa8cea3a6ac5"], "next_step": {"type": "indexer_origin_metadata", "policy": "oneshot", "arguments": {"args": [], "kwargs": {"origin_head": {"14342257": "2ade86$
    next_run | 2019-01-29 11:11:29.196488+00
    ...
    • Tue, Nov 3, 2:42 PM
    • 50 Lines
  • import asyncio
    from typing import AsyncGenerator, AsyncIterator
    async def gen() -> AsyncGenerator[int, None]:
    ...
    • Tue, Nov 3, 9:48 AM
    • 22 Lines
    • Python
  • delete from raw_extrinsic_metadata
    where id >= 'swh:1:snp:0000000000000000000000000000000000000000' and id < 'swh:1:snp:1111111111111111111111111111111111111111'
    and (format = 'replicate-npm-package-json'
    or format = 'pypi-project-json');
    ...
    • Mon, Nov 2, 5:20 PM
    • 74 Lines
  • 14:16 $ ./bin/deploy-on --apt --test moma.internal.softwareheritage.org
    *** swh-deploy: deploying recipes on pergamon.internal.softwareheritage.org...
    INFO -> Using Puppetfile '/etc/puppet/code/environments/production/Puppetfile'
    INFO -> Using Puppetfile '/etc/puppet/code/environments/staging/Puppetfile'
    INFO -> Deploying environment /etc/puppet/code/environments/production
    ...
    • Mon, Nov 2, 2:18 PM
    • 133 Lines
  • @startuml arch-context
    ' !includeurl https://raw.githubusercontent.com/RicardoNiepel/C4-PlantUML/master/C4_Container.puml
    !include /home/zack/dati/src/plantuml/C4-PlantUML/C4_Container.puml
    ' title System Context diagram for Software Heritage Filesystem (SwhFS)
    ...
    • Sat, Oct 31, 5:50 PM
    • 19 Lines
  • swh-deposit_1 | [2020-10-28 13:52:51 +0000] [34] [DEBUG] GET /deposit/1/servicedocument/
    swh-deposit_1 | [2020-10-28 13:52:51 +0000] [34] [DEBUG] Closing connection.
    swh-deposit_1 | [2020-10-28 13:52:51 +0000] [35] [DEBUG] POST /deposit/1/test/
    swh-deposit_1 | [2020-10-28 13:52:52 +0000] [35] [DEBUG] Closing connection.
    swh-deposit_1 | [2020-10-28 13:52:55 +0000] [35] [DEBUG] GET /1/private/test/1/check/
    ...
    • Wed, Oct 28, 4:40 PM
    • 12 Lines
  • 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
    ...
    • Wed, Oct 28, 10:52 AM
    • 44 Lines
  • 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
    ...
    • Wed, Oct 28, 10:49 AM
    • 44 Lines
  • 8: veth61ae6d9@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-d0d7f9beaec2 state UP group default
    link/ether 42:03:28:76:3b:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::4003:28ff:fe76:3b03/64 scope link
    valid_lft forever preferred_lft forever
    • Wed, Oct 28, 8:57 AM
    • 4 Lines
  • [Question] getdents() calls use very small buffer when FUSE mount
    Hello all!
    I am currently working on a FUSE filesystem where a directory can
    ...
    • Tue, Oct 27, 10:07 AM
    • 32 Lines
  • on master:
    Impossible to list history/ for more than a few hundreds commits because ls
    getdents() syscalls are done with a quite small buffer (19 entries on my
    machine), hence the readdir is regularly called with a different offset and the
    ...
    • Mon, Oct 26, 5:15 PM
    • 15 Lines
  • $ doco exec swh-scheduler swh scheduler task add load-git -p oneshot url=https://github.com/python/cpython
    $ doco logs -f --tail 0 | grep -vE "heartbeat|statsd"
    swh-scheduler-runner_1 | INFO:swh.scheduler.celery_backend.runner:Grabbed 1 tasks load-git
    amqp_1 |
    amqp_1 | =INFO REPORT==== 26-Oct-2020::14:58:45 ===
    ...
    • Mon, Oct 26, 4:28 PM
    • 748 Lines
  • [swh]
    dbname=softwareheritage
    host=db.internal.softwareheritage.org
    user=guest
    • Mon, Oct 26, 2:29 PM
    • 4 Lines
  • class Bar:
    def __init__(self):
    self.x = randint(1, 1000)
    self.y = [randint(1, 1000) for _ in range(100)]
    ...
    • Mon, Oct 26, 1:59 PM
    • 23 Lines
    • Python
  • pympler asizeof 25880
    vlorentz total_size 28
    original total_size 604
    pympler asizeof 32648
    ...
    • Mon, Oct 26, 1:08 PM
    • 23 Lines
  • swh  swh  tony  yavin4  ~  work  …  repo  swh  swh-environment  master  15+  SIGINT  %  doco exec swh-vault-worker /bin/bash
    swh@9d500dd5f25e:/$ python
    Python 3.7.4 (default, Jul 13 2019, 14:04:11)
    [GCC 8.3.0] on linux
    ...
    • Oct 22 2020, 7:57 PM
    • 34 Lines
  • 480722 activity
    480655 reviews
    480634 summary
    480140 support
    480088 files
    ...
    • Oct 22 2020, 12:23 PM
    • 24 Lines
    • reStructuredText
  • class HistoryCache(AbstractCache):
    """ The history cache map SWHIDs of type `rev` to a list of all their parent
    commits (until root commit). """
    async def __aenter__(self):
    ...
    • Oct 21 2020, 4:51 PM
    • 55 Lines
    • Python
  • swh/objstorage/misc/bench_reqrep.py --help
    Usage: bench_reqrep.py [OPTIONS] CFG_FROM [CFG_TO]
    Copy from one objstorage to the other. This tool is dedicated to
    stress testing swh-objstorage backends.
    ...
    • Oct 21 2020, 2:39 PM
    • 49 Lines
  • ➜ swh-jenkins-jobs git:(arcpatch-D4315) tox
    test installed: certifi==2020.6.20,chardet==3.0.4,fasteners==0.15,idna==2.10,importlib-metadata==2.0.0,jenkins-job-builder @ git+https://framagit.org/douardda/jenkins-job-builder@c89bf11ef3dffacf8f9e97b01bcc59e3fcf6f655,Jinja2==2.11.2,MarkupSafe==1.1.1,monotonic==1.5,multi-key-dict==2.0.3,pbr==5.5.1,python-jenkins==1.7.0,PyYAML==5.3.1,requests==2.24.0,six==1.15.0,stevedore==3.2.2,urllib3==1.25.11,zipp==3.3.1
    test run-test-pre: PYTHONHASHSEED='1338891736'
    test runtests: commands[0] | jenkins-jobs test -r jobs
    WARNING:jenkins_jobs.config:Config file, /etc/jenkins_jobs/jenkins_jobs.ini, not found. Using default config values.
    ...
    • Oct 21 2020, 9:43 AM
    • 63 Lines
  • ## with prometheus-statsd-exporter running
    root@7b2a6c0b041c:/# time curl http://127.0.0.1:5003/
    SWH Objstorage API server
    real 0m0.024s
    user 0m0.014s
    ...
    • Oct 19 2020, 5:48 PM
    • 13 Lines
  • Attaching to docker_swh-vault-worker_1, docker_swh-indexer-journal-client_1, docker_swh-vault_1, docker_swh-indexer_1, docker_swh-loader_1, docker_swh-lister_1, docker_swh-web_1, docker_swh-storage_1, docker_grafana_1, docker_swh-scheduler-runner_1, docker_swh-scheduler-listener_1, docker_swh-deposit_1, docker_kafka_1, docker_kafka-manager_1, docker_prometheus_1, docker_swh-scheduler_1, docker_swh-idx-storage_1, docker_swh-scheduler-db_1, docker_swh-objstorage_1, docker_prometheus-statsd-exporter_1, docker_swh-vault-db_1, docker_swh-deposit-db_1, docker_nginx_1, docker_prometheus-rabbitmq-exporter_1, docker_swh-listers-db_1, docker_zookeeper_1, docker_swh-storage-db_1, docker_amqp_1, docker_swh-idx-storage-db_1
    swh-vault-worker_1 | Using pip from /srv/softwareheritage/venv/bin/pip
    swh-vault-worker_1 | Installed Python packages:
    swh-vault-worker_1 | Package Version
    swh-vault-worker_1 | --------------------- -----------
    ...
    • Oct 16 2020, 3:19 PM
    • 29,788 Lines
    • sh
  • Collecting swh.core[http]>=0.3
    Using cached swh.core-0.4.0-py3-none-any.whl (1.2 MB)
    ERROR: Cannot install swh-core[testing] 0.4.1.dev1+g1657fb8, swh-model[testing] 0.7.1, swh-core[db,http,testing] 0.4.1.dev1+g1657fb8, swh-web-client[testing] 0.2.2.dev1+g0d962ea, swh-loader-core[testing] 0.13.2.dev4+gc30e485, swh-fuse[testing] 0.0.2.dev19+gbafeb48, swh-lister[testing] 0.2.1, swh-loader-git[testing] 0.4.1, swh-scanner[testing] 0.2.1.dev13+gc189dec, swh-web[testing] 0.0.264.dev8+gd948e60e, swh-lister 0.2.1, swh-web-client 0.2.2.dev1+g0d962ea, swh-loader-core 0.13.2.dev4+gc30e485, swh-loader-git 0.4.1, swh-fuse 0.0.2.dev19+gbafeb48, swh-scanner 0.2.1.dev13+gc189dec, swh-web 0.0.264.dev8+gd948e60e, swh-core[http] 0.4.1.dev1+g1657fb8, swh-core[testing] 0.4.1.dev1+g1657fb8, swh-model[testing] 0.7.1, swh-core[db,http,testing] 0.4.1.dev1+g1657fb8, swh-web-client[testing] 0.2.2.dev1+g0d962ea, swh-loader-core[testing] 0.13.2.dev4+gc30e485, swh-fuse[testing] 0.0.2.dev19+gbafeb48, swh-lister[testing] 0.2.1, swh-loader-git[testing] 0.4.1, swh-scanner[testing] 0.2.1.dev13+gc189dec, swh-web[testing] 0.0.264.dev8+gd948e60e, swh-lister 0.2.1, swh-web-client 0.2.2.dev1+g0d962ea, swh-loader-core 0.13.2.dev4+gc30e485, swh-loader-git 0.4.1, swh-fuse 0.0.2.dev19+gbafeb48, swh-scanner 0.2.1.dev13+gc189dec, swh-web 0.0.264.dev8+gd948e60e and swh-core[http] 0.4.0 because these package versions have conflicting dependencies.
    The conflict is caused by:
    ...
    • Oct 16 2020, 2:37 PM
    • 48 Lines
  • n.n.n / 2020-10-16
    ==================
    * configuration doc: convert tabs to spaces
    ...
    • Oct 16 2020, 2:05 PM
    • 24 Lines
  • /home/tony/work/inria/repo/swh/swh-environment/swh-core/swh/core/db/tests/test_cli.py: -C, --config-file FILE Configuration file.
    /home/tony/work/inria/repo/swh/swh-environment/swh-core/swh/core/cli/db.py: "--config-file",
    /home/tony/work/inria/repo/swh/swh-environment/swh-dataset/swh/dataset/cli.py: "--config-file",
    /home/tony/work/inria/repo/swh/swh-environment/swh-deposit/swh/deposit/cli/admin.py: "--config-file",
    /home/tony/work/inria/repo/swh/swh-environment/swh-fuse/swh/fuse/cli.py:# "--config-file",
    ...
    • Oct 15 2020, 10:13 AM
    • 24 Lines
  • terraform plan
    Refreshing Terraform state in-memory prior to plan...
    The refreshed state will be used to calculate this plan, but will not be
    persisted to local or remote state storage.
    ...
    • Oct 14 2020, 6:56 PM
    • 315 Lines
  • python swh/objstorage/misc/bench_reqrep.py -t 10 -w1 -c1 http://uffizi:5003
    PID 1919257 root: Pulling from {'cls': 'remote', 'args': {'url': 'http://uffizi:5003'}}
    PID 1919257 root: Droping objects
    PID 1919257 root: Let's start...
    PID 1919269 ObjID gen: Starting obj_id filler
    ...
    • Oct 14 2020, 3:43 PM
    • 28 Lines
  • import os
    import sys
    import concurrent
    import time
    import yaml
    ...
    • Oct 14 2020, 3:39 PM
    • 301 Lines
    • Python
  • Usage: swh fuse mount [OPTIONS] PATH [SWHID]...
    Try 'swh fuse mount -h' for help.
    Error: Invalid value for 'PATH': Directory 'swhfs' does not exist.
    • Oct 13 2020, 2:18 PM
    • 4 Lines
  • $ mkdir swhfs
    $ swh fuse mount --foreground swhfs
    ERROR:aiosqlite:returning exception unable to open database file
    Traceback (most recent call last):
    File "/home/zack/.virtualenvs/swh/lib/python3.8/site-packages/aiosqlite/core.py", line 95, in run
    ...
    • Oct 13 2020, 2:14 PM
    • 51 Lines
  • >>> from swh.storage import get_storage
    >>> from swh.model import model
    >>> from swh.model.identifiers import SWHID
    >>> s = get_storage('remote', url='http://uffizi.internal.softwareheritage.org:5002/')
    >>> s.raw_extrinsic_metadata_get(type=model.MetadataTargetType.REVISION, id=SWHID(object_type='revision', object_id='9d840874826a8cd768fc4124e7b284b0e0d7a90f'), authority=model.MetadataAuthority(type=model.MetadataAuthorityType.DEPOSIT_CLIENT, url='https://www.softwareheritage.org'))
    ...
    • Oct 13 2020, 12:10 PM
    • 6 Lines
    • Python
  • ====================================================================================================== ERRORS =======================================================================================================
    _______________________________________________________________ ERROR collecting .tox/py3/lib/python3.8/site-packages/swh/fuse/tests/gen-api-data.py ________________________________________________________________
    .tox/py3/lib/python3.8/site-packages/swh/fuse/tests/gen-api-data.py:53: in <module>
    parsed = json.loads(data)
    /usr/lib/python3.8/json/__init__.py:357: in loads
    ...
    • Oct 9 2020, 4:47 PM
    • 30 Lines
  • # client side
    $ SWH_CREDS="swh:<creds>"
    $ DEPOSIT_ID=605
    $ SWHID="swh:1:dir:ef04a768181417fbc5eef4243e2507915f24deea"
    # check status is fine
    ...
    • Oct 9 2020, 3:31 PM
    • 25 Lines
  • ___________________________________________________________________________________________________________ test_multisteps_deposit ___________________________________________________________________________________________________________
    sample_archive = {'data': b'PK\x03\x04\x14\x00\x00\x00\x00\x00\xf7HIQ\xcba\xb4c\x14\x00\x00\x00\x14\x00\x00\x00\x05\x00\x00\x00file1som...ony/pytest-104/test_multisteps_deposit0/tmpvxxda9zn', 'length': 128, 'md5sum': 'ddf8210e26805643f7f3511a878929a5', ...}
    datadir = '/home/tony/work/inria/repo/swh/swh-environment/swh-deposit/swh/deposit/tests/cli/../data', slug = 'f0529536-a65a-49ec-9936-bc3b8463bc1b', requests_mock_datadir_visits = <requests_mock.mocker.Mocker object at 0x7fc8a245e160>
    ...
    • Oct 9 2020, 11:08 AM
    • 50 Lines
  • Traceback (most recent call last):
    19: from /usr/local/bin/octocatalog-diff:23:in `<main>'
    18: from /usr/local/bin/octocatalog-diff:23:in `load'
    17: from /var/lib/gems/2.5.0/gems/octocatalog-diff-1.6.0/bin/octocatalog-diff:34:in `<top (required)>'
    16: from /var/lib/gems/2.5.0/gems/octocatalog-diff-1.6.0/lib/octocatalog-diff/cli.rb:125:in `cli'
    ...
    • Oct 8 2020, 5:56 PM
    • 53 Lines
  • 17:45 $ bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details -t staging kelvingrove.internal.softwareheritage.org
    Found host kelvingrove.internal.softwareheritage.org
    Cloning into '/tmp/swh-ocd.iqtw4oUx/environments/production/data/private'...
    done.
    Cloning into '/tmp/swh-ocd.iqtw4oUx/environments/staging/data/private'...
    ...
    • Oct 8 2020, 5:48 PM
    • 134 Lines
  • curl -i -u swh:$(swhpass ls operations/deposit.softwareheritage.org/http-auth/swh | head -1) https://deposit.softwareheritage.org/1/swh/1033/status/
    HTTP/1.1 200 OK
    Date: Thu, 08 Oct 2020 15:13:59 GMT
    Server: gunicorn/19.9.0
    Content-Type: application/xml
    ...
    • Oct 8 2020, 5:14 PM
    • 28 Lines
  • ================================================================================================= 4 passed in 1.66s =================================================================================================
    Process Process-4:
    Traceback (most recent call last):
    File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
    ...
    • Oct 8 2020, 2:11 PM
    • 20 Lines
  • terraform apply
    module.worker0.proxmox_vm_qemu.node: Refreshing state... [id=beaubourg/qemu/117]
    module.worker1.proxmox_vm_qemu.node: Refreshing state... [id=beaubourg/qemu/118]
    module.db0.proxmox_vm_qemu.node: Refreshing state... [id=orsay/qemu/115]
    module.journal0.proxmox_vm_qemu.node: Refreshing state... [id=beaubourg/qemu/122]
    ...
    • Oct 7 2020, 7:13 PM
    • 967 Lines
  • dynamic "disk" {
    for_each = var.storage0_disks
    content {
    id = disk.value["id"]
    ...
    • Oct 7 2020, 2:55 PM
    • 12 Lines
  • import copy
    import functools
    from typing import Callable, TypeVar
    ...
    • Oct 7 2020, 12:28 PM
    • 25 Lines
    • Python
  • $ mkdir -p ~/.terraform.d/plugins/github.com/telmate/proxmox/1.0.0/linux_amd64/
    $ ls ~/.terraform.d
    checkpoint_cache checkpoint_signature plugins
    $ ls ~/.terraform.d/plugins
    github.com linux_amd64 terraform-provider-proxmox terraform-provisioner-proxmox
    ...
    • Oct 7 2020, 9:34 AM
    • 94 Lines
  • Refreshing Terraform state in-memory prior to plan...
    The refreshed state will be used to calculate this plan, but will not be
    persisted to local or remote state storage.
    module.worker2.proxmox_vm_qemu.node: Refreshing state... [id=branly/qemu/112]
    ...
    • Oct 5 2020, 4:14 PM
    • 878 Lines
  • {
    "version": 4,
    "terraform_version": "0.12.28",
    "serial": 103,
    "lineage": "e2912de8-415d-4c35-8ee5-757008306227",
    ...
    • Oct 5 2020, 3:43 PM
    • 1,480 Lines
  • vagrant destroy qemutest -f ; vagrant up qemutest
    ==> qemutest: Removing domain...
    Bringing machine 'qemutest' up with 'libvirt' provider...
    ==> qemutest: Creating image (snapshot of base box volume).
    ==> qemutest: Creating domain with the following settings...
    ...
    • Oct 4 2020, 1:14 PM
    • 582 Lines
  • vagrant up test
    Bringing machine 'test' up with 'virtualbox' provider...
    ==> test: Cloning VM...
    ==> test: Matching MAC address for NAT networking...
    ==> test: Setting the name of the VM: test
    ...
    • Oct 4 2020, 10:55 AM
    • 567 Lines
  • vagrant up test
    Bringing machine 'test' up with 'libvirt' provider...
    ==> test: Creating image (snapshot of base box volume).
    ==> test: Creating domain with the following settings...
    ==> test: -- Name: puppet-environment_test
    ...
    • Oct 4 2020, 10:54 AM
    • 525 Lines
  • packer build debian_buster.qemu.json qemu: output will be in this color.
    ==> qemu: Retrieving ISO
    ==> qemu: Trying https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.6.0-amd64-netinst.iso
    ==> qemu: Trying https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.6.0-amd64-netinst.iso?checksum=2af8f43d4a7ab852151a7f630ba596572213e17d3579400b5648eba4cc974ed0
    ...
    • Oct 3 2020, 9:07 PM
    • 676 Lines
  • _______________________________________________________________________________ ERROR collecting swh/web/tests/misc/test_badges.py _______________________________________________________________________________
    swh/web/tests/misc/test_badges.py:40: in <module>
    @given(content())
    swh/web/tests/strategies.py:85: in content
    return _known_swh_object("contents")
    ...
    • Oct 2 2020, 3:02 PM
    • 22 Lines
  • diff --git a/swh/web/tests/data.py b/swh/web/tests/data.py
    index 873205b3..d0759c17 100644
    --- a/swh/web/tests/data.py
    +++ b/swh/web/tests/data.py
    @@ -178,10 +178,9 @@ def _init_tests_data():
    ...
    • Oct 2 2020, 3:01 PM
    • 17 Lines
  • python3 -m pytest --hypothesis-profile=swh-web-fast ./swh/web/tests
    ============================= test session starts ==============================
    platform linux -- Python 3.7.3, pytest-6.1.0, py-1.9.0, pluggy-0.13.1
    django: settings: swh.web.settings.tests (from ini)
    rootdir: /home/anlambert/swh/swh-environment/swh-web, configfile: pytest.ini
    ...
    • Oct 2 2020, 1:34 PM
    • 827 Lines
  • (python:.venv) ↪ git diff
    diff --git a/swh/fuse/tests/api_data.py b/swh/fuse/tests/api_data.py
    index 8bc2057..9fe1928 100644
    --- a/swh/fuse/tests/api_data.py
    +++ b/swh/fuse/tests/api_data.py
    ...
    • Oct 1 2020, 4:53 PM
    • 46 Lines
    • Diff
  • softwareheritage-deposit=> select distinct metadata->>'{http://www.w3.org/2005/Atom}title', metadata->>'{https://doi.org/10.5063/SCHEMA/CODEMETA-2.0}name', max(deposit_request.date) from deposit_request, deposit where deposit_request.deposit_id=deposit.id and raw_metadata is null and metadata is not null group by metadata->>'{http://www.w3.org/2005/Atom}title', metadata->>'{https://doi.org/10.5063/SCHEMA/CODEMETA-2.0}name';
    ?column? | ?column? | max
    ---------------------------------------+-------------------------------------------------------------------------------------------------------------+-------------------------------
    | The assignment problem | 2018-06-27 06:55:08.607636+00
    | Composing a Web of Audio Applications | 2018-06-27 06:55:13.838716+00
    ...
    • Oct 1 2020, 10:59 AM
    • 24 Lines
  • softwareheritage-deposit=> select count(*) from deposit where reception_date <= '2018-09-01' and client_id=2 and status = 'done' ;
    count
    -------
    67
    (1 row)
    ...
    • Oct 1 2020, 10:58 AM
    • 19 Lines
  • packer build debian_buster.json
    virtualbox-iso: output will be in this color.
    ==> virtualbox-iso: Retrieving Guest additions
    ==> virtualbox-iso: Trying /usr/share/virtualbox/VBoxGuestAdditions.iso
    ...
    • Sep 30 2020, 10:02 AM
    • 648 Lines
  • $ cat db.py
    def a():
    yield f(
    """
    select %s from (values %%s) as t(%s)
    ...
    • Sep 29 2020, 5:29 PM
    • 67 Lines
    • Diff
  • $ git checkout swh/indexer/storage/db.py
    $ git diff swh/indexer/storage/db.py | cat
    $ isort swh/indexer/storage/db.py
    ...
    • Sep 29 2020, 5:10 PM
    • 63 Lines
    • Diff
  • class TestMount(unittest.TestCase):
    def setUp(self):
    self.mntdir = TemporaryDirectory(suffix=".swh-fuse-test")
    def fuse_daemon():
    ...
    • Sep 29 2020, 3:25 PM
    • 18 Lines
    • Python
  • ```
    The following SSH command responded with a non-zero exit status.
    Vagrant assumes that this means the command failed!
    mount -o vers=3,udp 10.168.100.1:/tmp/puppet /tmp/puppet
    ...
    • Sep 28 2020, 8:29 PM
    • 32 Lines
  • ```
    |--------------+---------------------+-------------------------|
    | staging node | allocated RAM (Gib) | new allocated RAM (Gib) |
    |--------------+---------------------+-------------------------|
    | gateway | 1 | 1 |
    ...
    • Sep 28 2020, 7:33 PM
    • 18 Lines
  • 14:35 $ pytest -sv swh/web/tests/api/views/test_graph.py
    ============================================================================================== test session starts ===============================================================================================
    platform linux -- Python 3.7.3, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 -- /home/anlambert/.virtualenvs/swh/bin/python3
    cachedir: .pytest_cache
    django: settings: swh.web.settings.tests (from ini)
    ...
    • Sep 28 2020, 2:39 PM
    • 51 Lines
  •  ~/s/s/swh-model  1  gbp buildpackage --git-builder=sbuild -As --force-orig-source --extra-repository='deb [trusted=yes] https://debian.softwareheritage.org/ sid main'
    gbp:info: Performing the build
    dh clean --with python3 --buildsystem=pybuild
    dh: warning: Compatibility levels before 10 are deprecated (level 9 in use)
    dh_auto_clean -O--buildsystem=pybuild
    ...
    • Sep 25 2020, 6:37 PM
    • 32 Lines
  • from pprint import pformat
    @swh_cli_group.group(name="scanner", context_settings=CONTEXT_SETTINGS)
    @click.option(
    "-C",
    "--config-file",
    ...
    • Sep 25 2020, 3:55 PM
    • 17 Lines
    • Python
  • (swh) tenma@swh-ng1:~/Dev/swh-env/swh-scanner$ tox -r -e mypy
    GLOB sdist-make: /home/tenma/Dev/swh-env/swh-scanner/setup.py
    mypy recreate: /home/tenma/Dev/swh-env/swh-scanner/.tox/mypy
    mypy installdeps: mypy
    mypy inst: /home/tenma/Dev/swh-env/swh-scanner/.tox/.tmp/package/1/swh.scanner-0.2.1.dev2+ga7a17e7.d20200925.zip
    ...
    • Sep 25 2020, 2:15 PM
    • 20 Lines
  • commit a7a17e7ed6a45941d04e30f239a6e6eb62ecca25
    Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
    Date: Fri Sep 25 14:05:40 2020 +0200
    Pin black in tox to the same version as pre-commit
    ...
    • Sep 25 2020, 2:05 PM
    • 40 Lines
  • (swh) tenma@swh-ng1:~/Dev/swh-env/swh-scanner$ tox
    GLOB sdist-make: /home/tenma/Dev/swh-env/swh-scanner/setup.py
    black create: /home/tenma/Dev/swh-env/swh-scanner/.tox/black
    black installdeps: black
    black installed: appdirs==1.4.4,black==20.8b1,click==7.1.2,mypy-extensions==0.4.3,pathspec==0.8.0,regex==2020.7.14,toml==0.10.1,typed-ast==1.4.1,typing-extensions==3.7.4.3
    ...
    • Sep 25 2020, 1:59 PM
    • 83 Lines
  • py3 recreate: /home/ddouard/swh/swh-environment/swh-model/.tox/py3
    py3 installdeps: pytest-cov
    py3 inst: /home/ddouard/swh/swh-environment/swh-model/.tox/.tmp/package/1/swh.model-0.6.7.dev9+gdb723f0.d20200925.zip
    py3 installed: attrs==20.2.0,attrs-strict==0.2.0,certifi==2020.6.20,click==7.1.2,coverage==5.3,Deprecated==1.2.10,dulwich==0.20.6,hypothesis==5.36.0,iniconfig==1.0.1,iso8601==0.1.13,more-itertools==8.5.0,packaging==20.4,pluggy==0.13.1,py==1.9.0,pyparsing==2.4.7,pytest==6.0.2,pytest-cov==2.10.1,python-dateutil==2.8.1,pytz==2020.1,PyYAML==5.3.1,sentry-sdk==0.17.8,six==1.15.0,sortedcontainers==2.2.2,swh.core==0.3.0,swh.model @ file:///home/ddouard/swh/swh-environment/swh-model/.tox/.tmp/package/1/swh.model-0.6.7.dev9%2Bgdb723f0.d20200925.zip,toml==0.10.1,typing-extensions==3.7.4.3,urllib3==1.25.10,wrapt==1.12.1
    py3 run-test-pre: PYTHONHASHSEED='631895245'
    ...
    • Sep 25 2020, 12:59 PM
    • 56 Lines
  • Usage: swh [OPTIONS] COMMAND [ARGS]...
    Command line interface for Software Heritage.
    Options:
    ...
    • Sep 25 2020, 12:36 PM
    • 35 Lines
  • def inode2swhid(self, inode: int) -> SWHID:
    try:
    return self._inode2swhid[inode]
    except KeyError:
    raise pyfuse3.FUSEError(errno.ENOENT)
    • Sep 25 2020, 11:37 AM
    • 5 Lines
    • Python
  • $ workon swh
    WARNING:swh.core.cli:Could not load subcommand indexer: module 'swh.indexer.cli' has no attribute 'cli'
    WARNING:swh.core.cli:Could not load subcommand search: module 'swh.search.cli' has no attribute 'cli'
    WARNING:swh.core.cli:Could not load subcommand graph: (swh.core 0.2.4.dev2+g60c535d (/home/zack/dati/projects/sw-heritage/git/swh-environment/swh-core), Requirement.parse('swh.core[http]>=0.3'))
    WARNING:swh.core.cli:Could not load subcommand icinga_plugins: module 'swh.icinga_plugins.cli' has no attribute 'cli'
    ...
    • Sep 25 2020, 11:20 AM
    • 7 Lines
  • [2020-09-25 06:23:57] ERROR 8192: implode(): Passing glue string after array is deprecated. Swap the parameters at [/usr/share/libphutil/src/utils/utils.php:1572]
    arcanist(), phutil()
    #0 implode(array, string) called at [<phutil>/src/utils/utils.php:1572]
    #1 phutil_build_http_querystring(array) called at [<phutil>/src/future/http/HTTPSFuture.php:526]
    #2 HTTPSFuture::formatRequestDataForCURL() called at [<phutil>/src/future/http/HTTPSFuture.php:247]
    ...
    • Sep 25 2020, 8:24 AM
    • 11 Lines
  • #!/usr/bin/perl -w
    # converts markdown exported from hackmd.io to reStructuredText
    # SPDX-FileCopyrightText: © 2020 Stefano Zacchiroli <zack@upsilon.cc>
    ...
    • Sep 23 2020, 11:14 PM
    • 26 Lines
    • Perl
  • <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <profiles version="20">
    <profile kind="CodeFormatterProfile" name="Spotless" version="20">
    <setting id="org.eclipse.jdt.core.formatter.insert_space_after_ellipsis" value="insert"/>
    <setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_enum_declarations" value="insert"/>
    ...
    • Sep 23 2020, 4:27 PM
    • 383 Lines
    • XML