Page MenuHomeSoftware Heritage
Create Paste
Paste Active Pastes

Active Pastes
Use Results
Edit Query

  • P859 (An Untitled Masterwork)
     ddouard  (e) swh   master ✚ 1 ⚑ 1  …/s/s/docs  make
    sphinx-apidoc --ext-viewcode --separate -o apidoc .. . ../setup.py ../swh/*/tests ../swh/*/tests/* ../swh/*/*/tests/* ../swh/*/*/*/tests/* ../swh/*/migrations ../swh/*/migrations/* ../swh/*/*/migrations/* ../swh/*/*/*/migrations/* ../bin/* ../utils/* ../node_modules/* ../swh/*/settings/*
    Creating file apidoc/conftest.rst.
    Creating file apidoc/swh.rst.
    Creating file apidoc/swh.deposit.rst.
    ...
    • Nov 6 2020, 10:02 AM
    • 95 Lines
    Author: douardda
  • P858 (An Untitled Masterwork)
     ddouard  (e) swh   master ✚ 1 ⚑ 1  …/s/s/docs  make
    sphinx-build -b html . _build/html
    Running Sphinx v3.3.0
    loading translations [en]... done
    WARNING: logo file '_static/software-heritage-logo-title-motto-vertical-white.png' does not exist
    ...
    • Nov 6 2020, 10:00 AM
    • 159 Lines
    Author: douardda
  • P857 diff buster-preseed.cfg buster-preseed.vagrant.cfg
    diff buster-preseed.cfg buster-preseed.vagrant.cfg
    296c296
    < grub-pc grub-pc/install_devices multiselect /dev/vda
    ---
    > grub-pc grub-pc/install_devices multiselect /dev/disk/by-id/ata-VBOX_HARDDISK_VBbc5508de-d37f68c1
    ...
    • Nov 5 2020, 4:50 PM
    • 23 Lines
    Author: ardumont
  • P856 new and clearer swh webapps puppet roles
    @startuml
    title Webapp and Deposit roles with and without reverse proxy
    note top of role_swh_rp_webapp : production webapp0 (azure)
    note top of role_swh_rp_webapps : production moma (rocq)
    ...
    • Nov 5 2020, 1:54 PM
    • 25 Lines
    Author: ardumont
  • P855 current swh web apis puppet roles
    @startuml
    title Webapp and Deposit roles with and without reverse proxy
    role_swh_base_api *-- profile_swh_webapp
    note top of role_swh_base_api: - production webapp0 (azure)\n- staging webapp (rocq)
    ...
    • Nov 5 2020, 1:53 PM
    • 19 Lines
    Author: ardumont
  • P854 Command-Line Input
    (swh) ✘  ~/swh-environment/swhfs/archive  time (lf | wc -l)
    519
    ( ls --color=tty -f | wc -l; ) 0,01s user 0,00s system 4% cpu 0,362 total
    (swh) ~/swh-environment/swhfs/archive  time (lf | wc -l)
    519
    ...
    • Nov 5 2020, 10:28 AM
    • 12 Lines
    Author: zack
  • P853 Target staging reverse proxy
    @startuml
    title Staging Reverse Proxy
    participant User
    box "Firewall" #Orange
    ...
    • Nov 4 2020, 3:39 PM
    • 51 Lines
    Author: ardumont
  • P852 current hitch varnish apache gunicorn status on web frontend
    @startuml
    participant User
    box "webserver: moma (webapp, deposit), webapp.staging, deposit.staging..." #LightBlue
    participant FrontendHitch
    participant Varnish
    ...
    • Nov 4 2020, 2:02 PM
    • 41 Lines
    Author: ardumont
  • P851 (An Untitled Masterwork)
    Selecting previously unselected package linux-image-5.9.0-fusebuf-amd64-unsigned.
    (Reading database ... 478590 files and directories currently installed.)
    Preparing to unpack linux-image-5.9.0-fusebuf-amd64-unsigned_5.9.1-2_amd64.deb ...
    Unpacking linux-image-5.9.0-fusebuf-amd64-unsigned (5.9.1-2) ...
    Setting up linux-image-5.9.0-fusebuf-amd64-unsigned (5.9.1-2) ...
    ...
    • Nov 4 2020, 10:25 AM
    • 56 Lines
    Author: haltode
  • P850 rebuild a kernel Debian package with larger FUSE getdirplus() buffer
    apt-get source linux
    cd linux-*/
    $EDITOR debian/config/defines
    # change "debug-info:" line to "false"
    ...
    • Nov 3 2020, 8:08 PM
    • 14 Lines
    Author: zack
  • P849 revision metadata tasks with deprecated policy_update...
    softwareheritage-scheduler=> \x
    softwareheritage-scheduler=> select * from task where type='index-revision-metadata' limit 10;
    -[ RECORD 1 ]----+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------$
    id | 114244463
    type | index-revision-metadata
    ...
    • Nov 3 2020, 2:44 PM
    • 66 Lines
    Author: ardumont
  • P848 revision metadata tasks with deprecated policy_update...
    -[ RECORD 1 ]----+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------$
    id | 114244463
    type | index-revision-metadata
    arguments | {"args": [], "kwargs": {"ids": ["2ade866be83193ef461c8f661bbbaa8cea3a6ac5"], "next_step": {"type": "indexer_origin_metadata", "policy": "oneshot", "arguments": {"args": [], "kwargs": {"origin_head": {"14342257": "2ade86$
    next_run | 2019-01-29 11:11:29.196488+00
    ...
    • Nov 3 2020, 2:42 PM
    • 50 Lines
    Author: ardumont
  • P847 (An Untitled Masterwork)
    import asyncio
    from typing import AsyncGenerator, AsyncIterator
    async def gen() -> AsyncGenerator[int, None]:
    ...
    • Nov 3 2020, 9:48 AM
    • 22 Lines
    • Python
    Author: haltode
  • P846 clean up raw extrinsic metadata
    delete from raw_extrinsic_metadata
    where id >= 'swh:1:snp:0000000000000000000000000000000000000000' and id < 'swh:1:snp:1111111111111111111111111111111111111111'
    and (format = 'replicate-npm-package-json'
    or format = 'pypi-project-json');
    ...
    • Nov 2 2020, 5:20 PM
    • 74 Lines
    Author: ardumont
  • P845 (An Untitled Masterwork)
    14:16 $ ./bin/deploy-on --apt --test moma.internal.softwareheritage.org
    *** swh-deploy: deploying recipes on pergamon.internal.softwareheritage.org...
    INFO -> Using Puppetfile '/etc/puppet/code/environments/production/Puppetfile'
    INFO -> Using Puppetfile '/etc/puppet/code/environments/staging/Puppetfile'
    INFO -> Deploying environment /etc/puppet/code/environments/production
    ...
    • Nov 2 2020, 2:18 PM
    • 133 Lines
    Author: anlambert
  • P844 Command-Line Input
    @startuml arch-context
    ' !includeurl https://raw.githubusercontent.com/RicardoNiepel/C4-PlantUML/master/C4_Container.puml
    !include /home/zack/dati/src/plantuml/C4-PlantUML/C4_Container.puml
    ' title System Context diagram for Software Heritage Filesystem (SwhFS)
    ...
    • Oct 31 2020, 5:50 PM
    • 19 Lines
    Author: zack
  • P843 (An Untitled Masterwork)
    swh-deposit_1 | [2020-10-28 13:52:51 +0000] [34] [DEBUG] GET /deposit/1/servicedocument/
    swh-deposit_1 | [2020-10-28 13:52:51 +0000] [34] [DEBUG] Closing connection.
    swh-deposit_1 | [2020-10-28 13:52:51 +0000] [35] [DEBUG] POST /deposit/1/test/
    swh-deposit_1 | [2020-10-28 13:52:52 +0000] [35] [DEBUG] Closing connection.
    swh-deposit_1 | [2020-10-28 13:52:55 +0000] [35] [DEBUG] GET /1/private/test/1/check/
    ...
    • Oct 28 2020, 4:40 PM
    • 12 Lines
    Author: douardda
  • P842 Command-Line Input
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
    ...
    • Oct 28 2020, 10:52 AM
    • 44 Lines
    Author: zack
  • P841 Command-Line Input
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
    ...
    • Oct 28 2020, 10:49 AM
    • 44 Lines
    Author: zack
  • P840 Command-Line Input
    8: veth61ae6d9@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-d0d7f9beaec2 state UP group default
    link/ether 42:03:28:76:3b:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::4003:28ff:fe76:3b03/64 scope link
    valid_lft forever preferred_lft forever
    • Oct 28 2020, 8:57 AM
    • 4 Lines
    Author: zack
  • P839 (An Untitled Masterwork)
    [Question] getdents() calls use very small buffer when FUSE mount
    Hello all!
    I am currently working on a FUSE filesystem where a directory can
    ...
    • Oct 27 2020, 10:07 AM
    • 32 Lines
    Author: haltode
  • P838 time ls -f history/
    on master:
    Impossible to list history/ for more than a few hundreds commits because ls
    getdents() syscalls are done with a quite small buffer (19 entries on my
    machine), hence the readdir is regularly called with a different offset and the
    ...
    • Oct 26 2020, 5:15 PM
    • 15 Lines
    Author: haltode
  • P837 ingesting_cpython_connreset
    $ doco exec swh-scheduler swh scheduler task add load-git -p oneshot url=https://github.com/python/cpython
    $ doco logs -f --tail 0 | grep -vE "heartbeat|statsd"
    swh-scheduler-runner_1 | INFO:swh.scheduler.celery_backend.runner:Grabbed 1 tasks load-git
    amqp_1 |
    amqp_1 | =INFO REPORT==== 26-Oct-2020::14:58:45 ===
    ...
    • Oct 26 2020, 4:28 PM
    • 748 Lines
    Author: tenma
  • P836 (An Untitled Masterwork)
    [swh]
    dbname=softwareheritage
    host=db.internal.softwareheritage.org
    user=guest
    • Oct 26 2020, 2:29 PM
    • 4 Lines
    Author: douardda
  • P835 (An Untitled Masterwork)
    class Bar:
    def __init__(self):
    self.x = randint(1, 1000)
    self.y = [randint(1, 1000) for _ in range(100)]
    ...
    • Oct 26 2020, 1:59 PM
    • 23 Lines
    • Python
    Author: haltode
  • P834 (An Untitled Masterwork)
    pympler asizeof 25880
    vlorentz total_size 28
    original total_size 604
    pympler asizeof 32648
    ...
    • Oct 26 2020, 1:08 PM
    • 23 Lines
    Author: haltode
  • P833 computers says no (to be polite) ¯\_(ツ)_/¯
    swh  swh  tony  yavin4  ~  work  …  repo  swh  swh-environment  master  15+  SIGINT  %  doco exec swh-vault-worker /bin/bash
    swh@9d500dd5f25e:/$ python
    Python 3.7.4 (default, Jul 13 2019, 14:04:11)
    [GCC 8.3.0] on linux
    ...
    • Oct 22 2020, 7:57 PM
    • 34 Lines
    Author: ardumont
  • P832 SourceForge "tools" and VCS count
    480722 activity
    480655 reviews
    480634 summary
    480140 support
    480088 files
    ...
    • Oct 22 2020, 12:23 PM
    • 24 Lines
    • reStructuredText
    Author: zack
  • P831 (An Untitled Masterwork)
    class HistoryCache(AbstractCache):
    """ The history cache map SWHIDs of type `rev` to a list of all their parent
    commits (until root commit). """
    async def __aenter__(self):
    ...
    • Oct 21 2020, 4:51 PM
    • 55 Lines
    • Python
    Author: haltode
  • P830 (An Untitled Masterwork)
    swh/objstorage/misc/bench_reqrep.py --help
    Usage: bench_reqrep.py [OPTIONS] CFG_FROM [CFG_TO]
    Copy from one objstorage to the other. This tool is dedicated to
    stress testing swh-objstorage backends.
    ...
    • Oct 21 2020, 2:39 PM
    • 49 Lines
    Author: douardda
  • P829 D4315 test
    ➜ swh-jenkins-jobs git:(arcpatch-D4315) tox
    test installed: certifi==2020.6.20,chardet==3.0.4,fasteners==0.15,idna==2.10,importlib-metadata==2.0.0,jenkins-job-builder @ git+https://framagit.org/douardda/jenkins-job-builder@c89bf11ef3dffacf8f9e97b01bcc59e3fcf6f655,Jinja2==2.11.2,MarkupSafe==1.1.1,monotonic==1.5,multi-key-dict==2.0.3,pbr==5.5.1,python-jenkins==1.7.0,PyYAML==5.3.1,requests==2.24.0,six==1.15.0,stevedore==3.2.2,urllib3==1.25.11,zipp==3.3.1
    test run-test-pre: PYTHONHASHSEED='1338891736'
    test runtests: commands[0] | jenkins-jobs test -r jobs
    WARNING:jenkins_jobs.config:Config file, /etc/jenkins_jobs/jenkins_jobs.ini, not found. Using default config values.
    ...
    • Oct 21 2020, 9:43 AM
    • 63 Lines
    Author: vsellier
  • P828 (An Untitled Masterwork)
    ## with prometheus-statsd-exporter running
    root@7b2a6c0b041c:/# time curl http://127.0.0.1:5003/
    SWH Objstorage API server
    real 0m0.024s
    user 0m0.014s
    ...
    • Oct 19 2020, 5:48 PM
    • 13 Lines
    Author: douardda
  • P826 doco_logs_up_nodb
    Attaching to docker_swh-vault-worker_1, docker_swh-indexer-journal-client_1, docker_swh-vault_1, docker_swh-indexer_1, docker_swh-loader_1, docker_swh-lister_1, docker_swh-web_1, docker_swh-storage_1, docker_grafana_1, docker_swh-scheduler-runner_1, docker_swh-scheduler-listener_1, docker_swh-deposit_1, docker_kafka_1, docker_kafka-manager_1, docker_prometheus_1, docker_swh-scheduler_1, docker_swh-idx-storage_1, docker_swh-scheduler-db_1, docker_swh-objstorage_1, docker_prometheus-statsd-exporter_1, docker_swh-vault-db_1, docker_swh-deposit-db_1, docker_nginx_1, docker_prometheus-rabbitmq-exporter_1, docker_swh-listers-db_1, docker_zookeeper_1, docker_swh-storage-db_1, docker_amqp_1, docker_swh-idx-storage-db_1
    swh-vault-worker_1 | Using pip from /srv/softwareheritage/venv/bin/pip
    swh-vault-worker_1 | Installed Python packages:
    swh-vault-worker_1 | Package Version
    swh-vault-worker_1 | --------------------- -----------
    ...
    • Oct 16 2020, 3:19 PM
    • 29,788 Lines
    • sh
    Author: tenma
  • P825 (An Untitled Masterwork)
    Collecting swh.core[http]>=0.3
    Using cached swh.core-0.4.0-py3-none-any.whl (1.2 MB)
    ERROR: Cannot install swh-core[testing] 0.4.1.dev1+g1657fb8, swh-model[testing] 0.7.1, swh-core[db,http,testing] 0.4.1.dev1+g1657fb8, swh-web-client[testing] 0.2.2.dev1+g0d962ea, swh-loader-core[testing] 0.13.2.dev4+gc30e485, swh-fuse[testing] 0.0.2.dev19+gbafeb48, swh-lister[testing] 0.2.1, swh-loader-git[testing] 0.4.1, swh-scanner[testing] 0.2.1.dev13+gc189dec, swh-web[testing] 0.0.264.dev8+gd948e60e, swh-lister 0.2.1, swh-web-client 0.2.2.dev1+g0d962ea, swh-loader-core 0.13.2.dev4+gc30e485, swh-loader-git 0.4.1, swh-fuse 0.0.2.dev19+gbafeb48, swh-scanner 0.2.1.dev13+gc189dec, swh-web 0.0.264.dev8+gd948e60e, swh-core[http] 0.4.1.dev1+g1657fb8, swh-core[testing] 0.4.1.dev1+g1657fb8, swh-model[testing] 0.7.1, swh-core[db,http,testing] 0.4.1.dev1+g1657fb8, swh-web-client[testing] 0.2.2.dev1+g0d962ea, swh-loader-core[testing] 0.13.2.dev4+gc30e485, swh-fuse[testing] 0.0.2.dev19+gbafeb48, swh-lister[testing] 0.2.1, swh-loader-git[testing] 0.4.1, swh-scanner[testing] 0.2.1.dev13+gc189dec, swh-web[testing] 0.0.264.dev8+gd948e60e, swh-lister 0.2.1, swh-web-client 0.2.2.dev1+g0d962ea, swh-loader-core 0.13.2.dev4+gc30e485, swh-loader-git 0.4.1, swh-fuse 0.0.2.dev19+gbafeb48, swh-scanner 0.2.1.dev13+gc189dec, swh-web 0.0.264.dev8+gd948e60e and swh-core[http] 0.4.0 because these package versions have conflicting dependencies.
    The conflict is caused by:
    ...
    • Oct 16 2020, 2:37 PM
    • 48 Lines
    Author: seirl
  • P824 Command-Line Input
    n.n.n / 2020-10-16
    ==================
    * configuration doc: convert tabs to spaces
    ...
    • Oct 16 2020, 2:05 PM
    • 24 Lines
    Author: zack
  • P823 --config-file all around
    /home/tony/work/inria/repo/swh/swh-environment/swh-core/swh/core/db/tests/test_cli.py: -C, --config-file FILE Configuration file.
    /home/tony/work/inria/repo/swh/swh-environment/swh-core/swh/core/cli/db.py: "--config-file",
    /home/tony/work/inria/repo/swh/swh-environment/swh-dataset/swh/dataset/cli.py: "--config-file",
    /home/tony/work/inria/repo/swh/swh-environment/swh-deposit/swh/deposit/cli/admin.py: "--config-file",
    /home/tony/work/inria/repo/swh/swh-environment/swh-fuse/swh/fuse/cli.py:# "--config-file",
    ...
    • Oct 15 2020, 10:13 AM
    • 24 Lines
    Author: ardumont
  • P822 D4197: `terraform plan` on production nodes
    terraform plan
    Refreshing Terraform state in-memory prior to plan...
    The refreshed state will be used to calculate this plan, but will not be
    persisted to local or remote state storage.
    ...
    • Oct 14 2020, 6:56 PM
    • 315 Lines
    Author: ardumont
  • P821 (An Untitled Masterwork)
    python swh/objstorage/misc/bench_reqrep.py -t 10 -w1 -c1 http://uffizi:5003
    PID 1919257 root: Pulling from {'cls': 'remote', 'args': {'url': 'http://uffizi:5003'}}
    PID 1919257 root: Droping objects
    PID 1919257 root: Let's start...
    PID 1919269 ObjID gen: Starting obj_id filler
    ...
    • Oct 14 2020, 3:43 PM
    • 28 Lines
    Author: douardda
  • P820 bench objstorage
    import os
    import sys
    import concurrent
    import time
    import yaml
    ...
    • Oct 14 2020, 3:39 PM
    • 301 Lines
    • Python
    Author: douardda
  • P819 Command-Line Input
    Usage: swh fuse mount [OPTIONS] PATH [SWHID]...
    Try 'swh fuse mount -h' for help.
    Error: Invalid value for 'PATH': Directory 'swhfs' does not exist.
    • Oct 13 2020, 2:18 PM
    • 4 Lines
    Author: zack
  • P818 Command-Line Input
    $ mkdir swhfs
    $ swh fuse mount --foreground swhfs
    ERROR:aiosqlite:returning exception unable to open database file
    Traceback (most recent call last):
    File "/home/zack/.virtualenvs/swh/lib/python3.8/site-packages/aiosqlite/core.py", line 95, in run
    ...
    • Oct 13 2020, 2:14 PM
    • 51 Lines
    Author: zack
  • P817 How the check metadata in the ERMDS
    >>> from swh.storage import get_storage
    >>> from swh.model import model
    >>> from swh.model.identifiers import SWHID
    >>> s = get_storage('remote', url='http://uffizi.internal.softwareheritage.org:5002/')
    >>> s.raw_extrinsic_metadata_get(type=model.MetadataTargetType.REVISION, id=SWHID(object_type='revision', object_id='9d840874826a8cd768fc4124e7b284b0e0d7a90f'), authority=model.MetadataAuthority(type=model.MetadataAuthorityType.DEPOSIT_CLIENT, url='https://www.softwareheritage.org'))
    ...
    • Oct 13 2020, 12:10 PM
    • 6 Lines
    • Python
    Author: vlorentz
  • P816 Command-Line Input
    ====================================================================================================== ERRORS =======================================================================================================
    _______________________________________________________________ ERROR collecting .tox/py3/lib/python3.8/site-packages/swh/fuse/tests/gen-api-data.py ________________________________________________________________
    .tox/py3/lib/python3.8/site-packages/swh/fuse/tests/gen-api-data.py:53: in <module>
    parsed = json.loads(data)
    /usr/lib/python3.8/json/__init__.py:357: in loads
    ...
    • Oct 9 2020, 4:47 PM
    • 30 Lines
    Author: zack
  • P815 deposit client query update to actually update metadata on deposit done
    # client side
    $ SWH_CREDS="swh:<creds>"
    $ DEPOSIT_ID=605
    $ SWHID="swh:1:dir:ef04a768181417fbc5eef4243e2507915f24deea"
    # check status is fine
    ...
    • Oct 9 2020, 3:31 PM
    • 25 Lines
    Author: ardumont
  • P814 sample of pdb.set_trace() in code executed for click cli
    ___________________________________________________________________________________________________________ test_multisteps_deposit ___________________________________________________________________________________________________________
    sample_archive = {'data': b'PK\x03\x04\x14\x00\x00\x00\x00\x00\xf7HIQ\xcba\xb4c\x14\x00\x00\x00\x14\x00\x00\x00\x05\x00\x00\x00file1som...ony/pytest-104/test_multisteps_deposit0/tmpvxxda9zn', 'length': 128, 'md5sum': 'ddf8210e26805643f7f3511a878929a5', ...}
    datadir = '/home/tony/work/inria/repo/swh/swh-environment/swh-deposit/swh/deposit/tests/cli/../data', slug = 'f0529536-a65a-49ec-9936-bc3b8463bc1b', requests_mock_datadir_visits = <requests_mock.mocker.Mocker object at 0x7fc8a245e160>
    ...
    • Oct 9 2020, 11:08 AM
    • 50 Lines
    Author: ardumont
  • P813 (An Untitled Masterwork)
    Traceback (most recent call last):
    19: from /usr/local/bin/octocatalog-diff:23:in `<main>'
    18: from /usr/local/bin/octocatalog-diff:23:in `load'
    17: from /var/lib/gems/2.5.0/gems/octocatalog-diff-1.6.0/bin/octocatalog-diff:34:in `<top (required)>'
    16: from /var/lib/gems/2.5.0/gems/octocatalog-diff-1.6.0/lib/octocatalog-diff/cli.rb:125:in `cli'
    ...
    • Oct 8 2020, 5:56 PM
    • 53 Lines
    Author: anlambert
  • P812 octocatalog-diff kelvingrove
    17:45 $ bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details -t staging kelvingrove.internal.softwareheritage.org
    Found host kelvingrove.internal.softwareheritage.org
    Cloning into '/tmp/swh-ocd.iqtw4oUx/environments/production/data/private'...
    done.
    Cloning into '/tmp/swh-ocd.iqtw4oUx/environments/staging/data/private'...
    ...
    • Oct 8 2020, 5:48 PM
    • 134 Lines
    Author: anlambert
  • P811 check deposit status
    curl -i -u swh:$(swhpass ls operations/deposit.softwareheritage.org/http-auth/swh | head -1) https://deposit.softwareheritage.org/1/swh/1033/status/
    HTTP/1.1 200 OK
    Date: Thu, 08 Oct 2020 15:13:59 GMT
    Server: gunicorn/19.9.0
    Content-Type: application/xml
    ...
    • Oct 8 2020, 5:14 PM
    • 28 Lines
    Author: ardumont
  • P810 Command-Line Input
    ================================================================================================= 4 passed in 1.66s =================================================================================================
    Process Process-4:
    Traceback (most recent call last):
    File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
    ...
    • Oct 8 2020, 2:11 PM
    • 20 Lines
    Author: zack
  • P809 terraform apply
    terraform apply
    module.worker0.proxmox_vm_qemu.node: Refreshing state... [id=beaubourg/qemu/117]
    module.worker1.proxmox_vm_qemu.node: Refreshing state... [id=beaubourg/qemu/118]
    module.db0.proxmox_vm_qemu.node: Refreshing state... [id=orsay/qemu/115]
    module.journal0.proxmox_vm_qemu.node: Refreshing state... [id=beaubourg/qemu/122]
    ...
    • Oct 7 2020, 7:13 PM
    • 967 Lines
    Author: ardumont
  • P808 terraform for_each
    dynamic "disk" {
    for_each = var.storage0_disks
    content {
    id = disk.value["id"]
    ...
    • Oct 7 2020, 2:55 PM
    • 12 Lines
    Author: vsellier
  • P807 mutable default arguments don't have to be an issue
    import copy
    import functools
    from typing import Callable, TypeVar
    ...
    • Oct 7 2020, 12:28 PM
    • 25 Lines
    • Python
    Author: vlorentz
  • P806 update terraform to use more recent version -> plugin proxmox issue
    $ mkdir -p ~/.terraform.d/plugins/github.com/telmate/proxmox/1.0.0/linux_amd64/
    $ ls ~/.terraform.d
    checkpoint_cache checkpoint_signature plugins
    $ ls ~/.terraform.d/plugins
    github.com linux_amd64 terraform-provider-proxmox terraform-provisioner-proxmox
    ...
    • Oct 7 2020, 9:34 AM
    • 94 Lines
    Author: ardumont
  • P805 D4071 (staging): terraform plan -no-color
    Refreshing Terraform state in-memory prior to plan...
    The refreshed state will be used to calculate this plan, but will not be
    persisted to local or remote state storage.
    module.worker2.proxmox_vm_qemu.node: Refreshing state... [id=branly/qemu/112]
    ...
    • Oct 5 2020, 4:14 PM
    • 878 Lines
    Author: ardumont
  • P804 terraform.tfstate for D4071
    {
    "version": 4,
    "terraform_version": "0.12.28",
    "serial": 103,
    "lineage": "e2912de8-415d-4c35-8ee5-757008306227",
    ...
    • Oct 5 2020, 3:43 PM
    • 1,480 Lines
    Author: ardumont
  • P803 libvirt: vagrant up qemutest
    vagrant destroy qemutest -f ; vagrant up qemutest
    ==> qemutest: Removing domain...
    Bringing machine 'qemutest' up with 'libvirt' provider...
    ==> qemutest: Creating image (snapshot of base box volume).
    ==> qemutest: Creating domain with the following settings...
    ...
    • Oct 4 2020, 1:14 PM
    • 582 Lines
    Author: ardumont
  • P802 vbox: vagrant up test
    vagrant up test
    Bringing machine 'test' up with 'virtualbox' provider...
    ==> test: Cloning VM...
    ==> test: Matching MAC address for NAT networking...
    ==> test: Setting the name of the VM: test
    ...
    • Oct 4 2020, 10:55 AM
    • 567 Lines
    Author: ardumont
  • P801 libvirt: vagrant up test (fail: because missing gpg deps)
    vagrant up test
    Bringing machine 'test' up with 'libvirt' provider...
    ==> test: Creating image (snapshot of base box volume).
    ==> test: Creating domain with the following settings...
    ==> test: -- Name: puppet-environment_test
    ...
    • Oct 4 2020, 10:54 AM
    • 525 Lines
    Author: ardumont
  • P800 packer build debian_buster.qemu.json
    packer build debian_buster.qemu.json qemu: output will be in this color.
    ==> qemu: Retrieving ISO
    ==> qemu: Trying https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.6.0-amd64-netinst.iso
    ==> qemu: Trying https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.6.0-amd64-netinst.iso?checksum=2af8f43d4a7ab852151a7f630ba596572213e17d3579400b5648eba4cc974ed0
    ...
    • Oct 3 2020, 9:07 PM
    • 676 Lines
    Author: ardumont
  • P799 SWH_CONFIG_FILENAME environment variable is undefined.
    _______________________________________________________________________________ ERROR collecting swh/web/tests/misc/test_badges.py _______________________________________________________________________________
    swh/web/tests/misc/test_badges.py:40: in <module>
    @given(content())
    swh/web/tests/strategies.py:85: in content
    return _known_swh_object("contents")
    ...
    • Oct 2 2020, 3:02 PM
    • 22 Lines
    Author: anlambert
  • P798 GitLoaderFromArchive config fix
    diff --git a/swh/web/tests/data.py b/swh/web/tests/data.py
    index 873205b3..d0759c17 100644
    --- a/swh/web/tests/data.py
    +++ b/swh/web/tests/data.py
    @@ -178,10 +178,9 @@ def _init_tests_data():
    ...
    • Oct 2 2020, 3:01 PM
    • 17 Lines
    Author: anlambert
  • P797 swh-web tests after arc patch D4118
    python3 -m pytest --hypothesis-profile=swh-web-fast ./swh/web/tests
    ============================= test session starts ==============================
    platform linux -- Python 3.7.3, pytest-6.1.0, py-1.9.0, pluggy-0.13.1
    django: settings: swh.web.settings.tests (from ini)
    rootdir: /home/anlambert/swh/swh-environment/swh-web, configfile: pytest.ini
    ...
    • Oct 2 2020, 1:34 PM
    • 827 Lines
    Author: anlambert
  • P796 (An Untitled Masterwork)
    (python:.venv) ↪ git diff
    diff --git a/swh/fuse/tests/api_data.py b/swh/fuse/tests/api_data.py
    index 8bc2057..9fe1928 100644
    --- a/swh/fuse/tests/api_data.py
    +++ b/swh/fuse/tests/api_data.py
    ...
    • Oct 1 2020, 4:53 PM
    • 46 Lines
    • Diff
    Author: haltode
  • P795 deposits with no XML
    softwareheritage-deposit=> select distinct metadata->>'{http://www.w3.org/2005/Atom}title', metadata->>'{https://doi.org/10.5063/SCHEMA/CODEMETA-2.0}name', max(deposit_request.date) from deposit_request, deposit where deposit_request.deposit_id=deposit.id and raw_metadata is null and metadata is not null group by metadata->>'{http://www.w3.org/2005/Atom}title', metadata->>'{https://doi.org/10.5063/SCHEMA/CODEMETA-2.0}name';
    ?column? | ?column? | max
    ---------------------------------------+-------------------------------------------------------------------------------------------------------------+-------------------------------
    | The assignment problem | 2018-06-27 06:55:08.607636+00
    | Composing a Web of Audio Applications | 2018-06-27 06:55:13.838716+00
    ...
    • Oct 1 2020, 10:59 AM
    • 24 Lines
    Author: vlorentz
  • P794 deposits count and sample on time period prior to 2018-09-01
    softwareheritage-deposit=> select count(*) from deposit where reception_date <= '2018-09-01' and client_id=2 and status = 'done' ;
    count
    -------
    67
    (1 row)
    ...
    • Oct 1 2020, 10:58 AM
    • 19 Lines
    Author: ardumont
  • P793 packer build debian_buster.json
    packer build debian_buster.json
    virtualbox-iso: output will be in this color.
    ==> virtualbox-iso: Retrieving Guest additions
    ==> virtualbox-iso: Trying /usr/share/virtualbox/VBoxGuestAdditions.iso
    ...
    • Sep 30 2020, 10:02 AM
    • 648 Lines
    Author: ardumont
  • P792 WTF isort minimal
    $ cat db.py
    def a():
    yield f(
    """
    select %s from (values %%s) as t(%s)
    ...
    • Sep 29 2020, 5:29 PM
    • 67 Lines
    • Diff
    Author: vlorentz
  • P791 WTF isort
    $ git checkout swh/indexer/storage/db.py
    $ git diff swh/indexer/storage/db.py | cat
    $ isort swh/indexer/storage/db.py
    ...
    • Sep 29 2020, 5:10 PM
    • 63 Lines
    • Diff
    Author: vlorentz
  • P790 (An Untitled Masterwork)
    class TestMount(unittest.TestCase):
    def setUp(self):
    self.mntdir = TemporaryDirectory(suffix=".swh-fuse-test")
    def fuse_daemon():
    ...
    • Sep 29 2020, 3:25 PM
    • 18 Lines
    • Python
    Author: haltode
  • P789 vagrant up test failure
    ```
    The following SSH command responded with a non-zero exit status.
    Vagrant assumes that this means the command failed!
    mount -o vers=3,udp 10.168.100.1:/tmp/puppet /tmp/puppet
    ...
    • Sep 28 2020, 8:29 PM
    • 32 Lines
    Author: ardumont
  • P788 ram redistribution
    ```
    |--------------+---------------------+-------------------------|
    | staging node | allocated RAM (Gib) | new allocated RAM (Gib) |
    |--------------+---------------------+-------------------------|
    | gateway | 1 | 1 |
    ...
    • Sep 28 2020, 7:33 PM
    • 18 Lines
    Author: ardumont
  • P787 storage flush - no more default value for object_types argument
    14:35 $ pytest -sv swh/web/tests/api/views/test_graph.py
    ============================================================================================== test session starts ===============================================================================================
    platform linux -- Python 3.7.3, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 -- /home/anlambert/.virtualenvs/swh/bin/python3
    cachedir: .pytest_cache
    django: settings: swh.web.settings.tests (from ini)
    ...
    • Sep 28 2020, 2:39 PM
    • 51 Lines
    Author: anlambert
  • P786 (An Untitled Masterwork)
     ~/s/s/swh-model  1  gbp buildpackage --git-builder=sbuild -As --force-orig-source --extra-repository='deb [trusted=yes] https://debian.softwareheritage.org/ sid main'
    gbp:info: Performing the build
    dh clean --with python3 --buildsystem=pybuild
    dh: warning: Compatibility levels before 10 are deprecated (level 9 in use)
    dh_auto_clean -O--buildsystem=pybuild
    ...
    • Sep 25 2020, 6:37 PM
    • 32 Lines
    Author: douardda
  • P785 Cannot make click display my multiline help
    from pprint import pformat
    @swh_cli_group.group(name="scanner", context_settings=CONTEXT_SETTINGS)
    @click.option(
    "-C",
    "--config-file",
    ...
    • Sep 25 2020, 3:55 PM
    • 17 Lines
    • Python
    Author: tenma
  • P784 Tox another fail in swh-scanner
    (swh) tenma@swh-ng1:~/Dev/swh-env/swh-scanner$ tox -r -e mypy
    GLOB sdist-make: /home/tenma/Dev/swh-env/swh-scanner/setup.py
    mypy recreate: /home/tenma/Dev/swh-env/swh-scanner/.tox/mypy
    mypy installdeps: mypy
    mypy inst: /home/tenma/Dev/swh-env/swh-scanner/.tox/.tmp/package/1/swh.scanner-0.2.1.dev2+ga7a17e7.d20200925.zip
    ...
    • Sep 25 2020, 2:15 PM
    • 20 Lines
    Author: tenma
  • P783 Command-Line Input
    commit a7a17e7ed6a45941d04e30f239a6e6eb62ecca25
    Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
    Date: Fri Sep 25 14:05:40 2020 +0200
    Pin black in tox to the same version as pre-commit
    ...
    • Sep 25 2020, 2:05 PM
    • 40 Lines
    Author: olasd
  • P782 Tox fail in swh-scanner
    (swh) tenma@swh-ng1:~/Dev/swh-env/swh-scanner$ tox
    GLOB sdist-make: /home/tenma/Dev/swh-env/swh-scanner/setup.py
    black create: /home/tenma/Dev/swh-env/swh-scanner/.tox/black
    black installdeps: black
    black installed: appdirs==1.4.4,black==20.8b1,click==7.1.2,mypy-extensions==0.4.3,pathspec==0.8.0,regex==2020.7.14,toml==0.10.1,typed-ast==1.4.1,typing-extensions==3.7.4.3
    ...
    • Sep 25 2020, 1:59 PM
    • 83 Lines
    Author: tenma
  • P781 (An Untitled Masterwork)
    py3 recreate: /home/ddouard/swh/swh-environment/swh-model/.tox/py3
    py3 installdeps: pytest-cov
    py3 inst: /home/ddouard/swh/swh-environment/swh-model/.tox/.tmp/package/1/swh.model-0.6.7.dev9+gdb723f0.d20200925.zip
    py3 installed: attrs==20.2.0,attrs-strict==0.2.0,certifi==2020.6.20,click==7.1.2,coverage==5.3,Deprecated==1.2.10,dulwich==0.20.6,hypothesis==5.36.0,iniconfig==1.0.1,iso8601==0.1.13,more-itertools==8.5.0,packaging==20.4,pluggy==0.13.1,py==1.9.0,pyparsing==2.4.7,pytest==6.0.2,pytest-cov==2.10.1,python-dateutil==2.8.1,pytz==2020.1,PyYAML==5.3.1,sentry-sdk==0.17.8,six==1.15.0,sortedcontainers==2.2.2,swh.core==0.3.0,swh.model @ file:///home/ddouard/swh/swh-environment/swh-model/.tox/.tmp/package/1/swh.model-0.6.7.dev9%2Bgdb723f0.d20200925.zip,toml==0.10.1,typing-extensions==3.7.4.3,urllib3==1.25.10,wrapt==1.12.1
    py3 run-test-pre: PYTHONHASHSEED='631895245'
    ...
    • Sep 25 2020, 12:59 PM
    • 56 Lines
    Author: douardda
  • P780 (An Untitled Masterwork)
    Usage: swh [OPTIONS] COMMAND [ARGS]...
    Command line interface for Software Heritage.
    Options:
    ...
    • Sep 25 2020, 12:36 PM
    • 35 Lines
    Author: douardda
  • P779 (An Untitled Masterwork)
    def inode2swhid(self, inode: int) -> SWHID:
    try:
    return self._inode2swhid[inode]
    except KeyError:
    raise pyfuse3.FUSEError(errno.ENOENT)
    • Sep 25 2020, 11:37 AM
    • 5 Lines
    • Python
    Author: haltode
  • P778 Masterwork From Distant Lands
    $ workon swh
    WARNING:swh.core.cli:Could not load subcommand indexer: module 'swh.indexer.cli' has no attribute 'cli'
    WARNING:swh.core.cli:Could not load subcommand search: module 'swh.search.cli' has no attribute 'cli'
    WARNING:swh.core.cli:Could not load subcommand graph: (swh.core 0.2.4.dev2+g60c535d (/home/zack/dati/projects/sw-heritage/git/swh-environment/swh-core), Requirement.parse('swh.core[http]>=0.3'))
    WARNING:swh.core.cli:Could not load subcommand icinga_plugins: module 'swh.icinga_plugins.cli' has no attribute 'cli'
    ...
    • Sep 25 2020, 11:20 AM
    • 7 Lines
    Author: zack
  • P777 Masterwork From Distant Lands
    [2020-09-25 06:23:57] ERROR 8192: implode(): Passing glue string after array is deprecated. Swap the parameters at [/usr/share/libphutil/src/utils/utils.php:1572]
    arcanist(), phutil()
    #0 implode(array, string) called at [<phutil>/src/utils/utils.php:1572]
    #1 phutil_build_http_querystring(array) called at [<phutil>/src/future/http/HTTPSFuture.php:526]
    #2 HTTPSFuture::formatRequestDataForCURL() called at [<phutil>/src/future/http/HTTPSFuture.php:247]
    ...
    • Sep 25 2020, 8:24 AM
    • 11 Lines
    Author: zack
  • P776 hackmd2rst
    #!/usr/bin/perl -w
    # converts markdown exported from hackmd.io to reStructuredText
    # SPDX-FileCopyrightText: © 2020 Stefano Zacchiroli <zack@upsilon.cc>
    ...
    • Sep 23 2020, 11:14 PM
    • 26 Lines
    • Perl
    Author: zack
  • P775 Eclipse Java coding style config
    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <profiles version="20">
    <profile kind="CodeFormatterProfile" name="Spotless" version="20">
    <setting id="org.eclipse.jdt.core.formatter.insert_space_after_ellipsis" value="insert"/>
    <setting id="org.eclipse.jdt.core.formatter.insert_space_after_comma_in_enum_declarations" value="insert"/>
    ...
    • Sep 23 2020, 4:27 PM
    • 383 Lines
    • XML
    Author: haltode
  • P774 Masterwork From Distant Lands
    <code_scheme name="Default" version="173">
    <JavaCodeStyleSettings>
    <option name="JD_ALIGN_PARAM_COMMENTS" value="false" />
    <option name="JD_ALIGN_EXCEPTION_COMMENTS" value="false" />
    <option name="JD_DO_NOT_WRAP_ONE_LINE_COMMENTS" value="true" />
    ...
    • Sep 23 2020, 3:03 PM
    • 12 Lines
    Author: seirl
  • P773 storage check at startup in deposit tests: dbversion discrepancy make the tests refuse to start...
    ```
    >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
    > /home/tony/work/inria/repo/swh/swh-environment/swh-deposit/.tox/py3-django2/lib/python3.7/site-packages/swh/storage/postgresql/db.py(1344)check_dbversion()
    -> return self.dbversion()["version"] == self.current_version
    (Pdb) self.dbversion()
    ...
    • Sep 23 2020, 10:08 AM
    • 11 Lines
    Author: ardumont
  • P772 (An Untitled Masterwork)
    14:22:57 <+seirl> huh, D3906 didn't autoclose for some reason, but i updated it before merging it in master
    14:22:58 -- Notice(swhbot): D3906 (author: seirl, Accepted) on swh-graph: Add edge labelling prototype <https://forge.softwareheritage.org/D3906>
    14:23:13 <+seirl> i really don't understand the heuristic it uses
    14:23:29 <+seirl> or maybe it's just slow
    14:24:19 <+olasd> seirl: I see the arc diff --update at 14:21 and the push at 14:17
    ...
    • Sep 18 2020, 12:07 PM
    • 28 Lines
    Author: seirl
  • P771 Masterwork From Distant Lands
    make[1]: Entering directory '/home/zack/dati/projects/sw-heritage/git/swh-environment/swh-web'
    pre-commit run --all-files
    Trim Trailing Whitespace.................................................Passed
    Check JSON...............................................................Passed
    Check Yaml...............................................................Passed
    ...
    • Sep 18 2020, 10:49 AM
    • 161 Lines
    Author: zack
  • P770 tb in the mirror
    swh-mirror_graph-replayer-all.1.yixgwkzu2zwj@libra | Traceback (most recent call last):
    swh-mirror_graph-replayer-all.1.yixgwkzu2zwj@libra | File "/usr/bin/swh", line 11, in <module>
    swh-mirror_graph-replayer-all.1.yixgwkzu2zwj@libra | load_entry_point('swh.core==0.2.3', 'console_scripts', 'swh')()
    swh-mirror_graph-replayer-all.1.yixgwkzu2zwj@libra | File "/usr/lib/python3/dist-packages/swh/core/cli/__init__.py", line 122, in main
    swh-mirror_graph-replayer-all.1.yixgwkzu2zwj@libra | return swh(auto_envvar_prefix="SWH")
    ...
    • Sep 16 2020, 1:12 PM
    • 40 Lines
    Author: douardda
  • P769 Python 3.10 GADT
    import dataclasses
    import typing
    # need to set _root so typing._Final thinks we're allowed to be a subclass
    class MyGenericAlias(typing._GenericAlias, _root=True):
    ...
    • Sep 15 2020, 9:53 PM
    • 85 Lines
    • Python
    Author: vlorentz
  • P768 For haltode :-)
    #!/bin/bash
    set -e
    mkdir -p tmp
    ...
    • Sep 15 2020, 2:28 PM
    • 50 Lines
    • Bash Scripting
    Author: seirl
  • P767 Command-Line Input
    diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml
    index 00e502c..be1a07d 100644
    --- a/docker/docker-compose.yml
    +++ b/docker/docker-compose.yml
    @@ -23,6 +23,7 @@ services:
    ...
    • Sep 15 2020, 12:52 PM
    • 29 Lines
    Author: olasd
  • P766 current size of directory-related DB entities (swh-replica cluster)
    entity entity type size (GB)
    directory table 1666
    directory_entry_dir table 413
    directory_entry_file table 749
    directory_entry_rev table 1
    ...
    • Sep 15 2020, 12:38 PM
    • 11 Lines
    Author: zack
  • P765 (An Untitled Masterwork)
    DEBUG:swh.journal.client.rdkafka:SUBSCRIBE [rdkafka#consumer-1] [thrd:main]: Group "check-journal-data-0002": subscribe to new subscription of 8 topics (join state init)
    DEBUG:swh.journal.client.rdkafka:REBALANCE [rdkafka#consumer-1] [thrd:main]: Group "check-journal-data-0002" is rebalancing in state wait-broker-transport (join-state init) without assignment: unsubscribe
    ...
    • Sep 15 2020, 12:12 PM
    • 7 Lines
    Author: douardda
  • P764 developer setup debian packages doc
    python3 python3-pip python3-venv virtualenvwrapper \ # Python development tools
    build-essential pkg-config lzip \ # basic system utilities
    libsystemd-dev libpython3-dev libsvn-dev \ # to build against
    git myrepos \ # to manage repositories
    postgresql libpq-dev \ # main database
    ...
    • Sep 15 2020, 11:35 AM
    • 7 Lines
    Author: tenma
  • P763 pg-dev package dependecies
    apt-cache depends --installed postgresql-server-dev-12
    postgresql-server-dev-12
    Depends: clang-7
    Depends: libpq-dev
    Depends: llvm-7-dev
    ...
    • Sep 15 2020, 10:45 AM
    • 8 Lines
    Author: tenma
  • P762 (An Untitled Masterwork)
    >>> bar
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    NameError: name 'bar' is not defined
    >>> class Foo:
    ...
    • Sep 13 2020, 8:59 PM
    • 14 Lines
    • Python
    Author: vlorentz
  • P761 Python 3.10 Maybe
    import typing
    from typing import *
    @typing.sealed
    class Maybe(Generic[T]):
    ...
    • Sep 13 2020, 2:48 PM
    • 61 Lines
    • Python
    Author: vlorentz
  • P760 Python 3.10 ADT
    from __future__ import annotations
    import dataclasses
    import typing
    ...
    • Sep 13 2020, 10:43 AM
    • 35 Lines
    • Python
    Author: vlorentz
  • P759 git diff WTF on imports
    import asyncio
    <<<<<<< HEAD
    from pathlib import PosixPath
    from typing import List, Dict, Tuple, Iterator, Union, Iterable, Pattern, Any
    ...
    • Sep 11 2020, 5:06 PM
    • 30 Lines
    Author: tenma
First
Prev
Next
About · Developer information · Legal