Page MenuHomeSoftware Heritage
Create Paste
Paste Active Pastes

Active Pastes
Use Results
Edit Query

  • P566 test_origin_url_repeat_order2
    self.search.origin_update([
    {'url': 'https://bitbucket.org/bitbucket0145/bitbucket_repo.git'},
    {'url': 'https://gitorious.org/railstutorial/railstutorial.git'},
    {'url': 'https://bitbucket.org/bittelc/railstutorial.git'},
    ])
    ...
    • Dec 4 2019, 4:40 PM
    • 11 Lines
    • Python
    Author: vlorentz
  • P564 revisions in journal
    [b'3\xbe\x07S\xcf(~j\xc0\xc0C\xd6\xea\xe6-\x1f\xd3&7;',
    b'\xa2\x82b\xfcGdS\x82O\xe0\x00\xf9\xda{\x85!\x1b\x82\x9d\x07',
    b'\x1c\xff\x7f\xb26\xdb[\xda3\xde\x11\xe5H\xa04\x02\x12\x9b\x8c\xf4',
    b'?J\x9c]\xc1\x1c\x13|\xb9\xd3.\x0eO\xf0\x9e2\xaf\x15M\x15',
    b'-q\x82E\x03\xd1\xf6\xa0G\x14S\xf8\xa0\t\xfdu?\x9e\xb02',
    ...
    • Nov 28 2019, 4:04 PM
    • 20 Lines
    Author: douardda
  • P563 dump_elasticsearch.py
    import json
    from pprint import pprint
    import re
    import elasticsearch
    ...
    • Nov 25 2019, 2:07 PM
    • 46 Lines
    • Python
    Author: vlorentz
  • P561 (An Untitled Masterwork)
    default: &default_settings
    memory: 200G
    java_tool_options: -XXlol
    compress:
    <<: *default_settings
    ...
    • Nov 8 2019, 2:49 PM
    • 6 Lines
    Author: olasd
  • P560 (An Untitled Masterwork)
    default:
    memory: 200G
    java_tool_options: -XXlol
    compress:
    memory: 1000G
    • Nov 8 2019, 2:47 PM
    • 5 Lines
    • YAML
    Author: seirl
  • P558 about where/when to put the lister's needed scheduler task types: conclusion in the constructor
    11:18 <+ardumont> douardda: D2237 draft to check for missing task types
    11:18 -- Notice(swhbot): D2237 (author: ardumont, Needs Review) on swh-lister: lister: Add checks on expected scheduler's output tasks <https://forge.softwareheritage.org/D2237>
    11:18 <+ardumont> i'm not sure when/where to plug that check though
    11:21 <+olasd> if we're doing that, we might just as well create the task type with the proper settings
    11:25 <+ardumont> mmm, unsure
    ...
    • Nov 8 2019, 12:00 PM
    • 33 Lines
    Author: ardumont
  • P556 swh/graph/tests/test_cli.py::TestCompress::test_pipeline
    swh/graph/tests/test_cli.py::TestCompress::test_pipeline
    -------------------------------------------------------------------------------------------------------------------------------- live log call --------------------------------------------------------------------------------------------------------------------------------
    webgraph.py 233 INFO starting compression
    webgraph.py 242 INFO starting compression step MPH (1/11)
    webgraph.py 153 INFO running: java it.unimi.dsi.sux4j.mph.GOVMinimalPerfectHashFunction --zipped /tmp/tmp465gdf9e.swh-graph-test/example.mph --temp-dir /tmp/tmp465gdf9e.swh-graph-test/tmp /home/antoine/swh/swh-environment/swh-graph/swh/graph/tests/dataset/example.nodes.csv.gz
    ...
    • Nov 4 2019, 12:01 PM
    • 49 Lines
    Author: anlambert
  • P549 Origins with a codemeta.json at root
    url
    ------------------------------------------------------------------------------------------------
    https://github.com/rootpy/root_numpy
    https://github.com/stevengj/mpb
    https://github.com/barbagroup/pygbe
    ...
    • Oct 15 2019, 2:49 PM
    • 263 Lines
    Author: vlorentz
  • P548 Masterwork From Distant Lands
    package org.softwareheritage.graph;
    import org.softwareheritage.graph.algo.Traversal;
    import java.io.OutputStream;
    ...
    • Oct 14 2019, 1:52 PM
    • 67 Lines
    • Java
    Author: seirl
  • P547 (An Untitled Masterwork)
    ERR-CONDUIT-CORE: Graph cycle detected (type=5, cycle=PHID-DREV-vvlfxmyjkkrcdnbfzb5l, PHID-DREV-sz2tjc63iowtoyay6mey, PHID-DREV-ni5tcyma6542fypzaalx, PHID-DREV-q6y4zi7eciqddatexzxl, PHID-DREV-tsal5vzfklovorhkqvzn, PHID-DREV-rtmmo2kgo7wacl4warhv, PHID-DREV-5muna6xupz7kvfkz2yuo, PHID-DREV-ee7627jtcof4i73qqrfm, PHID-DREV-kf4oob54dw4sttlkv2az, PHID-DREV-vvlfxmyjkkrcdnbfzb5l).
    • Oct 14 2019, 12:28 PM
    • 1 Line
    Author: douardda
  • P546 git pull --rebase
    git pull --rebase
    remote: Enumerating objects: 21, done.
    remote: Counting objects: 100% (21/21), done.
    remote: Compressing objects: 100% (11/11), done.
    remote: Total 12 (delta 6), reused 0 (delta 0)
    ...
    • Oct 9 2019, 2:58 PM
    • 15 Lines
    Author: ardumont
  • P545 Data
    {'_id': ObjectId('5ad99f9fbd95630dfc4b9a4e'),
    'graphql': {'user': {'biography': 'International educator, traveler, music '
    'lover, photographer, Californian.',
    'blocked_by_viewer': False,
    'connected_fb_page': None,
    ...
    • Oct 8 2019, 6:30 PM
    • 640 Lines
    Author: nahimilega
  • P544 about pytest-plugin out of the swh arborescence tree
    17:09 <+douardda> and the pb should not happen if we put the plugin out of the swh package (i.e. ar the root of swh-core directory)
    17:10 <+douardda> (I've reproduced the issue in a minimal src repo)
    17:12 <+douardda> the problem seems to be that when the plugin lives under our package's hat, it's loaded very soon, thus swh.core is loaded but from the installed location (in the tox venv here)
    17:13 <+douardda> so when the subsequent import statement for a conftest or a testfile occurs, it's looked under this package's root directory first
    17:14 <+douardda> so the simple solution is to put this plugin in a dedicated python module not under the swh's (and especially the swh.core one I think) package
    ...
    • Oct 8 2019, 11:49 AM
    • 6 Lines
    Author: ardumont
  • P543 (An Untitled Masterwork)
    Bad:
    Returns
    List of tarball urls and their associated metadata (time, length).
    For example:
    ...
    • Oct 8 2019, 11:33 AM
    • 21 Lines
    Author: vlorentz
  • P540 List R package versions
    ```
    $ sudo apt install r-base
    ```
    ```
    ...
    • Oct 6 2019, 11:49 AM
    • 34 Lines
    Author: ardumont
  • P537 about asking gnu sysadmins to add checksum to their listing manifest
    Hello,
    It's the time of the year where I ask you (again!) for your help to better archive GNU source code in the Software Heritage archive.
    Would it be possible to change the format of the GNU file listing [1] to also include SHA256 checksums?
    ...
    • Oct 1 2019, 12:10 PM
    • 16 Lines
    Author: ardumont
  • P536 ~/.config/swh/kibana/query.yml
    indexes:
    - swh_workers-2018.03.*
    size: 100
    from: 0
    ...
    • Sep 30 2019, 7:47 PM
    • 27 Lines
    Author: ardumont
  • P534 (An Untitled Masterwork)
    diff --git a/Makefile b/Makefile
    index 524175c..be63f09 100644
    --- a/Makefile
    +++ b/Makefile
    @@ -3,3 +3,12 @@
    ...
    • Sep 27 2019, 7:10 PM
    • 29 Lines
    • Diff
    Author: seirl
  • P531 pytest segfault swh-journal
    ✘ ⚙ dev@desktop5  ~/swh-environment/swh-journal   bencode-key ●  pytest
    ========================================================================================================= test session starts =========================================================================================================
    platform linux -- Python 3.5.3, pytest-5.0.1, py-1.7.0, pluggy-0.12.0
    hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/dev/swh-environment/swh-journal/.hypothesis/examples')
    rootdir: /home/dev/swh-environment/swh-journal, inifile: pytest.ini
    ...
    • Sep 23 2019, 3:00 PM
    • 51 Lines
    Author: vlorentz
  • P529 migrate swh-lister tasks (in scheduler db) following the lister normalization
    -- gitlab (renamed key "api_baseurl" to "url")
    update task set arguments='{"args": [], "kwargs": {"instance": "inria", "url": "https://gitlab.inria.fr/api/v4"}}' where arguments#>>'{kwargs,instance}' = 'inria' and type in ('list-gitlab-full', 'list-gitlab-incremental');
    update task set arguments='{"args": [], "kwargs": {"instance": "framagit", "url": "https://framagit.org/api/v4"}}' where arguments#>>'{kwargs,instance}' = 'framagit' and type in ('list-gitlab-full', 'list-gitlab-incremental');
    update task set arguments='{"args": [], "kwargs": {"instance": "riseup", "url": "https://0xacab.org/api/v4"}}' where arguments#>>'{kwargs,instance}' = 'riseup' and type in ('list-gitlab-full', 'list-gitlab-incremental');
    update task set arguments='{"args": [], "kwargs": {"instance": "gitlab", "url": "https://gitlab.com/api/v4"}}' where arguments#>>'{kwargs,instance}' = 'gitlab' and type in ('list-gitlab-full', 'list-gitlab-incremental');
    ...
    • Sep 11 2019, 3:54 PM
    • 35 Lines
    • SQL
    Author: ardumont
  • P528 (An Untitled Masterwork)
    messages: 168300
    messages: 168400
    messages: 168500
    messages: 168600
    messages: 168700
    ...
    • Sep 11 2019, 3:18 PM
    • 23 Lines
    Author: douardda
  • P527 replayer TB
    swh_graph-replayer.1.khy6rzsfohdz@mirror-node-3 | INFO:swh.journal.cli:Processed 35000 messages.
    swh_graph-replayer.1.khy6rzsfohdz@mirror-node-3 | Traceback (most recent call last):
    swh_graph-replayer.1.khy6rzsfohdz@mirror-node-3 | File "/usr/bin/swh", line 11, in <module>
    swh_graph-replayer.1.khy6rzsfohdz@mirror-node-3 | load_entry_point('swh.core==0.0.67', 'console_scripts', 'swh')()
    swh_graph-replayer.1.khy6rzsfohdz@mirror-node-3 | File "/usr/lib/python3/dist-packages/swh/core/cli/__init__.py", line 56, in main
    ...
    • Sep 10 2019, 5:14 PM
    • 36 Lines
    Author: douardda
  • P526 replayer
    swh_graph-replayer.1.p18dcgg4lwq5@mirror-node-3 | INFO:swh.journal.cli:Processed 124000 messages.
    swh_graph-replayer.1.p18dcgg4lwq5@mirror-node-3 | INFO:kafka.client:Closing idle connection 1, last active 540038 ms ago
    swh_graph-replayer.1.p18dcgg4lwq5@mirror-node-3 | INFO:kafka.conn:<BrokerConnection node_id=1 host=kafka01.euwest.azure.internal.softwareheritage.org:9092 <connected> [IPv4 ('192.168.200.24', 9092)]>: Closing connection.
    swh_graph-replayer.1.p18dcgg4lwq5@mirror-node-3 | INFO:kafka.client:Closing idle connection 3, last active 540139 ms ago
    swh_graph-replayer.1.p18dcgg4lwq5@mirror-node-3 | INFO:kafka.conn:<BrokerConnection node_id=3 host=kafka03.euwest.azure.internal.softwareheritage.org:9092 <connected> [IPv4 ('192.168.200.31', 9092)]>: Closing connection.
    ...
    • Sep 10 2019, 1:50 PM
    • 9 Lines
    Author: douardda
  • P525 Kafka consumer crash
    Sep 10 11:59:01 desktop5 replayer-12301[2746]: INFO:kafka.conn:<BrokerConnection node_id=11 host=esnode1.internal.softwareheritage.org:9092 <connecting> [IPv4 ('192.168.100.61', 9092)]>: connecting to esnode1.internal.softwareheritage.org:9092 [('192.168.100.61', 9092) IPv4]
    Sep 10 11:59:06 desktop5 replayer-12301[2746]: INFO:kafka.conn:<BrokerConnection node_id=12 host=esnode2.internal.softwareheritage.org:9092 <connecting> [IPv4 ('192.168.100.62', 9092)]>: connecting to esnode2.internal.softwareheritage.org:9092 [('192.168.100.62', 9092) IPv4]
    Sep 10 11:59:06 desktop5 replayer-12301[2746]: INFO:kafka.conn:<BrokerConnection node_id=11 host=esnode1.internal.softwareheritage.org:9092 <connecting> [IPv4 ('192.168.100.61', 9092)]>: Connection complete.
    Sep 10 11:59:06 desktop5 replayer-12301[2746]: INFO:kafka.conn:<BrokerConnection node_id=12 host=esnode2.internal.softwareheritage.org:9092 <connecting> [IPv4 ('192.168.100.62', 9092)]>: Connection complete.
    Sep 10 11:59:19 desktop5 replayer-12301[2746]: WARNING:kafka.coordinator:Heartbeat session expired, marking coordinator dead
    ...
    • Sep 10 2019, 12:00 PM
    • 43 Lines
    Author: vlorentz
  • P524 (An Untitled Masterwork)
    ERROR:root:An error occurred while calling o0.visit.
    : java.lang.ArrayIndexOutOfBoundsException: Index 272313807 out of bounds for length 121824
    at it.unimi.dsi.bits.LongArrayBitVector.getBoolean(LongArrayBitVector.java:374)
    at org.softwareheritage.graph.algo.Traversal.visitNodesVisitor(Traversal.java:160)
    at org.softwareheritage.graph.Entry.visit(Entry.java:28)
    ...
    • Sep 9 2019, 7:33 PM
    • 108 Lines
    Author: seirl
  • P523 swh-web origin id requests
    /browse/origin/26984/latest_snapshot/
    /browse/origin/34423/latest_snapshot/
    /browse/origin/21387/latest_snapshot/
    /browse/origin/48567/latest_snapshot/
    /browse/origin/29526/latest_snapshot/
    ...
    • Sep 9 2019, 2:59 PM
    • 851 Lines
    Author: anlambert
  • P522 swh-web origin requests
    import json
    import requests
    from pprint import pprint
    ...
    • Sep 9 2019, 2:53 PM
    • 74 Lines
    • Python
    Author: anlambert
  • P520 [fixed] broken ci: index failure status
    swh/indexer/tests/test_metadata.py .................................F
    ================================================================================================================== FAILURES ===================================================================================================================
    ___________________________________________________________________________________________________ Metadata.test_revision_metadata_indexer ___________________________________________________________________________________________________
    ...
    • Sep 5 2019, 2:55 PM
    • 92 Lines
    Author: ardumont
  • P516 Updated README for compressed graph on annex
    # Graph compression output
    Compression script and environment used: https://forge.softwareheritage.org/source/swh-graph/browse/master/dockerfiles/
    - Direct compressed graph: `all.{graph,obl,offsets,properties}`
    ...
    • Aug 27 2019, 9:20 PM
    • 8 Lines
    Author: haltode
  • P514 haltode swh-graph commits list
    https://forge.softwareheritage.org/rDGRPHb31d2e86a80cf8b85d4bf51f30be8e463fe994e4
    https://forge.softwareheritage.org/rDGRPH0b46253799f43a25a8528926052340f93a1a911b
    https://forge.softwareheritage.org/rDGRPHc7363b064ae1ed52c271b9831b934cd196589c8e
    https://forge.softwareheritage.org/rDGRPHb6c6e1eec131a002a44e01cef17abb81ec958421
    https://forge.softwareheritage.org/rDGRPHd5dcbfcdf245777a8753ccc6ac5414e762605abe
    ...
    • Aug 27 2019, 2:02 PM
    • 186 Lines
    Author: haltode
  • P511 swh-web : run-migrations-tests fails
    Command : $ django-admin shell --settings=swh.web.settings.tests
    /home/kalpitk/.virtualenvs/swh/lib/python3.6/site-packages/swh/scheduler/__init__.py:69: DeprecationWarning: Call to deprecated class SWHRemoteAPI. (Use the RPCClient instead) -- Deprecated since version 0.0.64.
    return SchedulerBackend(**args)
    Traceback (most recent call last):
    ...
    • Aug 22 2019, 11:45 AM
    • 38 Lines
    • Bash Scripting
    Author: kalpitk
  • P510 dulwich-based fake date git commit proto-prototype
    #!/usr/bin/env python3
    import sys
    import dateutil.parser
    ...
    • Aug 20 2019, 2:40 PM
    • 36 Lines
    • Python
    Author: olasd
  • P509 (An Untitled Masterwork)
    On rioc:
    (base) [zacchiro@rioc graph]$ cat all+ori.*.count
    164513699014
    11683687950
    ...
    • Aug 19 2019, 10:44 AM
    • 10 Lines
    Author: haltode
  • P508 Error while running gnu lister
    (swh) archit@work-pc:~/swh-environment/swh-lister/swh/lister$ python jjj.py
    DEBUG:swh.lister.core.lister_base:Loading config from lister_gnu
    INFO:swh.core.config:Loading config file /home/archit/.config/swh/lister_gnu.yml
    DEBUG:swh.lister.core.lister_base:<swh.lister.gnu.lister.GNULister object at 0x7ffacb9f7a20> CONFIG={'content_size_limit': 104857600, 'log_db': 'dbname=softwareheritage-log', 'storage': {'cls': 'remote', 'args': {'url': 'http://localhost:5002/'}}, 'scheduler': {'cls': 'remote', 'args': {'url': 'http://localhost:5008/'}}, 'lister': {'cls': 'local', 'args': {'db': 'postgresql:///lister-gnu'}}, 'credentials': [], 'cache_responses': True, 'cache_dir': '/home/archit/.cache/swh/lister/gnu/'}
    DEBUG:urllib3.util.retry:Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)
    ...
    • Aug 18 2019, 2:35 PM
    • 91 Lines
    Author: nahimilega
  • P507 Graph benchmarks: vault use-case
    Done on rioc:
    Used 1000 random nodes (results are in seconds):
    'git bundle' use-case
    ...
    • Aug 15 2019, 9:46 PM
    • 15 Lines
    Author: haltode
  • P506 result after running loader
    swh-storage=# select * from origin_visit where origin=424 ;
    origin | visit | date | type | status | metadata | snapshot
    --------+-------+-------------------------------+------+---------+----------+--------------------------------------------
    424 | 1 | 2019-08-14 10:02:03.190532+00 | gnu | partial | |
    424 | 2 | 2019-08-14 10:02:46.964868+00 | gnu | full | | \x1f3305edbd687c27ca005f7cafea3d6f809c38d1
    ...
    • Aug 14 2019, 1:32 PM
    • 16 Lines
    Author: nahimilega
  • P505 Browsing benchmarks early results
    Done on rioc:
    Used 100000 random nodes (results are in seconds):
    'ls' use-case
    ...
    • Aug 14 2019, 11:22 AM
    • 27 Lines
    Author: haltode
  • P504 All revision
    Print of `revisions` at this line
    https://forge.softwareheritage.org/source/swh-storage/browse/master/swh/storage/storage.py$692
    [{'author': {'email': 'robot@softwareheritage.org',
    'fullname': 'Software Heritage',
    ...
    • Aug 14 2019, 10:32 AM
    • 182 Lines
    Author: nahimilega
  • P503 swh storage logs
    swh-storage_1 | [2019-08-13 20:10:39 +0000] [41] [DEBUG] POST /content/missing
    swh-storage_1 | [2019-08-13 20:10:39 +0000] [41] [DEBUG] POST /directory/missing
    swh-storage_1 | [2019-08-13 20:10:39 +0000] [41] [DEBUG] POST /revision/missing
    swh-storage_1 | [2019-08-13 20:10:39 +0000] [41] [DEBUG] POST /revision/add
    swh-storage_1 | ERROR:root:Object of type bytes is not JSON serializable
    ...
    • Aug 13 2019, 10:12 PM
    • 39 Lines
    Author: nahimilega
  • P502 Error in the postactivate
    utkarsh@G3:~$ workon swh
    Usage:: command not found
    Command: command not found
    Options:: command not found
    INFO: command not found
    ...
    • Aug 12 2019, 3:26 PM
    • 12 Lines
    Author: GoodwillHunter
  • P501 Warning in swh scheduler
    (swh) utkarsh@G3:~/swh-environment$ swh scheduler task-type add list-packagist-full2 "swh.lister.packagist.tasks.PackagistListerTask" "Full PACKAGIST lister" --default-interval '1 day' --backoff-factor 1
    /home/utkarsh/.virtualenvs/swh/lib/python3.7/site-packages/swh/scheduler/__init__.py:69: DeprecationWarning: Call to deprecated class SWHRemoteAPI. (Use the RPCClient instead) -- Deprecated since version 0.0.64.
    return SchedulerBackend(**args)
    OK
    • Aug 12 2019, 2:48 PM
    • 4 Lines
    Author: GoodwillHunter
  • P500 full logs and revisions of avl
    There were two revisions for this package
    {'author': {'email': b'robot@softwareheritage.org',
    'fullname': b'Software Heritage',
    'name': b'Software Heritage'},
    ...
    • Aug 11 2019, 9:10 PM
    • 104 Lines
    Author: nahimilega
  • P499 Full logs and revision for hello
    swh-loader_1 | [2019-08-11 19:02:59,149: INFO/MainProcess] Received task: swh.loader.package.tasks.LoadGNU[11fa45c7-ffc8-47ad-9e18-dcff159f1bd2]
    swh-loader_1 | [2019-08-11 19:02:59,152: INFO/ForkPoolWorker-1] Loading config file /loader.yml
    swh-loader_1 | [2019-08-11 19:02:59,169: DEBUG/ForkPoolWorker-1] Creating gnu origin for https://ftp.gnu.org/gnu/hello/
    swh-loader_1 | [2019-08-11 19:02:59,304: DEBUG/ForkPoolWorker-1] Done creating gnu origin for https://ftp.gnu.org/gnu/hello/
    swh-loader_1 | [2019-08-11 19:02:59,304: DEBUG/ForkPoolWorker-1] Creating origin_visit for origin https://ftp.gnu.org/gnu/hello/ at time 2019-08-11 19:02:59.304333+00:00
    ...
    • Aug 11 2019, 9:06 PM
    • 82 Lines
    Author: nahimilega
  • P498 Full logs of dap gnu loader
    iled'}
    swh-loader_1 | [2019-08-11 18:50:23,779: INFO/MainProcess] Received task: swh.loader.package.tasks.LoadGNU[7a7e43e1-dd5b-4ebe-be5b-2ea10fd0bdbc]
    swh-loader_1 | [2019-08-11 18:50:23,871: INFO/ForkPoolWorker-1] Loading config file /loader.yml
    swh-loader_1 | [2019-08-11 18:50:23,915: DEBUG/ForkPoolWorker-1] Creating gnu origin for https://ftp.gnu.org/gnu/dap/
    swh-loader_1 | [2019-08-11 18:50:23,924: DEBUG/ForkPoolWorker-1] Done creating gnu origin for https://ftp.gnu.org/gnu/dap/
    ...
    • Aug 11 2019, 8:58 PM
    • 91 Lines
    Author: nahimilega
  • P497 logs
    wh-loader_1 | Using pip from /srv/softwareheritage/venv/bin/pip
    swh-loader_1 | Processing /src/swh-loader
    swh-loader_1 | Requirement already satisfied: vcversioner in /srv/softwareheritage/venv/lib/python3.7/site-packages (from swh.loader.core==0.0.44.post3) (2.16.0.0)
    swh-loader_1 | Requirement already satisfied: retrying in /srv/softwareheritage/venv/lib/python3.7/site-packages (from swh.loader.core==0.0.44.post3) (1.3.3)
    swh-loader_1 | Requirement already satisfied: psutil in /srv/softwareheritage/venv/lib/python3.7/site-packages (from swh.loader.core==0.0.44.post3) (5.6.3)
    ...
    • Aug 9 2019, 4:35 PM
    • 295 Lines
    Author: nahimilega
  • P496 NPM and GNU Revision
    NPM Revision
    {'synthetic': True,
    'metadata': {'package_source': {'name': 'ja','version': '0.0.1', 'filename': 'ja-0.0.1.tgz', 'sha1': '31399c51d3024f6eb91c626a31a175dc30f343e5', 'date': '2014-04-07T16:02:07.453Z', 'url': 'https://registry.npmjs.org/ja/-/ja-0.0.1.tgz', 'sha256': '8101e284d5846e77f9698628da660e92f8627535c266a3aef28549b25e63a597', 'blake2s256': 'fcf5438822e12c8a6973b9ecb40df4978947edd675c4269d728d5279dc82b203'},
    'package': {'author': 'Goldeneye Solutions', 'name': 'ja', 'description': 'Compose a cross-platform application from various things.',
    ...
    • Aug 9 2019, 3:56 PM
    • 95 Lines
    Author: nahimilega
  • P495 scheduler task
    id | type | arguments | next_run | current_interval | status | policy | retries_left | priority
    --------+---------------+----------------------------+-------------------------------+------------------+--------------------+---------+--------------+----------
    225784 | list-gnu-full | {"args": [], "kwargs": {}} | 2019-08-08 19:22:27.338807+00 | 90 days | next_run_scheduled | oneshot | 0 |
    225786 | list-gnu-full | {"args": [], "kwargs": {}} | 2019-08-08 19:25:28.450559+00 | 90 days | next_run_scheduled | oneshot | 0 |
    225787 | list-gnu-full | {"args": [], "kwargs": {}} | 2019-08-08 19:29:55.666438+00 | 90 days | disabled | oneshot | 0 |
    ...
    • Aug 9 2019, 7:47 AM
    • 8 Lines
    Author: nahimilega
  • P494 Error while running gnu lister
    swh-lister_1 | [2019-08-09 05:32:01,482: ERROR/ForkPoolWorker-1] Task swh.lister.gnu.tasks.GNUListerTask[e997c3cf-0dfc-423e-9460-3e8d685321b7] raised unexpected: NotNullViolation('null value in column "retries_left" violates not-null constraint\nDETAIL: Failing row contains (225791, load-tar, {"args": ["gcal", "https://ftp.gnu.org/gnu/gcal/"], "kwargs": {"..., 2019-08-09 05:32:01.13707+00, null, next_run_not_scheduled, recurring, null, null).\nCONTEXT: SQL statement "insert into task (type, arguments, next_run, status, current_interval, policy,\n retries_left, priority)\n select type, arguments, next_run, status, current_interval, policy,\n retries_left, priority\n from tmp_task t\n where not exists(select 1\n from task\n where type = t.type and\n arguments->\'args\' = t.arguments->\'args\' and\n arguments->\'kwargs\' = t.arguments->\'kwargs\' and\n policy = t.policy and\n priority is not distinct from t.priority and\n status = t.status)"\nPL/pgSQL function swh_scheduler_create_tasks_from_temp() line 12 at SQL statement\n')
    swh-lister_1 | Traceback (most recent call last):
    swh-lister_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/celery/app/trace.py", line 385, in trace_task
    swh-lister_1 | R = retval = fun(*args, **kwargs)
    swh-lister_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/scheduler/task.py", line 45, in __call__
    ...
    • Aug 9 2019, 7:35 AM
    • 41 Lines
    Author: nahimilega
  • P493 to puppetize
    Notice: /Stage[main]/Profile::Ssh::Server/Sshkey[ssh-worker1.internal.staging.swh.network-ecdsa-sha2-nistp256]/ensure: current_value absent, should be present (noop)
    Notice: /Stage[main]/Profile::Ssh::Server/Sshkey[ssh-worker0.internal.staging.swh.network-rsa]/ensure: current_value absent, should be present (noop)
    Notice: /Stage[main]/Profile::Ssh::Server/Sshkey[ssh-worker0.internal.staging.swh.network-ecdsa-sha2-nistp256]/ensure: current_value absent, should be present (noop)
    Notice: /Stage[main]/Profile::Ssh::Server/Sshkey[ssh-worker0.internal.staging.swh.network-ed25519]/ensure: current_value absent, should be present (noop)
    Notice: /Stage[main]/Profile::Ssh::Server/Sshkey[ssh-worker0.internal.staging.swh.network-dsa]/ensure: current_value absent, should be present (noop)
    ...
    • Aug 8 2019, 7:28 PM
    • 172 Lines
    Author: ardumont
  • P492 docker compose override
    version: '2'
    services:
    swh-objstorage:
    volumes:
    ...
    • Aug 8 2019, 3:41 PM
    • 15 Lines
    Author: nahimilega
  • P491 cassandra vs postgresql read benchmark
    Benchmark results for content_find:
    hash_algo = sha1 (sample size=263):
    cassandra: avg = 9 ms, stdev = 2.5 ms
    postgres: avg = 14 ms, stdev = 14.9 ms
    ...
    • Aug 8 2019, 1:47 PM
    • 72 Lines
    Author: vlorentz
  • P490 benchmark script cassandra vs postgresql
    from collections import defaultdict
    import csv
    import itertools
    import os
    from pprint import pprint
    ...
    • Aug 8 2019, 12:21 PM
    • 207 Lines
    • Python
    Author: vlorentz
  • P489 warm cache cassandra vs postgresql read benchmark
    Benchmark results for content_find:
    hash_algo = sha1 (sample size=248):
    avg cassandra = 5 ms
    avg postgres = 14 ms
    hash_algo = sha1_git (sample size=242):
    ...
    • Aug 8 2019, 12:16 PM
    • 25 Lines
    Author: vlorentz
  • P488 Error while loading snapshot
    DEBUG:swh.loader.package.GNULoader:Sending 2 revisions
    DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
    DEBUG:urllib3.connectionpool:http://localhost:5002 "POST /revision/add HTTP/1.1" 400 85
    ERROR:swh.loader.package.GNULoader:Loading failure, updating to `partial` status
    Traceback (most recent call last):
    ...
    • Aug 7 2019, 1:12 PM
    • 38 Lines
    Author: nahimilega
  • P487 [fixed] fresh new install, fresh new failure
    pip install $( ./bin/pip-swh-packages --with-testing )
    Obtaining file:///home/tony/work/inria/repo/swh/swh-environment/swh-core
    Obtaining file:///home/tony/work/inria/repo/swh/swh-environment/swh-model
    Obtaining file:///home/tony/work/inria/repo/swh/swh-environment/swh-core
    Obtaining file:///home/tony/work/inria/repo/swh/swh-environment/swh-objstorage
    ...
    • Aug 2 2019, 12:03 PM
    • 52 Lines
    Author: ardumont
  • P486 graph.tf
    variable "region" {
    type = "string"
    default = "northeurope"
    }
    ...
    • Aug 1 2019, 5:40 PM
    • 68 Lines
    Author: ardumont
  • P484 Get available free port
    # Get an available port number
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.bind(('127.0.0.1', 0))
    self.port = sock.getsockname()[1]
    sock.close()
    • Aug 1 2019, 10:05 AM
    • 5 Lines
    • Python
    Author: haltode
  • P483 gateway's update \m/
    Info: Using configured environment 'new_staging'
    Info: Retrieving pluginfacts
    Info: Retrieving plugin
    Info: Loading facts
    Info: Applying configuration version '1564591379'
    ...
    • Jul 31 2019, 6:45 PM
    • 59 Lines
    Author: ardumont
  • P482 CRAN Description
    Package: DZEXPM
    Type: Package
    Title: Estimation and Prediction of Skewed Spatial Processes
    Version: 1.0
    Date: 2017-06-24
    ...
    • Jul 25 2019, 6:56 PM
    • 14 Lines
    Author: nahimilega
  • P481 annex manual modifications
    root@pergamon:~# puppet agent --test --noop
    Info: Using configured environment 'production'
    Info: Retrieving pluginfacts
    Info: Retrieving plugin
    Info: Loading facts
    ...
    • Jul 24 2019, 4:27 PM
    • 68 Lines
    Author: ardumont
  • P480 git old dates
    $ git commit --date '1999-01-01' -m "foo" --allow-empty | grep Date
    Date: Fri Jan 1 12:59:34 1999 +0100
    $ git commit --date '1973-03-03' -m "foo" --allow-empty | grep Date
    Date: Sat Mar 3 13:00:21 1973 +0100
    $ git commit --date '1973-03-02' -m "foo" --allow-empty | grep Date
    ...
    • Jul 23 2019, 2:02 PM
    • 10 Lines
    Author: vlorentz
  • P479 mounting annex issues
    ardumont@pergamon:~% cat /etc/fstab
    # /etc/fstab: static file system information.
    #
    # Use 'blkid' to print the universally unique identifier for a
    # device; this may be used with UUID= as a more robust way to name devices
    ...
    • Jul 22 2019, 11:41 AM
    • 32 Lines
    Author: ardumont
  • P478 branch structure and a doubt
    /
    -----master----. b
    \ /
    ...
    • Jul 19 2019, 2:10 PM
    • 18 Lines
    Author: nahimilega
  • P477 (An Untitled Masterwork)
    (swh) archit@work-pc:~/swh-environment/swh-loader-core/swh/loader/gnu$ tox .
    GLOB sdist-make: /home/archit/swh-environment/swh-loader-core/setup.py
    flake8 installed: entrypoints==0.3,flake8==3.7.8,mccabe==0.6.1,pycodestyle==2.5.0,pyflakes==2.1.1,swh.loader.core==0.0.44.post2
    flake8 run-test-pre: PYTHONHASHSEED='1804426209'
    flake8 run-test: commands[0] | /home/archit/swh-environment/swh-loader-core/.tox/flake8/bin/python -m flake8
    ...
    • Jul 19 2019, 9:48 AM
    • 65 Lines
    Author: nahimilega
  • P476 pyblake2
    swh_content-replayer.1.bgt2xzycn7wo@desktop6 | Starting the SWH mirror content replayer
    swh_content-replayer.1.bgt2xzycn7wo@desktop6 | /usr/lib/python3/dist-packages/swh/storage/api/server.py:20: DeprecationWarning: Call to deprecated class SWHServerAPIApp. (Use the RPCServerApp instead) -- Deprecated since version 0.0.64.
    swh_content-replayer.1.bgt2xzycn7wo@desktop6 | app = SWHServerAPIApp(__name__)
    swh_content-replayer.1.bgt2xzycn7wo@desktop6 | Traceback (most recent call last):
    swh_content-replayer.1.bgt2xzycn7wo@desktop6 | File "/usr/bin/swh", line 11, in <module>
    ...
    • Jul 18 2019, 3:42 PM
    • 15 Lines
    Author: douardda
  • P475 docker compose
    kafka:
    image: wurstmeister/kafka
    environment:
    KAFKA_ADVERTISED_HOST_NAME: 127.0.0.1
    ports:
    ...
    • Jul 14 2019, 1:30 PM
    • 9 Lines
    Author: nahimilega
  • P474 kafka logs
    (swh) archit@work-pc:~/swh-environment$ doco logs kafka
    kafka_1 | log.cleaner.min.cleanable.ratio = 0.5
    kafka_1 | log.cleaner.min.compaction.lag.ms = 0
    ...
    • Jul 14 2019, 1:23 PM
    • 177 Lines
    Author: nahimilega
  • P473 swh storage logs
    (swh) archit@work-pc:~/swh-environment$ doco logs swh-storage
    ...
    • Jul 14 2019, 1:21 PM
    • 47 Lines
    Author: nahimilega
  • P472 origin not able to load
    INFO:swh.core.config:Loading config file /home/archit/.config/swh/loader/cran.yml
    /home/archit/swh-environment/swh-storage/swh/storage/api/client.py:13: DeprecationWarning: Call to deprecated class MetaRPCClient. (Use the MetaRPCClient instead) -- Deprecated since version 0.0.64.
    class RemoteStorage(SWHRemoteAPI):
    /home/archit/swh-environment/swh-storage/swh/storage/__init__.py:43: DeprecationWarning: Call to deprecated class RPCClient. (Use the RPCClient instead) -- Deprecated since version 0.0.64.
    return Storage(**args)
    ...
    • Jul 14 2019, 12:55 PM
    • 48 Lines
    Author: nahimilega
  • P471 error with base loader
    Traceback (most recent call last):
    File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/web/browse/views/utils/snapshot_context.py", line 239, in browse_snapshot_directory
    browse_context='directory') # noqa
    File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/web/browse/views/utils/snapshot_context.py", line 135, in _process_snapshot_request
    origin_url, timestamp, visit_id)
    ...
    • Jul 13 2019, 9:00 PM
    • 10 Lines
    Author: nahimilega
  • P470 swh-storage package list
    ttaching to swh-docker-dev_swh-storage_1
    swh-storage_1 | Using pip from /srv/softwareheritage/venv/bin/pip
    swh-storage_1 | Installed Python packages:
    swh-storage_1 | Package Version
    swh-storage_1 | ----------------------------- ---------
    ...
    • Jul 11 2019, 5:09 PM
    • 107 Lines
    Author: nahimilega
  • P469 pip list
    (swh) archit@work-pc:~/swh-environment/swh-loader-core/swh/loader$ pip list
    Package Version Location
    ----------------------------- -------------- -------------------------------------------------
    aiohttp 3.5.4
    alabaster 0.7.12
    ...
    • Jul 11 2019, 5:05 PM
    • 164 Lines
    Author: nahimilega
  • P468 error while running pypi loader
    DEBUG:urllib3.connectionpool:http://localhost:5002 "POST /origin/visit/get_latest HTTP/1.1" 400 74
    ERROR:swh.loader.pypi.PyPILoader:Loading failure, updating to `partial` status
    Traceback (most recent call last):
    File "/home/archit/swh-environment/swh-loader-core/swh/loader/core/loader.py", line 868, in load
    self.prepare(*args, **kwargs)
    ...
    • Jul 11 2019, 3:21 PM
    • 71 Lines
    Author: nahimilega
  • P467 (An Untitled Masterwork)
    FROM debian:stretch
    RUN apt-get update
    RUN apt-get install -y python3-pip tox git make
    RUN git clone https://github.com/t-b/sphinxcontrib-images.git
    ...
    • Jul 11 2019, 2:57 PM
    • 18 Lines
    Author: vlorentz
  • P466 swh-web logs for @Shisus using docker-compose swh-web logs
    Attaching to swh-docker-dev_swh-web_1
    swh-web_1 | Using pip from /srv/softwareheritage/venv/bin/pip
    swh-web_1 | Installed Python packages:
    swh-web_1 | Package Version
    swh-web_1 | ----------------------------- ---------
    ...
    • Jul 11 2019, 11:09 AM
    • 491 Lines
    Author: shisus
  • P465 sphinx error
    (swh) archit@work-pc:~/swh-environment$ pip uninstall sphinx
    (swh) archit@work-pc:~/swh-environment$ ./bin/update
    (swh) archit@work-pc:~/swh-environment$ pip install $(./bin/pip-swh-packages --with-testing)
    Stored in directory: /home/archit/.cache/pip/wheels/7b/6c/23/eb26369b77904c8963fae9e64338b0f0b948b4d59710760834
    ...
    • Jul 10 2019, 10:58 PM
    • 9 Lines
    Author: nahimilega
  • P464 error while installation
    (swh) ~/swh-environment$ pip install $(./bin/pip-swh-packages --with-testing)
    Successfully built PyYAML systemd-python psycopg2 vcversioner python-cephlibs retrying pyld fastimport dulwich SQLAlchemy python-hglib sqlitedict subvertpy htmlmin pypandoc sphinxcontrib-images wrapt yarl billiard snowballstemmer future
    ERROR: sphinxcontrib-images 0.8.0 has requirement sphinx<2,>=1.1.3, but you'll have sphinx 2.1.2 which is incompatible.
    ERROR: swh-docs 0.0.0.post156 has requirement sphinx<2,>=1.3, but you'll have sphinx 2.1.2 which is incompatible.
    ...
    • Jul 10 2019, 9:08 PM
    • 6 Lines
    Author: nahimilega
  • P463 tox test stack trace for swh-indexer
    Traceback (most recent call last):
    File "/home/twitu/Code/swh-environment/swh-indexer/.tox/py3/bin/pytest", line 10, in <module>
    sys.exit(main())
    File "/home/twitu/Code/swh-environment/swh-indexer/.tox/py3/lib/python3.5/site-packages/_pytest/config/__init__.py", line 58, in main
    config = _prepareconfig(args, plugins)
    ...
    • Jul 10 2019, 6:11 PM
    • 77 Lines
    Author: twitu
  • P461 swh-graph use cases
    This document lists use cases and benchmark scenarii for the Software Heritage
    graph service.
    Conventions
    ...
    • Jul 7 2019, 1:20 PM
    • 170 Lines
    • reStructuredText
    Author: zack
  • P460 bitbucket lister last entries (indexed by repo's creation date)
    ```
    swh-lister=> select * from bitbucket_repo order by indexable desc limit 10;
    name | full_name | html_url | origin_url | origin_type | last_seen | task_id | uid | indexable
    ------------------------------+----------------------------------------------+--------------------------------------------------------------------+------------------------------------------------------------------------+-------------+----------------------------+-----------+----------------------------------------+-------------------------------
    issues_crud_hg | bb-synthetic_2-bot/issues_crud_hg | https://bitbucket.org/bb-synthetic_2-bot/issues_crud_hg | https://bitbucket.org/bb-synthetic_2-bot/issues_crud_hg | hg | 2019-07-06 07:23:56.735599 | 169799503 | {bf9c4d33-712b-4da3-b420-a959f53db27f} | 2019-07-06 07:23:52.331212+00
    ...
    • Jul 6 2019, 9:31 AM
    • 17 Lines
    Author: ardumont
  • P459 ugly bitbucket lister workaround
    on worker09:
    ```
    @app.task(name=__name__ + '.IncrementalBitBucketLister')
    def incremental_bitbucket_lister(**lister_args):
    lister = new_lister(**lister_args)
    ...
    • Jul 4 2019, 4:53 PM
    • 13 Lines
    Author: ardumont
  • P458 journal-client apparently no longer works with configuration file
    swhstorage@getty:~$ swh indexer --config-file /etc/softwareheritage/indexer/journal_client.yml journal-client
    INFO:swh.core.config:Loading config file /etc/softwareheritage/indexer/journal_client.yml
    Usage: swh indexer journal-client [OPTIONS]
    Error: You must specify at least one kafka broker.
    • Jul 3 2019, 4:20 PM
    • 5 Lines
    Author: ardumont
  • P457 Authorities example for extrinsic metadata specs
    ========== ============== ================================== ====================================== ==========
    type name url format Example
    ========== ============== ================================= ====================================== ===========
    deposit hal https://hal.archives-ouvertes.fr/ codemeta-ish [1]
    deposit intel https://software.intel.com/ codemeta-ish [2]
    ...
    • Jul 3 2019, 2:24 PM
    • 17 Lines
    Author: moranegg
  • P456 swh-web error with test settings
    Done in 5.97s.
    python3 swh/web/manage.py migrate -v0 2>/dev/null
    python3 swh/web/manage.py createcachetable -v0 2>/dev/null
    bash -c "trap 'trap - SIGINT SIGTERM ERR EXIT; jobs -p | head -1 | xargs pkill -P' SIGINT SIGTERM ERR EXIT; python3 swh/web/manage.py runserver --nostatic --settings=swh.web.settings.tests & sleep 10 && yarn run cypress open"
    Loading failure, updating to `partial` status
    ...
    • Jul 3 2019, 12:23 PM
    • 88 Lines
    Author: kalpitk
  • P451 web app -> 500
    {"exception":"BadInputExc","reason":"Error when parsing identifier. Wrong format: There should be 4 mandatory values","traceback":"Traceback (most recent call last):\n File \"/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/web/common/utils.py\", line 235, in resolve_swh_persistent_id\n swh_id_parsed = parse_persistent_identifier(swh_id)\n File \"/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/model/identifiers.py\", line 747, in parse_persistent_identifier\n 'Wrong format: There should be 4 mandatory values')\nswh.model.exceptions.ValidationError: ['Wrong format: There should be 4 mandatory values']\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/web/api/apidoc.py\", line 307, in documented_view\n response = f(request, **kwargs)\n File \"/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/web/api/views/identifiers.py\", line 57, in...
    • Jun 26 2019, 3:33 PM
    • 1 Line
    Author: douardda
  • P450 graphql result
    {
    "data": {
    "origins": [
    {
    "id": 53607151,
    ...
    • Jun 25 2019, 4:42 PM
    • 142 Lines
    • JSON
    Author: vlorentz
  • P449 graphql query
    query {
    origins(urls: ["https://github.com/SoftwareHeritage/swh-model"]) {
    id
    type
    url
    ...
    • Jun 25 2019, 4:42 PM
    • 24 Lines
    • Plain Text
    Author: vlorentz
  • P448 Pytest error stack trace
    swh-environment/swh-web/.tox/py3/lib/python3.5/site-packages/swh/web/tests/api/views/test_origin.py(330)test_api_origin_intrinsic_metadata_get()
    -> def test_api_origin_intrinsic_metadata_get(self, origin):
    swh-environment/swh-web/.tox/py3/lib/python3.5/site-packages/hypothesis/core.py(1023)wrapped_test()
    -> raise the_error_hypothesis_found
    • Jun 23 2019, 2:55 PM
    • 4 Lines
    • Bash Scripting
    Author: twitu
  • P447 Run a single test file
    /swh-environment/swh-web/.tox/py3/bin/pytest --hypothesis-profile=swh-web-fast --cov .tox/py3/lib/python3.5/site-packages/swh/web --cov-branch .tox/py3/lib/python3.5/site-packages/swh/web/tests/api/views/test_origin.py
    • Jun 22 2019, 12:26 PM
    • 1 Line
    • Bash Scripting
    Author: twitu
  • P446 Multiple context managers
    with (
    patch('swh.web.common.service.idx_storage') as mock_idx_storage,
    patch('swh.web.common.service.storage' as mock_storage
    ):
    • Jun 22 2019, 9:50 AM
    • 4 Lines
    • Python
    Author: twitu
  • P445 get_links
    LinkNode([('', LinkNode()),
    ('1',
    LinkNode([('content',
    LinkNode([('ctags', LinkNode()),
    ('filetype', LinkNode()),
    ...
    • Jun 20 2019, 1:57 PM
    • 49 Lines
    Author: anlambert
  • P444 swh-web schema
    info:
    description: ''
    title: Bookings API
    version: ''
    openapi: 3.0.0
    ...
    • Jun 20 2019, 1:38 PM
    • 639 Lines
    • YAML
    Author: vlorentz
  • P443 swh-web generate schema
    diff --git a/swh/web/api/urls.py b/swh/web/api/urls.py
    index afbcaf44..4267c73d 100644
    --- a/swh/web/api/urls.py
    +++ b/swh/web/api/urls.py
    @@ -15,6 +15,25 @@ import swh.web.api.views.snapshot # noqa
    ...
    • Jun 20 2019, 1:37 PM
    • 30 Lines
    • Diff
    Author: vlorentz
  • P442 code
    from bs4 import BeautifulSoup
    from collections import defaultdict
    import requests
    base_url = "https://git.savannah.gnu.org/cgit/"
    ...
    • Jun 19 2019, 11:27 PM
    • 81 Lines
    Author: nahimilega
  • P441 comments in pyarc diff
    (swh) ddouard@desktop6:~/src/swh-environment/swh-journal$ pyarc diff -c
    Accepted D1608
    Repo: swh-journal (DJNL)
    Author: douardda
    Created: 3 hours ago
    ...
    • Jun 19 2019, 2:38 PM
    • 26 Lines
    Author: douardda
  • P440 Migrate lister db
    alter table gnu_repo drop column origin_id;
    alter table gnu_repo drop column description;
    alter table phabricator_repo drop column origin_id;
    alter table phabricator_repo drop column description;
    ...
    • Jun 19 2019, 10:34 AM
    • 24 Lines
    • SQL
    Author: ardumont
  • P439 Error while git push
    swh-lister git:(master) git push
    Counting objects: 5, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (5/5), done.
    Writing objects: 100% (5/5), 629 bytes | 0 bytes/s, done.
    ...
    • Jun 19 2019, 7:59 AM
    • 17 Lines
    Author: NancyChauhan
  • P438 Grunt: Incremental phabricator lister fails with the following
    Jun 18 11:51:45 worker01 python3[29020]: [2019-06-18 11:51:45,602: INFO/MainProcess] Received task: swh.lister.phabricator.tasks.IncrementalPhabricatorLister[6511673e-9881-494d-bc32-e2d5631809df]
    Jun 18 11:51:45 worker01 python3[29026]: [2019-06-18 11:51:45,820: ERROR/ForkPoolWorker-3] Task swh.lister.phabricator.tasks.IncrementalPhabricatorLister[6511673e-9881-494d-bc32-e2d5631809df] raised unexpected: ValueError('Problem during information fetch: ERR-CONDUIT-CORE',)
    Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/celery/app/trace.py", line 382, in trace_task
    R = retval = fun(*args, **kwargs)
    ...
    • Jun 18 2019, 1:57 PM
    • 23 Lines
    Author: ardumont
  • P437 wikimedia phabricator lister: ChunkedEncodingError
    Jun 18 11:24:42 worker01 python3[12919]: [2019-06-18 11:24:42,238: INFO/MainProcess] Received task: swh.lister.phabricator.tasks.FullPhabricatorLister[551ff6d5-473b-4e6f-8e70-9d8041ab5fce]
    Jun 18 11:27:32 worker01 python3[12934]: [2019-06-18 11:27:32,544: ERROR/ForkPoolWorker-5] Task swh.lister.phabricator.tasks.FullPhabricatorLister[551ff6d5-473b-4e6f-8e70-9d8041ab5fce] raised unexpected: ChunkedEncodingError(ProtocolError('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read)),)
    Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/urllib3/response.py", line 601, in _update_chunk_length
    self.chunk_left = int(line, 16)
    ...
    • Jun 18 2019, 1:34 PM
    • 69 Lines
    Author: ardumont
  • P436 blender phabricator lister failure
    Jun 18 10:25:08 worker06 python3[14188]: [2019-06-18 10:25:08,661: WARNING/ForkPoolWorker-5] Failed to fetch https://developers.blender.org/api/diffusion.repository.search?order=oldest&attachments[uris]=1&after=&order=oldest&limit=1: HTTPSConnectionPool(host='developers.blender.org', port=443): Max retries exceeded with url: /api/diffusion.repository.search?order=oldest&attachment
    s[uris]=1&after=&order=oldest&limit=1&api.token=api-e45yly42hzfvmxi62nssekngtl5w (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f8109bbbbe0>: Failed to establish a new connection: [Errno -2] Name or service not known',))
    Jun 18 10:25:08 worker06 python3[14188]: [2019-06-18 10:25:08,661: WARNING/ForkPoolWorker-5] connection error on &order=oldest&limit=1: sleep for 10 seconds
    Jun 18 10:25:18 worker06 python3[14188]: [2019-06-18 10:25:18,671: WARNING/ForkPoolWorker-5] Failed to fetch https://developers.blender.org/api/diffusion.repository.search?order=oldest&attachments[uris]=1&after=&order=oldest&limit=1: HTTPSConnectionPool(host='developers.blender.org', port=443): Max retries exceeded with url: /api/diffusion.repository.search?order=oldest&attachment
    s[uris]=1&after=&order=oldest&limit=1&api.token=api-e45yly42hzfvmxi62nssekngtl5w (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f8109bbba20>: Failed to establish a new connection: [Errno -2] Name or service not known',))
    ...
    • Jun 18 2019, 1:11 PM
    • 31 Lines
    Author: ardumont
First
Prev
Next
About · Developer information · Legal