Page MenuHomeSoftware Heritage
Create Paste
Paste Active Pastes

Active Pastes
Use Results
Edit Query

  • P355 failing master build on repositories... due to latest commit on swh-jenkins-jobs?
    Started by timer
    Running in Durability level: MAX_SURVIVABILITY
    Loading library swh@master
    Attempting to resolve master from remote references...
    > git --version # timeout=10
    ...
    • Feb 7 2019, 3:38 PM
    • 138 Lines
    Author: ardumont
  • P354 metadata indexing stats on 2019-02-04
    gemspec | codemeta | pkginfo | npm | maven | total | total_nonempty
    ---------+----------+---------+---------+--------+----------+----------------
    143688 | 139 | 3525 | 1313459 | 447697 | 17730303 | 2033490
    • Feb 4 2019, 10:55 AM
    • 3 Lines
    Author: vlorentz
  • P353 Masterwork From Distant Lands
    #!/bin/bash
    # wrapper to run GitHub Licensee license detection tool form a git clone of its
    # repo, setting up the appropriate Ruby load path
    ...
    • Feb 2 2019, 1:35 PM
    • 8 Lines
    Author: zack
  • P352 storage0.euwest.azure crashes with the following error from time to time
    Jan 31 13:04:36 storage0 python3 [2434334]: 2019-01-31 13:04:36 [2434334] [ERROR] canceling statement due to statement timeout
    Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request ()
    File "/usr/lib/python3/dist-packages/flask/app.py", line 1598, in dispatch_request
    ...
    • Jan 31 2019, 2:11 PM
    • 31 Lines
    Author: ardumont
  • P351 swh-web deployment issue
    *** swh-deploy: deploying on moma.internal.softwareheritage.org...
    Info: Using configured environment 'production'
    Info: Retrieving pluginfacts
    Info: Retrieving plugin
    Info: Loading facts
    ...
    • Jan 25 2019, 2:49 PM
    • 51 Lines
    Author: anlambert
  • P350 scheduler api errors
    swh-scheduler-api_1 | [INFO] werkzeug -- 172.20.0.15 - - [24/Jan/2019 10:48:57] "POST /create_tasks HTTP/1.1" 200 -
    swh-scheduler-api_1 | [INFO] werkzeug -- 172.20.0.15 - - [24/Jan/2019 10:48:57] "POST /create_tasks HTTP/1.1" 200 -
    swh-scheduler-api_1 | [INFO] werkzeug -- 172.20.0.15 - - [24/Jan/2019 10:48:57] "POST /create_tasks HTTP/1.1" 200 -
    swh-scheduler-api_1 | [ERROR] root -- relation "tmp_task" already exists
    ...
    • Jan 24 2019, 11:56 AM
    • 47 Lines
    Author: douardda
  • P349 (An Untitled Masterwork)
    $ pip3 show pyld | grep Version
    Version: 1.0.3
    $ python3
    Python 3.5.3 (default, Sep 27 2018, 17:25:39)
    [GCC 6.3.0 20170516] on linux
    ...
    • Jan 16 2019, 5:57 PM
    • 29 Lines
    Author: vlorentz
  • P348 metadata search
    >>> import pprint
    >>> import swh.indexer.storage.api.client
    >>> s = swh.indexer.storage.api.client.RemoteStorage(url='http://uffizi.internal.softwareheritage.org:5007/')
    >>> pprint.pprint(s.origin_intrinsic_metadata_search_fulltext(['James']))
    [{'from_revision': b'\xd4bM\xa6\x9eH\x06\x15\x0c\x1ap\xbc\x84~\x11\x17'
    ...
    • Jan 14 2019, 1:40 PM
    • 85 Lines
    • Python
    Author: vlorentz
  • P347 (An Untitled Masterwork)
    import sys
    import time
    #import kafka
    ...
    • Jan 10 2019, 1:38 PM
    • 32 Lines
    Author: vlorentz
  • P346 (An Untitled Masterwork)
    # Copyright (C) 2018 The Software Heritage developers
    # See the AUTHORS file at the top-level directory of this distribution
    # License: GNU General Public License version 3, or any later version
    # See top-level LICENSE file for more information
    ...
    • Jan 10 2019, 11:53 AM
    • 50 Lines
    • Python
    Author: vlorentz
  • P345 (An Untitled Masterwork)
    -- DONE
    CREATE TABLE ctas_dataset_dir_to_rev
    WITH (format = 'TEXTFILE', external_location =
    's3://softwareheritage/edges_dataset/dir_to_rev/', field_delimiter = ' ')
    AS SELECT to_hex(directory.id) as source, to_hex(target) as dest
    ...
    • Jan 7 2019, 7:15 PM
    • 72 Lines
    • SQL
    Author: seirl
  • P342 error when pip install --with-testing
    swh-environment $ pip install $( ./bin/pip-swh-packages --with-testing )
    swh-storage[schemata,listener][testing] should either be a path to a local project or a VCS url beginning with svn+, git+, hg+, or bzr+
    • Dec 20 2018, 3:31 PM
    • 2 Lines
    Author: ardumont
  • P341 issues running tests with repositories needing postgresql as dependencies (pytest, tox)
    ```
    $ pifpaf run postgresql -- pytest
    WARNING [pifpaf.drivers] `psutil.Popen(pid=23360, status='terminated')` is already gone, sending SIGKILL to its process group
    ERROR [pifpaf] sequence item 0: expected str instance, bytes found
    ```
    ...
    • Dec 20 2018, 3:24 PM
    • 45 Lines
    Author: ardumont
  • P340 swh-scheduler-api_1 ERROR:root:fe_sendauth: no password supplied
    swh-scheduler-api_1 | ERROR:root:fe_sendauth: no password supplied
    swh-scheduler-api_1 | Traceback (most recent call last):
    swh-scheduler-api_1 | File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
    swh-scheduler-api_1 | rv = self.dispatch_request()
    swh-scheduler-api_1 | File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1799, in dispatch_request
    ...
    • Dec 17 2018, 3:22 PM
    • 19 Lines
    Author: vlorentz
  • P337 sphinx image path
    ✘ dev@desktop5  ~/swh-environment/swh-docs   master  git pull
    Already up-to-date.
    dev@desktop5  ~/swh-environment/swh-docs   master  tox -r -e sphinx-dev
    GLOB sdist-make: /home/dev/swh-environment/swh-docs/setup.py
    sphinx-dev recreate: /home/dev/swh-environment/swh-docs/.tox/sphinx-dev
    ...
    • Nov 29 2018, 6:29 PM
    • 46 Lines
    Author: vlorentz
  • P336 python preferences
    When i have something like:
    ```
    if a:
    r = do_something(a)
    ...
    • Nov 22 2018, 11:46 AM
    • 21 Lines
    Author: ardumont
  • P335 indexer: debian package build failure due to hypothesis 3.6 (debian stable) too old
    Hypothesis issue with old version: https://github.com/HypothesisWorks/hypothesis/issues/290
    build output:
    ```
    ============================= test session starts ==============================
    ...
    • Nov 19 2018, 11:38 AM
    • 98 Lines
    Author: ardumont
  • P334 hypothesis storage mimetype
    pytest swh/indexer/tests/storage/test_storage.py -x
    ========================================================================================================= test session starts =========================================================================================================
    platform linux -- Python 3.5.3, pytest-3.9.3, py-1.7.0, pluggy-0.8.0
    hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/dev/swh-environment/swh-indexer/.hypothesis/examples')
    rootdir: /home/dev/swh-environment/swh-indexer, inifile: pytest.ini
    ...
    • Nov 15 2018, 10:52 AM
    • 99 Lines
    Author: vlorentz
  • P333 storage API?
    diff <(grep 'def ' swh/storage/in_memory.py | grep -v 'def _' | sed -e 's/(.*//' | sort) <(grep 'def ' swh/storage/storage.py | grep -v 'def _' | sed -e 's/(.*//' | sort)
    0a1
    > def add_to_objstorage
    3a5
    > def content_get
    ...
    • Nov 15 2018, 10:07 AM
    • 21 Lines
    • Diff
    Author: douardda
  • P332 Metadata files count 22.5.2017
    | context | file_name | counted | percentage | percentage | percentage on 3,424,000,000 files |
    |-------------------------------|-------------------|----------------------|------------|------------|-----------------------------------|
    | CodeMeta | CODE | 320 | 0.00% | | 8.85E-06 |
    | haskell | .cabal | 676053 | 1.27% | 0.01% | 0.01870298068 |
    | java- Maven | pom.xml | 15509125 | 29.03% | 0.43% | 0.4290593566 |
    ...
    • Nov 12 2018, 5:11 PM
    • 33 Lines
    Author: moranegg
  • P331 (An Untitled Masterwork)
    def _naive_sig(param_names):
    return inspect.Signature([
    inspect.Parameter(name, inspect.Parameter.POSITIONAL_OR_KEYWORD)
    for name in param_names])
    ...
    • Nov 9 2018, 1:07 PM
    • 82 Lines
    • Python
    Author: vlorentz
  • P330 example docstrings
    def content_mimetype_missing(self, mimetypes, db=None, cur=None):
    """Generates mimetypes missing from storage.
    Args:
    mimetypes (iterable): iterable of dict with keys:
    ...
    • Nov 8 2018, 12:03 PM
    • 13 Lines
    • Python
    Author: douardda
  • P329 remove testrepo.zip
    commit edebe6a4a42bae99a1819898b14bb0951cfe6b8b
    Author: Valentin Lorentz <vlorentz@softwareheritage.org>
    Date: Mon Nov 5 14:54:06 2018 +0100
    Remove testrepo.zip.
    ...
    • Nov 5 2018, 2:52 PM
    • 64 Lines
    • Diff
    Author: vlorentz
  • P328 swh-indexer test fails with tox -r
    GLOB sdist-make: /home/morane/Documents/code/swh-environment/swh-indexer/setup.py
    flake8 recreate: /home/morane/Documents/code/swh-environment/swh-indexer/.tox/flake8
    flake8 installdeps: flake8
    flake8 installed: flake8==3.6.0,mccabe==0.6.1,pkg-resources==0.0.0,pycodestyle==2.4.0,pyflakes==2.0.0
    flake8 runtests: PYTHONHASHSEED='3997505493'
    ...
    • Oct 30 2018, 4:31 PM
    • 3,664 Lines
    Author: moranegg
  • P327 (An Untitled Masterwork)
    dev@desktop5  ~/swh-environment/swh-indexer   master  git pull
    remote: Counting objects: 14, done.
    remote: Compressing objects: 100% (14/14), done.
    remote: Total 14 (delta 10), reused 0 (delta 0)
    Unpacking objects: 100% (14/14), done.
    ...
    • Oct 29 2018, 10:22 AM
    • 14 Lines
    Author: vlorentz
  • P326 Deploy the latest indexers m.o (new v0.0.55)
    If it's urgent to redeploy indexers, here is what i foresee in the
    current state of affairs,
    - After review and acceptance, merge the diffs:
    ...
    • Oct 27 2018, 11:16 AM
    • 58 Lines
    Author: ardumont
  • P325 EncodeError(TypeError("b'xxx' is not JSON serializable")) in debian stable chroot
    celery.worker.strategy: INFO: Received task: swh.indexer.tests.test_origin_metadata.test_revision_metadata_task[785b1155-89f2-4aec-ac50-c2d3eb34b4d3]
    celery.app.trace: ERROR: Task swh.indexer.tests.test_origin_metadata.test_revision_metadata_task[785b1155-89f2-4aec-ac50-c2d3eb34b4d3] raised unexpected: EncodeError(TypeError("b'8dbb6aeb036e7fd80664eb8bfd1507881af1ba9f' is not JSON serializable",),)
    Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/kombu/serialization.py", line 50, in _reraise_errors
    yield
    ...
    • Oct 25 2018, 3:08 PM
    • 81 Lines
    Author: ardumont
  • P324 ghost error with Meatadata Indexer
    ======================================================================
    FAIL: test_pipeline (swh.indexer.tests.test_origin_metadata.TestOriginMetadata)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
    ...
    • Oct 25 2018, 2:47 PM
    • 26 Lines
    Author: moranegg
  • P323 Codemeta test
    def test_compute_metadata_codemeta(self):
    """
    test that a codemeta file is not altered with translation
    """
    ...
    • Oct 25 2018, 12:06 PM
    • 50 Lines
    Author: moranegg
  • P322 tox issues on vault
    pre-requisite:
    ```
    arc patch D582
    ```
    ...
    • Oct 24 2018, 11:12 AM
    • 1,748 Lines
    Author: ardumont
  • P321 Error with D558
    KeyboardInterrupt
    Exception ignored in: <bound method AsyncResult.__del__ of <AsyncResult: 8731ccc1-5068-41ff-9b83-e0e2ef930843>>
    Traceback (most recent call last):
    File "/home/morane/.local/lib/python3.5/site-packages/celery/result.py", line 385, in __del__
    self.backend.remove_pending_result(self)
    ...
    • Oct 23 2018, 5:01 PM
    • 60 Lines
    Author: moranegg
  • P320 loader errors per loader type: ~/.config/swh/kibana/group-by.yml
    ---
    errors:
    hg:
    - "Failed to uncompress archive"
    - "OSError: [Errno 12] Cannot allocate memory"
    ...
    • Oct 18 2018, 11:13 AM
    • 134 Lines
    Author: ardumont
  • P319 reschedule.pypi.csv - first origins with missing author resulted in T1246 issue (fixed since)
    origin-update-pypi;oneshot;["manhattan_seo", "https://pypi.org/project/manhattan_seo/"];{"project_metadata_url": "https://pypi.org/pypi/manhattan_seo/json"}
    origin-update-pypi;oneshot;["hypermax", "https://pypi.org/project/hypermax/"];{"project_metadata_url": "https://pypi.org/pypi/hypermax/json"}
    origin-update-pypi;oneshot;["Flask-Security-Bundle", "https://pypi.org/project/Flask-Security-Bundle/"];{"project_metadata_url": "https://pypi.org/pypi/Flask-Security-Bundle/json"}
    origin-update-pypi;oneshot;["collective.dms.mailcontent", "https://pypi.org/project/collective.dms.mailcontent/"];{"project_metadata_url": "https://pypi.org/pypi/collective.dms.mailcontent/json"}
    origin-update-pypi;oneshot;["certbot-dns-cloudxns", "https://pypi.org/project/certbot-dns-cloudxns/"];{"project_metadata_url": "https://pypi.org/pypi/certbot-dns-cloudxns/json"}
    ...
    • Oct 17 2018, 4:21 PM
    • 1,411 Lines
    Author: ardumont
  • P318 (An Untitled Masterwork)
    return self.RevisionMetadataTask().apply_async(
    kwargs={
    'ids': [res['revision_id'] for res in results],
    'policy_update': 'update-dups',
    },
    ...
    • Oct 16 2018, 2:25 PM
    • 9 Lines
    • Python
    Author: vlorentz
  • P317 pypi errors group by end of exception message (~50 characters from the end).
    jq . pypi.group-output.txt | grep -v 'Reason: 404'
    {
    "googlecode": {
    "total": 3933,
    "errors": {
    ...
    • Oct 16 2018, 1:50 PM
    • 178 Lines
    Author: ardumont
  • P316 fetch logs and compute group by end of exception message to have a sensible output of error occurrences
    ```
    export PYTHONPATH=$SWH_ENVIRONMENT_HOME/snippets/ardumont:$PYTHONPATH
    python3 -m kibana_fetch_logs > output.txt
    cat output.txt | python3 -m group_by_exception --loader-type hg > output-group-by.txt
    ```
    ...
    • Oct 15 2018, 4:22 PM
    • 7 Lines
    Author: ardumont
  • P315 querying mercurial loader errors
    curl -XPOST 'http://esnode3.internal.softwareheritage.org:9200/swh_workers-2018.10.11,swh_workers-2018.10.12,swh_workers-2018.10.13,swh_workers-2018.10.14/_search' -d '{
    "from": 10,
    "_source": [
    "message",
    ...
    • Oct 15 2018, 2:33 PM
    • 36 Lines
    Author: ardumont
  • P314 elasticsearch: indices: swh_workers-2018.09.30 mapping
    {
    "swh_workers-2018.09.30": {
    "mappings": {
    "doc": {
    "properties": {
    ...
    • Oct 15 2018, 1:36 PM
    • 617 Lines
    Author: ardumont
  • P313 elasticsearch: indices: template template_swh_workers
    {
    "order": 0,
    "index_patterns": [
    "swh_workers-*"
    ],
    ...
    • Oct 15 2018, 1:33 PM
    • 16 Lines
    Author: ardumont
  • P312 D505/507 test errors
     tony  (e) .venv   arcpatch-D505_1  …  swh  swh-environment  swh-indexer  2  make test
    python3 -m nose -sv --with-doctest .
    Failure: TypeError (metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases) ... ERROR
    Failure: TypeError (metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases) ... ERROR
    Failure: TypeError (metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases) ... ERROR
    ...
    • Oct 11 2018, 10:46 AM
    • 465 Lines
    Author: ardumont
  • P311 loader-pypi errors on first round
    http://kibana0.internal.softwareheritage.org:5601/app/kibana#/dashboard/32632370-c0bd-11e8-8222-07f3ec376cd5?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:%272018-08-31T22:00:00.000Z%27,mode:absolute,to:%272018-10-05T21:59:59.999Z%27))&_a=(description:%27This%20is%20a%20general%20dashboard%20listing%20the%20full%20logs%20of%20the%20swh-workers%27,filters:!((%27$state%27:(store:appState),meta:(alias:!n,disabled:!f,index:%2720de3150-c0bb-11e8-8222-07f3ec376cd5%27,key:systemd_unit,negate:!f,params:(query:%27swh-worker@swh_loader_pypi.service%27,type:phrase),type:phrase,value:%27swh-worker@swh_loader_pypi.service%27),query:(match:(systemd_unit:(query:%27swh-worker@swh_loader_pypi.service%27,type:phrase)))),(%27$state%27:(store:appState),meta:(alias:!n,disabled:!f,index:%2720de3150-c0bb-11e8-8222-07f3ec376cd5%27,key:priority,negate:!f,params:(query:%273%27,type:phrase),type:phrase,value:%273%27),query:(match:(priority:(query:%273%27,type:phrase)))),(%27$state%27:(store:appState),meta:(alias:!n...
    • Oct 5 2018, 6:29 PM
    • 1 Line
    Author: ardumont
  • P310 (An Untitled Masterwork)
    dev@desktop5  ~/swh-environment/swh-indexer   origin-head-indexer  grep content_metadata_add **/*.{py,sql}
    swh/indexer/metadata.py: self.idx_storage.content_metadata_add(
    swh/indexer/storage/__init__.py: def content_metadata_add(self, metadata, conflict_update=False, db=None,
    swh/indexer/storage/__init__.py: db.content_metadata_add_from_temp(conflict_update, cur)
    swh/indexer/storage/api/client.py: def content_metadata_add(self, metadata, conflict_update=False):
    ...
    • Oct 5 2018, 4:22 PM
    • 20 Lines
    Author: vlorentz
  • P309 OriginHead click error
    Traceback (most recent call last):
    File "origin_head.py", line 121, in <module>
    main()
    File "/usr/lib/python3/dist-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
    ...
    • Oct 5 2018, 2:51 PM
    • 26 Lines
    Author: moranegg
  • P308 indexer: make test ko
    ```
    test_revision_metadata_indexer (swh.indexer.tests.test_metadata.Metadata) ... ERROR
    ...
    ======================================================================
    ...
    • Oct 4 2018, 4:19 PM
    • 28 Lines
    Author: ardumont
  • P307 swhpass alias
    # swhpass and completion mechanism
    SWH_PASSWORD_STORE_DIR=${HOME}/work/inria/repo/swh/credentials/
    function swhpass() {
    PASSWORD_STORE_DIR=$SWH_PASSWORD_STORE_DIR pass $@
    }
    ...
    • Oct 4 2018, 3:00 PM
    • 12 Lines
    Author: ardumont
  • P305 (An Untitled Masterwork)
    diff --git a/setup.py b/setup.py
    index 42e154f..93d3192 100644
    --- a/setup.py
    +++ b/setup.py
    @@ -34,6 +34,7 @@ setup(
    ...
    • Oct 1 2018, 2:55 PM
    • 12 Lines
    • Diff
    Author: seirl
  • P304 mixed crlf origin in error
    http://acsccg.googlecode.com/svn/ /srv/storage/space/mirrors/code.google.com/sources/v2/code.google.com/a/acsccg/acsccg-repo.svndump.gz
    http://anarchintosh-projects.googlecode.com/svn/ /srv/storage/space/mirrors/code.google.com/sources/v2/code.google.com/a/anarchintosh-projects/anarchintosh-projects-repo.svndump.gz
    http://bastian.googlecode.com/svn/ /srv/storage/space/mirrors/code.google.com/sources/v2/code.google.com/b/bastian/bastian-repo.svndump.gz
    http://calculapdf.googlecode.com/svn/ /srv/storage/space/mirrors/code.google.com/sources/v2/code.google.com/c/calculapdf/calculapdf-repo.svndump.gz
    http://dagamers.googlecode.com/svn/ /srv/storage/space/mirrors/code.google.com/sources/v2/code.google.com/d/dagamers/dagamers-repo.svndump.gz
    ...
    • Sep 28 2018, 4:04 PM
    • 23 Lines
    Author: ardumont
  • P303 svn loader: all time origins in error with UnicodeDecodeError - 81 (with duplicates)
    http://9i00.googlecode.com/svn/ f Traceback (most recent call last):\n File "/usr/lib/python3/dist-packages/swh/loader/core/loader.py", line 862, in load\n self.store_data()\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 313, in store_data\n start_from_scratch=self.start_from_scratch)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 503, in process_repository\n svnrepo, revision_start, revision_end, revision_parents)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 240, in process_swh_revisions\n raise e\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 219, in process_swh_revisions\n self.config['revision_packet_size']):\n File "/usr/lib/python3/dist-packages/swh/core/utils.py", line 40, in grouper\n for _data in itertools.zip_longest(*args, fillvalue=None):\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 163, in process_svn_revisions\n for rev, nextrev, commit, new_objects, root_directory in gen_revs:\n File "/usr/lib/python3/dist-packages/swh/loader/svn/svn.py", line 267, in swh_hash_data_per_revision\n objects = self.swhreplay.compute_hashes(rev)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/ra.py", line 374, in compute_hashes\n self.replay(rev)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/ra.py", line 359, in replay\n self.conn.replay(rev, rev+1, self.editor)\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 0: invalid start byte\n
    http://9i00.googlecode.com/svn/ f Traceback (most recent call last):\n File "/usr/lib/python3/dist-packages/swh/loader/core/loader.py", line 742, in load\n self.store_data()\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 308, in store_data\n self.last_known_swh_revision)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 496, in process_repository\n svnrepo, revision_start, revision_end, revision_parents)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 238, in process_swh_revisions\n raise e\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 217, in process_swh_revisions\n self.config['revision_packet_size']):\n File "/usr/lib/python3/dist-packages/swh/core/utils.py", line 40, in grouper\n for _data in itertools.zip_longest(*args, fillvalue=None):\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 161, in process_svn_revisions\n for rev, nextrev, commit, new_objects, root_directory in gen_revs:\n File "/usr/lib/python3/dist-packages/swh/loader/svn/svn.py", line 266, in swh_hash_data_per_revision\n objects = self.swhreplay.compute_hashes(rev)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/ra.py", line 333, in compute_hashes\n self.replay(rev)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/ra.py", line 318, in replay\n self.conn.replay(rev, rev+1, self.editor)\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 0: invalid start byte\n
    http://9i00.googlecode.com/svn/ f Traceback (most recent call last):\n File "/usr/lib/python3/dist-packages/swh/loader/core/loader.py", line 742, in load\n self.store_data()\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 308, in store_data\n self.last_known_swh_revision)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 496, in process_repository\n svnrepo, revision_start, revision_end, revision_parents)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 238, in process_swh_revisions\n raise e\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 217, in process_swh_revisions\n self.config['revision_packet_size']):\n File "/usr/lib/python3/dist-packages/swh/core/utils.py", line 40, in grouper\n for _data in itertools.zip_longest(*args, fillvalue=None):\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 161, in process_svn_revisions\n for rev, nextrev, commit, new_objects, root_directory in gen_revs:\n File "/usr/lib/python3/dist-packages/swh/loader/svn/svn.py", line 266, in swh_hash_data_per_revision\n objects = self.swhreplay.compute_hashes(rev)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/ra.py", line 333, in compute_hashes\n self.replay(rev)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/ra.py", line 318, in replay\n self.conn.replay(rev, rev+1, self.editor)\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 0: invalid start byte\n
    http://9i00.googlecode.com/svn/ f Traceback (most recent call last):\n File "/usr/lib/python3/dist-packages/swh/loader/core/loader.py", line 732, in load\n self.store_data()\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 308, in store_data\n self.last_known_swh_revision)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 496, in process_repository\n svnrepo, revision_start, revision_end, revision_parents)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 238, in process_swh_revisions\n raise e\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 217, in process_swh_revisions\n self.config['revision_packet_size']):\n File "/usr/lib/python3/dist-packages/swh/core/utils.py", line 40, in grouper\n for _data in itertools.zip_longest(*args, fillvalue=None):\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 161, in process_svn_revisions\n for rev, nextrev, commit, new_objects, root_directory in gen_revs:\n File "/usr/lib/python3/dist-packages/swh/loader/svn/svn.py", line 266, in swh_hash_data_per_revision\n objects = self.swhreplay.compute_hashes(rev)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/ra.py", line 336, in compute_hashes\n self.replay(rev)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/ra.py", line 321, in replay\n self.conn.replay(rev, rev+1, self.editor)\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 0: invalid start byte\n
    http://9i00.googlecode.com/svn/ f Traceback (most recent call last):\n File "/usr/lib/python3/dist-packages/swh/loader/core/loader.py", line 628, in load\n """Detailed visit status.\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 300, in store_data\n Note:\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 489, in process_repository\n self.log.info(msg)\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 229, in process_swh_revisions\n _id = known_swh_rev.get('id')\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 213, in process_swh_revisions\n revs = []\n File "/usr/lib/python3/dist-packages/swh/core/utils.py", line 40, in grouper\n for _data in itertools.zip_longest(*args, fillvalue=None):\n File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 155, in process_svn_revisions\n gen_revs = svnrepo.swh_hash_data_per_revision(\n File "/usr/lib/python3/dist-packages/swh/loader/svn/svn.py", line 267, in swh_hash_data_per_revision\n File "/usr/lib/python3/dist-packages/swh/loader/svn/ra.py", line 453, in compute_hashes\n File "/usr/lib/python3/dist-packages/swh/loader/svn/ra.py", line 411, in replay\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 0: invalid start byte\n
    ...
    • Sep 27 2018, 3:47 PM
    • 81 Lines
    Author: ardumont
  • P302 Problem during venv install without libsystemd-dev
    $ python3 -m venv .venv
    $ source .venv/bin/activate
    $ pip install $( bin/pip-swh-packages )
    Obtaining file:///home/tony/work/inria/repo/swh/swh-environment/swh-core
    Obtaining file:///home/tony/work/inria/repo/swh/swh-environment/swh-model
    ...
    • Sep 25 2018, 5:35 PM
    • 68 Lines
    Author: ardumont
  • P301 (An Untitled Masterwork)
    root@scratch01-euwest ~ # du -sh /srv/hdd/swh-parquet/origin_visit/*
    128M /srv/hdd/swh-parquet/origin_visit/019ed7acd0d64f309dcfb3a977f11480.parquet
    113M /srv/hdd/swh-parquet/origin_visit/05160c9af2bf41379f605259a3e1cb24.parquet
    122M /srv/hdd/swh-parquet/origin_visit/07e9ce57dd544eebb539c376f7c81aaa.parquet
    136M /srv/hdd/swh-parquet/origin_visit/2ccd24604c194508909a3ae7f431d731.parquet
    ...
    • Sep 25 2018, 1:33 PM
    • 36 Lines
    Author: seirl
  • P300 (An Untitled Masterwork)
    Table origin_visit...
    0%|▏ | 500000/230786000 [00:10<1:22:12, 46691.30it/s]
    Table origin...
    1%|▍ | 500000/85109300 [00:04<13:35, 103756.07it/s]
    Table occurrence_history...
    ...
    • Sep 20 2018, 2:32 PM
    • 34 Lines
    Author: seirl
  • P299 (An Untitled Masterwork)
    Sep 12 14:20:04 worker12 python3[25846]: [2018-09-12 14:20:04,287: ERROR/MainProcess] Task swh.vault.cooking_tasks.SWHCookingTask[a129a228-79b1-47e1-8c37-70e913c1f544] raised unexpected: ConnectionResetError(104, 'Connection reset by peer')
    Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
    File "/usr/lib/python3/dist-packages/celery/app/trace.py", line 438, in __protected_call__
    ...
    • Sep 19 2018, 1:59 PM
    • 38 Lines
    Author: seirl
  • P296 count total size of indexes of the db
    begin;
    create or replace function swh_count_total_indexes_size()
    returns bigint
    language plpgsql
    ...
    • Sep 6 2018, 2:43 PM
    • 27 Lines
    Author: ardumont
  • P293 logstash0: puppet agent --test strangeness
    logstash0: Error: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold --force-yes install logstash=6.3.2' returned 100: Reading package lists...
    logstash0: Building dependency tree...
    logstash0: Reading state information...
    logstash0: E: Version '6.3.2' for 'logstash' was not found
    logstash0: Error: /Stage[main]/Profile::Logstash/Package[logstash]/ensure: change from 1:6.3.2-1 to 6.3.2 failed: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--$
    ...
    • Aug 31 2018, 10:10 AM
    • 31 Lines
    Author: ardumont
  • P292 estimation of gitlab ingestion remaining days
    #+BEGIN_SRC emacs-lisp
    (defun swh-percent (current total)
    (* 100 (/ current (* total 1.0))))
    ...
    • Aug 24 2018, 11:49 AM
    • 25 Lines
    Author: ardumont
  • P291 empty pypi project release (402 out of 6077, 6.62%)
    alpha
    Asuka
    Automiko
    appReviewToSlack
    aio2gis
    ...
    • Aug 22 2018, 4:03 PM
    • 402 Lines
    Author: ardumont
  • P290 pypi project with no sdist (378 out of 6077, 6.22 %)
    allspark
    archiveorg
    audit
    astar
    apicheckr
    ...
    • Aug 22 2018, 2:53 PM
    • 378 Lines
    Author: ardumont
  • P289 pypi sample 2
    {'info': {'author': 'Nathan Harrington',
    'author_email': 'nharrington@wasatchphotonics.com',
    'bugtrack_url': None,
    'classifiers': [],
    'description': 'UNKNOWN',
    ...
    • Aug 2 2018, 3:07 PM
    • 96 Lines
    Author: ardumont
  • P288 pypi metadata sample
    {
    "info": {
    "author": "bernardfrk",
    "author_email": "bernard.frk@gmail.com",
    "bugtrack_url": null,
    ...
    • Aug 2 2018, 3:03 PM
    • 95 Lines
    Author: ardumont
  • P287 loader pypi taking form - wip
    python3
    Python 3.6.6 (default, Jun 27 2018, 14:44:17)
    [GCC 8.1.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import logging; logging.basicConfig(level=logging.DEBUG); from swh.loader.pypi.tasks import LoadPyPiTsk; LoadPyPiTsk().run('arrow', 'https://pypi.org/pypi/arrow/', project_metadata_url='https://pypi.org/pypi/arrow/json')
    ...
    • Aug 1 2018, 2:29 PM
    • 309 Lines
    Author: ardumont
  • P286 mercurial loading release issue - Cleanup release/snapshot/origin_visits procedure
    begin;
    -- identify release and snapshots to mark them for cleanup
    create or replace function mark_wrong_release_and_snapshot_for_cleanup()
    returns void
    ...
    • Jul 25 2018, 6:23 PM
    • 180 Lines
    • SQL
    Author: ardumont
  • P285 D400 added tests
    @istest
    def empty_merge(self):
    """
    """
    full_dict = {
    ...
    • Jul 24 2018, 2:03 PM
    • 146 Lines
    Author: moranegg
  • P284 D400 output-text
    Makefile.local:30: warning: overriding recipe for target 'test'
    ../Makefile.python:22: warning: ignoring old recipe for target 'test'
    ./swh/manage.py test
    Without authentication, endpoint refuses access with 401 response ... ok
    Without authentication, endpoint refuses access with 401 response ... ok
    ...
    • Jul 24 2018, 1:43 PM
    • 283 Lines
    Author: moranegg
  • P283 D400 errors
    ======================================================================
    ERROR: Existing status for update with info should return a 204 response
    ----------------------------------------------------------------------
    Traceback (most recent call last):
    File "/home/morane/Documents/code/swh-environment/swh-deposit/swh/deposit/tests/api/test_deposit_update_status.py", line 80, in update_deposit_status_with_info
    ...
    • Jul 24 2018, 1:41 PM
    • 92 Lines
    Author: moranegg
  • P282 Je-suis-gpl metadata
    <?xml version="1.0" encoding="utf-8"?>
    <entry xmlns="http://www.w3.org/2005/Atom"
    xmlns:codemeta="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0">
    <title>Je suis GPL</title>
    <client>swh</client>
    ...
    • Jul 23 2018, 2:03 PM
    • 25 Lines
    Author: moranegg
  • P281 Arc patch D380 resulting in an error
    This diff is against commit f29e3f49ad973df93bfbf9e543b808fe43003bf9, but
    the commit is nowhere in the working copy. Try to apply it against the
    current working copy state? (9b4eb1e1b43349e4356307e1b5f924c870fc0b25)
    [Y/n] Y
    ...
    • Jul 16 2018, 11:41 AM
    • 25 Lines
    Author: moranegg
  • P280 python3-swh.core build issue due to python3-psycopg2 deps
    Setup apt archive
    -----------------
    Merged Build-Depends: dose-distcheck
    Filtered Build-Depends: dose-distcheck
    ...
    • Jul 2 2018, 2:35 PM
    • 78 Lines
    Author: ardumont
  • P279 deposit error 500
    Jul 02 11:44:49 moma gunicorn3[27165]: [02/Jul/2018 11:44:49] ERROR [django.request:132] Internal Server Error: /1/hal/131/media/
    Jul 02 11:44:49 moma gunicorn3[27165]: Traceback (most recent call last):
    Jul 02 11:44:49 moma gunicorn3[27165]: File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 42, in inner
    Jul 02 11:44:49 moma gunicorn3[27165]: response = get_response(request)
    Jul 02 11:44:49 moma gunicorn3[27165]: File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 187, in _get_response
    ...
    • Jul 2 2018, 1:59 PM
    • 58 Lines
    Author: olasd
  • P277 kibana logs warn
    elasticsearch | [2018-06-22T14:17:31,321][WARN ][o.e.l.LicenseService ] [-7edhpM]
    elasticsearch | #
    elasticsearch | # LICENSE [EXPIRED] ON [THURSDAY, JUNE 21, 2018]. IF YOU HAVE A NEW LICENSE, PLEASE UPDATE IT.
    elasticsearch | # OTHERWISE, PLEASE REACH OUT TO YOUR SUPPORT CONTACT.
    elasticsearch | #
    ...
    • Jun 22 2018, 4:19 PM
    • 26 Lines
    Author: ardumont
  • P276 replication note (somerset - swh-indexer -> dbreplica1 - swh-indexer)
    # master (somerset)
    ## bootstrap db data replication from master to slave:
    ```
    ...
    • Jun 20 2018, 11:31 AM
    • 142 Lines
    Author: ardumont
  • P275 (An Untitled Masterwork)
    select distinct content.sha1 from (
    select distinct walk.target as target
    from (
    select distinct snapshot_branch.target as target
    from (
    ...
    • Jun 19 2018, 3:21 PM
    • 25 Lines
    • SQL
    Author: seirl
  • P274 (An Untitled Masterwork)
    select snapshot.id as snapshot_id, snapshot_branch.name as branch_name, content.sha1, walk.*
    from (
    select snapshot.object_id as object_id
    from origin tablesample system (0.00001)
    inner join origin_visit on origin_visit.origin = origin.id
    ...
    • Jun 18 2018, 7:28 PM
    • 18 Lines
    • SQL
    Author: seirl
  • P273 (An Untitled Masterwork)
    select snapshot.id as snapshot_id, snapshot_branch.name as branch_name, content.sha1, walk.*
    from origin tablesample system (0.00001)
    inner join origin_visit on origin_visit.origin = origin.id
    inner join snapshot on origin_visit.snapshot_id = snapshot.object_id
    inner join snapshot_branches on snapshot.object_id = snapshot_branches.snapshot_id
    ...
    • Jun 18 2018, 7:21 PM
    • 14 Lines
    • SQL
    Author: seirl
  • P272 (An Untitled Masterwork)
    select snapshot.id as snapshot_id, snapshot_branch.name as branch_name, content.sha1, walk.*
    from origin
    inner join origin_visit on origin_visit.origin = origin.id
    inner join snapshot on origin_visit.snapshot_id = snapshot.object_id
    inner join snapshot_branches on snapshot.object_id = snapshot_branches.snapshot_id
    ...
    • Jun 18 2018, 7:20 PM
    • 15 Lines
    • SQL
    Author: seirl
  • P271 (An Untitled Masterwork)
    select snapshot.id as snapshot_id, snapshot_branch.name as branch_name, content.sha1, walk.*
    from snapshot
    inner join snapshot_branches on snapshot.object_id = snapshot_branches.snapshot_id
    inner join snapshot_branch on snapshot_branch.object_id = snapshot_branches.branch_id
    inner join revision on snapshot_branch.target = revision.id
    ...
    • Jun 18 2018, 7:14 PM
    • 13 Lines
    • PostgreSQL
    Author: olasd
  • P270 (An Untitled Masterwork)
    select *
    from snapshot tablesample system (0.0002) repeatable (564)
    inner join snapshot_branches on snapshot_branches.snapshot_id = snapshot.object_id
    inner join snapshot_branch on snapshot_branch.object_id = snapshot_branches.branch_id
    • Jun 18 2018, 7:03 PM
    • 4 Lines
    • SQL
    Author: seirl
  • P269 (An Untitled Masterwork)
    select entries.target
    from
    (
    select swh_directory_walk(revision.directory)
    from snapshot tablesample system (0.001)
    ...
    • Jun 18 2018, 6:33 PM
    • 11 Lines
    • SQL
    Author: seirl
  • P267 first puppet run not enough
    Info: /Stage[main]/Profile::Prometheus::Node/File[/etc/default/prometheus-node-exporter]: Scheduling refresh of Service[prometheus-node-exporter]
    Notice: /Stage[main]/Profile::Prometheus::Node/Service[prometheus-node-exporter]: Triggered 'refresh' from 2 events
    Error: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install prometheus-sql-exporter' returned 100: Reading package lists...
    Building dependency tree...
    Reading state information...
    ...
    • Jun 13 2018, 6:10 PM
    • 74 Lines
    Author: ardumont
  • P266 (An Untitled Masterwork)
    Jun 13 13:56:38 worker11 python3[663]: [2018-06-13 13:56:38,481: INFO/MainProcess] Received task: swh.vault.cooking_tasks.SWHBatchCookingTask[85c43f4b-b97c-4d2f-b6f9-ebbae74d8f7e]
    Jun 13 13:56:40 worker11 python3[682]: [2018-06-13 13:56:40,263: ERROR/Worker-11] Bundle cooking failed.
    Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/swh/vault/cookers/base.py", line 114, in cook
    self.prepare_bundle()
    ...
    • Jun 13 2018, 3:57 PM
    • 17 Lines
    Author: seirl
  • P265 (An Untitled Masterwork)
    Jun 10 06:17:36 worker11 python3[57236]: [2018-06-10 06:17:36,767: INFO/MainProcess] Task swh.vault.cooking_tasks.SWHBatchCookingTask[6a8b530a-2ae6-4d01-bc13-b04314896256] succeeded in 11380.885421179235s: None
    Jun 10 06:17:36 worker11 python3[57236]: [2018-06-10 06:17:36,769: INFO/MainProcess] Task swh.vault.cooking_tasks.SWHBatchCookingTask[fd5ef220-830b-4d78-88c7-b5494ca4cd82] succeeded in 11400.2180494708s: None
    Jun 10 06:17:36 worker11 python3[57236]: [2018-06-10 06:17:36,771: INFO/MainProcess] Task swh.vault.cooking_tasks.SWHBatchCookingTask[d87679ec-3ea7-4c5f-be2e-2ab71b9f92d6] succeeded in 11406.673143764958s: None
    Jun 10 06:17:36 worker11 python3[57236]: [2018-06-10 06:17:36,772: INFO/MainProcess] Task swh.vault.cooking_tasks.SWHBatchCookingTask[64d1ce65-bb21-40cf-bd16-89b44faa352c] succeeded in 11318.264325065073s: None
    Jun 10 06:17:36 worker11 python3[57236]: [2018-06-10 06:17:36,773: WARNING/MainProcess] consumer: Connection to broker lost. Trying to re-establish the connection...
    ...
    • Jun 13 2018, 3:55 PM
    • 59 Lines
    Author: seirl
  • P264 (An Untitled Masterwork)
    Jun 13 10:07:28 moma python3[26617]: 2018-06-13 10:07:28 [26617] [ERROR] Internal Server Error: /browse/directory/977fc4b98c0e85816348cebd3b12026407c368b6/
    Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 42, in inner
    response = get_response(request)
    File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 187, in _get_response
    ...
    • Jun 13 2018, 12:08 PM
    • 15 Lines
    Author: olasd
  • P260 (An Untitled Masterwork)
    swh-dedup=# select count(*) from chunk tablesample system (0.0003) left join chunked_content_chunk on chunk_id = chunk.id;
    count
    -------
    4181
    (1 row)
    ...
    • Jun 5 2018, 3:41 PM
    • 15 Lines
    Author: seirl
  • P258 swh-dedup performance stats query planner
    swh-dedup=# explain
    select
    count(distinct chunk.id) as count_chunk,
    sum(chunk.length) as total_size,
    sum(chunk.compressed_length) as total_compressed_size
    ...
    • May 24 2018, 9:13 PM
    • 25 Lines
    • PostgreSQL
    Author: seirl
  • P257 Strange Web-UI error for not found readme.md
    On this directory: https://archive.softwareheritage.org/browse/directory/31b5c8cc985d190b5a7ef4878128ebfdc2358f49/
    The readme didn't show up and when i clicked to see the content i got the error below.
    After I backed to the directory view- i have the error text in the readme placeholder (I have a local screenshot of this)
    ----------------------------------------------------------------------------------------------------------
    ...
    • May 23 2018, 3:51 PM
    • 14 Lines
    Author: moranegg
  • P256 (An Untitled Masterwork)
    swh-dedup=# explain select count(chunk.length)
    from chunk tablesample system(0.001)
    left join chunked_content_chunk on chunk_id = chunk.id;
    QUERY PLAN
    --------------------------------------------------------------------------------------------------
    ...
    • May 7 2018, 6:47 PM
    • 29 Lines
    Author: seirl
  • P255 Sphinx error
    Traceback (most recent call last):
    File "/home/morane/.venv/lib/python3.5/site-packages/sphinx/config.py", line 161, in __init__
    execfile_(filename, config)
    File "/home/morane/.venv/lib/python3.5/site-packages/sphinx/util/pycompat.py", line 150, in execfile_
    exec_(code, _globals)
    ...
    • May 4 2018, 4:28 PM
    • 29 Lines
    Author: moranegg
  • P254 Broken packaging for unstable
    report:
    -
    package: sbuild-build-depends-swh-web-dummy
    version: 0.invalid.0
    architecture: amd64
    ...
    • May 4 2018, 3:10 PM
    • 42 Lines
    Author: anlambert
  • P253 swh-vault test results anlambert psql 10.3
    python3 -m nose -sv --with-doctest ./swh/vault/tests
    createdb : la création de la base de données a échoué : ERREUR: la base de données « softwareheritage-test-storage » existe déjà
    createdb : la création de la base de données a échoué : ERREUR: la base de données « softwareheritage-test-vault » existe déjà
    test_available (test_backend.TestBackend) ... ok
    test_cache_expire_oldest (test_backend.TestBackend) ... ok
    ...
    • Apr 26 2018, 2:44 PM
    • 206 Lines
    Author: anlambert
  • P252 swh-vault test results anlambert
    python3 -m nose -sv --with-doctest ./swh/vault/tests
    createdb : la création de la base de données a échoué : ERREUR: la base de données « softwareheritage-test-vault » existe déjà
    createdb : la création de la base de données a échoué : ERREUR: la base de données « softwareheritage-test-storage » existe déjà
    pg_restore: [programme d'archivage (db)] Erreur pendant le traitement de la TOC (« PROCESSING TOC ») :
    pg_restore: [programme d'archivage (db)] Erreur à partir de l'entrée TOC 240 ; 1259 2506400 SEQUENCE metadata_provider_id_seq ndandrim
    ...
    • Apr 26 2018, 2:36 PM
    • 415 Lines
    Author: anlambert
  • P251 [fixed] swh-vault: traceback `make test` not working locally
    python3 -m nose -sv --with-doctest ./swh/vault/tests
    createdb: database creation failed: ERROR: database "softwareheritage-test-vault" already exists
    Time: 2.739 ms
    Time: 0.344 ms
    Time: 0.454 ms
    ...
    • Apr 26 2018, 10:32 AM
    • 506 Lines
    Author: ardumont
  • P250 (An Untitled Masterwork)
    In [51]: c.batch_progress(4)
    Out[51]:
    {'bundles': [{'id': 3,
    'obj_id': '7d4aecffc20478ea6807b9649b25b71e22ebbcb6',
    'obj_type': 'revision_gitfast',
    ...
    • Apr 24 2018, 6:23 PM
    • 22 Lines
    • JSON
    Author: seirl
  • P249 replication issue?
    softwareheritage=> \conninfo
    You are connected to database "softwareheritage" as user "guest" on host "somerset.internal.softwareheritage.org" at port "5433".
    SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)
    softwareheritage=> select * from origin where type='deposit' and url like '%hal-01767263%';
    ┌────┬──────┬─────┬────────┬─────────┐
    ...
    • Apr 17 2018, 9:35 AM
    • 23 Lines
    Author: ardumont
  • P248 .psqlrc
    \pset linestyle unicode
    \pset border 2
    \pset null ¤
    \set PROMPT1 '%[%033[1;;40m%]%`date +%H:%M:%S`%[%033[0m%] %[%033[33;1m%]%x%[%033[0m%]%[%033[1m%]%/%[%033[0m%]@%[%033[33;1m%]%m%[%033[0m%]:%[%033[33;1m%]%>%[%033[0m%]%R%# '
    \timing
    ...
    • Apr 13 2018, 2:41 PM
    • 7 Lines
    • PostgreSQL
    Author: olasd
  • P247 mimetype indexer error
    worker09.euwest.azure: Apr 12 07:45:28 worker09 python3[128430]: [2018-04-12 07:45:27,737: ERROR/Worker-65266] Problem when reading contents metadata.
    worker09.euwest.azure: Traceback (most recent call last):
    worker09.euwest.azure: File "/usr/lib/python3/dist-packages/swh/indexer/indexer.py", line 359, in run
    worker09.euwest.azure: raw_content = self.objstorage.get(sha1)
    worker09.euwest.azure: File "/usr/lib/python3/dist-packages/swh/objstorage/multiplexer/multiplexer_objstorage.py", line 137, in get
    ...
    • Apr 12 2018, 9:47 AM
    • 18 Lines
    • Plain Text
    Author: ardumont
  • P246 (An Untitled Masterwork)
    swh-dedup=# select algo, avg(duration_us) from chunked_content left join chunking_method on chunking_method.id = chunked_content.method_id group by algo;
    algo | avg
    ---------+-----------------------
    buzhash | 976.7281333333333333
    rabin | 1222.3993333333333333
    ...
    • Apr 10 2018, 6:35 PM
    • 6 Lines
    Author: seirl
  • P245 Crossminer first batch 0_1000_origin.csv
    https://github.com/Vishcn/vishWeb,ded9ec3e8ccc8f84d3a0541ed0c5a4a8606038ea
    https://github.com/YxhWife/firstPoj,19f0650345e0ff2cade69f6a105908ea47d0afe5
    https://github.com/ZergyPoo/VolunteerManager,e7cdde6ddc8e4500648de72cfce3aa6bd28d0a12
    https://github.com/adriancosma88/test-java-ocp,f8381b7ea001fe1db9b55c4fbdd6f7bacae74ef6
    https://github.com/ansell/bio2rdf-helpers,9ea852f3a0c73cab0403509f67d4b4164f1b30b2
    ...
    • Apr 10 2018, 4:15 PM
    • 10 Lines
    Author: moranegg
  • P244 (An Untitled Masterwork)
    17:52:45 :
    550 new
    17:54:03 :
    66 done
    ...
    • Mar 29 2018, 5:58 PM
    • 18 Lines
    Author: seirl
  • P243 (An Untitled Masterwork)
    +---------------------------------------------------------------------------------------------+
    | UNIBO NETWORK |
    | |
    | |
    | +-----------------------------------------+ +-------------------------------+ |
    ...
    • Mar 28 2018, 1:36 PM
    • 45 Lines
    Author: seirl
  • P242 [fixed] Hanging 'task reading' function on filtering period without data
    begin;
    create type task_record as (
    task_id bigint,
    task_policy task_policy,
    task_status task_status,
    ...
    • Mar 22 2018, 12:07 PM
    • 64 Lines
    • SQL
    Author: ardumont
  • P241 first indexing which seems to work ;)
    {
    "took": 1,
    "timed_out": false,
    "_shards": {
    "total": 1,
    ...
    • Mar 22 2018, 10:38 AM
    • 43 Lines
    • JSON
    Author: ardumont
First
Prev
Next
About · Developer information · Legal