Page MenuHomeSoftware Heritage

api: Make next links in response headers contain full URLs
ClosedPublic

Authored by anlambert on Dec 13 2019, 11:52 AM.

Details

Summary

For commodity of use by api users, return full URLS in Link response header.

Related to T2147

Diff Detail

Repository
rDWAPPS Web applications
Branch
add-full-uris-in-api-next-links
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 9749
Build 14380: Cypress tests for swh-web diffsJenkins
Build 14379: tox-on-jenkinsJenkins
Build 14378: arc lint + arc unit

Event Timeline

Some code factorization can be applied here.

Rebase to master and factorize code

Cypress Jenkins build is currently failing since the fresh release of confluent_kafka 1.3.0

Running setup.py bdist_wheel for confluent-kafka: started
  Running setup.py bdist_wheel for confluent-kafka: finished with status 'error'
  Complete output from command /home/jenkins/swh-web-env/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-iv7br_vy/confluent-kafka/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-1fkykhft --python-tag cp37:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/confluent_kafka
  copying confluent_kafka/__init__.py -> build/lib.linux-x86_64-3.7/confluent_kafka
  creating build/lib.linux-x86_64-3.7/confluent_kafka/admin
  copying confluent_kafka/admin/__init__.py -> build/lib.linux-x86_64-3.7/confluent_kafka/admin
  creating build/lib.linux-x86_64-3.7/confluent_kafka/kafkatest
  copying confluent_kafka/kafkatest/verifiable_client.py -> build/lib.linux-x86_64-3.7/confluent_kafka/kafkatest
  copying confluent_kafka/kafkatest/__init__.py -> build/lib.linux-x86_64-3.7/confluent_kafka/kafkatest
  copying confluent_kafka/kafkatest/verifiable_consumer.py -> build/lib.linux-x86_64-3.7/confluent_kafka/kafkatest
  copying confluent_kafka/kafkatest/verifiable_producer.py -> build/lib.linux-x86_64-3.7/confluent_kafka/kafkatest
  creating build/lib.linux-x86_64-3.7/confluent_kafka/avro
  copying confluent_kafka/avro/load.py -> build/lib.linux-x86_64-3.7/confluent_kafka/avro
  copying confluent_kafka/avro/cached_schema_registry_client.py -> build/lib.linux-x86_64-3.7/confluent_kafka/avro
  copying confluent_kafka/avro/__init__.py -> build/lib.linux-x86_64-3.7/confluent_kafka/avro
  copying confluent_kafka/avro/error.py -> build/lib.linux-x86_64-3.7/confluent_kafka/avro
  creating build/lib.linux-x86_64-3.7/confluent_kafka/avro/serializer
  copying confluent_kafka/avro/serializer/__init__.py -> build/lib.linux-x86_64-3.7/confluent_kafka/avro/serializer
  copying confluent_kafka/avro/serializer/message_serializer.py -> build/lib.linux-x86_64-3.7/confluent_kafka/avro/serializer
  running build_ext
  building 'confluent_kafka.cimpl' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/confluent_kafka
  creating build/temp.linux-x86_64-3.7/confluent_kafka/src
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/jenkins/swh-web-env/include -I/usr/include/python3.7m -c confluent_kafka/src/confluent_kafka.c -o build/temp.linux-x86_64-3.7/confluent_kafka/src/confluent_kafka.o
  In file included from confluent_kafka/src/confluent_kafka.c:17:
  confluent_kafka/src/confluent_kafka.h:22:10: fatal error: librdkafka/rdkafka.h: No such file or directory
   #include <librdkafka/rdkafka.h>
            ^~~~~~~~~~~~~~~~~~~~~~
  compilation terminated.
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
  
  ----------------------------------------
  Failed building wheel for confluent-kafka
  Running setup.py clean for confluent-kafka
  Running setup.py bdist_wheel for psutil: started
  Running setup.py bdist_wheel for psutil: finished with status 'done'
  Stored in directory: /home/jenkins/.cache/pip/wheels/52/41/b0/bf50409fe2b1d3b79afa3eed71b54b3e30fe5b695db2c7ba2e

Nevertheless, inside my virtualenv, the linux binary wheels are correctly downloaded:

13:40 $ pip install --upgrade confluent_kafka
Collecting confluent_kafka
  Downloading https://files.pythonhosted.org/packages/1e/87/2d4d49f2aa93eed080e0a7fe3f37e9e1e228199c6381357928fe45876884/confluent_kafka-1.3.0-cp37-cp37m-manylinux2010_x86_64.whl (9.1MB)
     |████████████████████████████████| 9.1MB 13.7MB/s 
Installing collected packages: confluent-kafka
  Found existing installation: confluent-kafka 1.2.0
    Uninstalling confluent-kafka-1.2.0:
      Successfully uninstalled confluent-kafka-1.2.0
Successfully installed confluent-kafka-1.3.0

The manylinux2010 platform tag is quite recent so upgrading pip to its latest version in the CI job should fix the issue.

Update: Solution confirmed by hot patching the CI job configuration, D2450 created to update the concerned Jenkins jobs.

vlorentz added a subscriber: vlorentz.

the api_request_factory fixture doesn't seem to be used in all the tests you added it for. Intentional?

This revision is now accepted and ready to land.Dec 13 2019, 2:26 PM

the api_request_factory fixture doesn't seem to be used in all the tests you added it for. Intentional?

It is used in every test it is declared, you got mistaken by the Phabricator diff that truncates the files.

the api_request_factory fixture doesn't seem to be used in all the tests you added it for. Intentional?

It is used in every test it is declared, you got mistaken by the Phabricator diff that truncates the files.

Indeed. But it was also Firefox's fault for not updating its list of search matches after I unfolded these.

This revision was landed with ongoing or failed builds.Dec 13 2019, 3:01 PM
This revision was automatically updated to reflect the committed changes.