Page MenuHomeSoftware Heritage

Fix the bitbucket lister;
ClosedPublic

Authored by douardda on Jan 31 2019, 4:15 PM.

Details

Summary
  • Handle properly the date-like bounds
  • Fix the SWHIndexingLister.db_partition_indices the db query actually returns a table-like (2d) structure.

Also

  • Log errors when fetching an url in SWHListerHttpTransport
  • Use the named logger instead of the root logger in lister_base.py

Diff Detail

Repository
rDLS Listers
Branch
master
Lint
Lint Skipped
Unit
Unit Tests Skipped
Build Status
Buildable 3921
Build 5137: tox-on-jenkinsJenkins
Build 5136: arc lint + arc unit

Event Timeline

vlorentz added inline comments.
requirements.txt
6

Did you check if you can use arrow (which we already depend on) instead?

douardda added inline comments.
requirements.txt
6

nope, old habits I guess... let me check...

ah, doc says

Some ISO-8601 compliant strings are recognized and parsed without a format string

so maybe it's enough... maybe

ardumont added a subscriber: ardumont.

Just a couple of non blocking questions.

swh/lister/bitbucket/lister.py
45

What is that error?

Also why not a ValueError?

This revision is now accepted and ready to land.Feb 1 2019, 8:51 AM

What is that error?

Don't remember, looks pretty much like a "debug" statement in fact. Let me check that

douardda added inline comments.
swh/lister/bitbucket/lister.py
45

Ok, in fact this looks very much like a "debug" statement before the hunk below in swh/lister/core/indexing_lister.py

douardda added inline comments.
requirements.txt
6

https://github.com/crsmithdev/arrow/issues/91 makes me feel I want to stick to iso8601 for now

remove a debug piece of code

Do really remove a debug piece of code and fix the test

This revision was automatically updated to reflect the committed changes.