Page MenuHomeSoftware Heritage

lister.debian: Refactor and update the debian initialization step
ClosedPublic

Authored by ardumont on Dec 18 2019, 5:11 PM.

Details

Summary
  • Clarify debian_init (use the correct semantic, suite and component)
  • Integrate new bullseye suite

Note:
In production, we deactivated the jessie suite (which no longer exists).
It was source of trouble in the lister run.
This ultimately lead us to add the missing suite bullseye.
Thus this refactoring.

Test Plan

tox

Diff Detail

Repository
rDLS Listers
Branch
refactor-debian-init
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 9886
Build 14596: tox-on-jenkinsJenkins
Build 14595: arc lint + arc unit

Event Timeline

Plug to branch debian-init-step

olasd requested changes to this revision.Dec 19 2019, 11:49 AM

I have a few marginal change suggestions for you. I think it'd be useful for the function to be fully idempotent.

I guess it'd be nice to also make the distribution name configurable, but that can happen in a separate diff.

swh/lister/debian/__init__.py
18–19

s/build/register with the lister/?

39–43

Maybe this bit should be made idempotent as well. I'm not entirely sure what happens if we call the function twice now.

This revision now requires changes to proceed.Dec 19 2019, 11:49 AM

Thanks will take a closer look at this.

I'll also add tests so we'll know a tad better ;)

swh/lister/debian/__init__.py
39–43

it explodes alright ;)
nice catch.

E       sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "area_distribution_id_name_key"
E       DETAIL:  Key (distribution_id, name)=(1, wheezy/main) already exists.
E
E       [SQL: INSERT INTO area (distribution_id, name, active) VALUES (%(distribution_id)s, %(name)s, %(active)s) RETURNING area.id]
E       [parameters: {'distribution_id': 1, 'name': 'wheezy/main', 'active': True}]
E       (Background on this error at: http://sqlalche.me/e/gkpj)
  • Make debian init step idempotent
  • Add integration test to ensure it is
This revision is now accepted and ready to land.Dec 19 2019, 2:03 PM