Page MenuHomeSoftware Heritage

lister.debian: Refactor and update the debian initialization step
ClosedPublic

Authored by ardumont on Dec 18 2019, 5:11 PM.

Details

Summary
  • Clarify debian_init (use the correct semantic, suite and component)
  • Integrate new bullseye suite

Note:
In production, we deactivated the jessie suite (which no longer exists).
It was source of trouble in the lister run.
This ultimately lead us to add the missing suite bullseye.
Thus this refactoring.

Test Plan

tox

Diff Detail

Repository
rDLS Listers
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Dec 18 2019, 5:11 PM
ardumont edited the summary of this revision. (Show Details)Dec 19 2019, 8:19 AM
ardumont edited the summary of this revision. (Show Details)Dec 19 2019, 9:24 AM
ardumont updated this revision to Diff 8793.Dec 19 2019, 10:52 AM

Plug to branch debian-init-step

olasd requested changes to this revision.Dec 19 2019, 11:49 AM

I have a few marginal change suggestions for you. I think it'd be useful for the function to be fully idempotent.

I guess it'd be nice to also make the distribution name configurable, but that can happen in a separate diff.

swh/lister/debian/__init__.py
24–27

s/build/register with the lister/?

45–49

Maybe this bit should be made idempotent as well. I'm not entirely sure what happens if we call the function twice now.

This revision now requires changes to proceed.Dec 19 2019, 11:49 AM

Thanks will take a closer look at this.

I'll also add tests so we'll know a tad better ;)

ardumont added inline comments.Dec 19 2019, 1:19 PM
swh/lister/debian/__init__.py
45–49

it explodes alright ;)
nice catch.

E       sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "area_distribution_id_name_key"
E       DETAIL:  Key (distribution_id, name)=(1, wheezy/main) already exists.
E
E       [SQL: INSERT INTO area (distribution_id, name, active) VALUES (%(distribution_id)s, %(name)s, %(active)s) RETURNING area.id]
E       [parameters: {'distribution_id': 1, 'name': 'wheezy/main', 'active': True}]
E       (Background on this error at: http://sqlalche.me/e/gkpj)
ardumont updated this revision to Diff 8799.Dec 19 2019, 2:00 PM
  • Make debian init step idempotent
  • Add integration test to ensure it is
olasd accepted this revision.Dec 19 2019, 2:03 PM
This revision is now accepted and ready to land.Dec 19 2019, 2:03 PM
ardumont updated this revision to Diff 8800.Dec 19 2019, 2:04 PM

Plug to master branch