Page MenuHomeSoftware Heritage

db_utils: Make connect_to_conninfo use through contextmanager
ClosedPublic

Authored by ardumont on Apr 25 2022, 2:14 PM.

Details

Summary

This allows to reduce the boilerplate regarding initial connection failures.

While trying to figure out the type issue, i tampered with this a bit ¯\_(ツ)_/¯.

Test Plan

tests should still be happy

Diff Detail

Repository
rDCORE Foundations and core functionalities
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D7644 (id=27663)

Rebasing onto 95709cffdd...

Current branch diff-target is up to date.
Changes applied before test
commit 68c2cc8fd7b0e32693dc5b39185b59ba191f2d91
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Apr 25 13:25:36 2022 +0200

    db_utils: Make connect_to_conninfo use through contextmanager
    
    This allows to reduce the boilerplate regarding initial connection failures.

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/404/ for more details.

This revision is now accepted and ready to land.Apr 25 2022, 2:48 PM
This revision now requires changes to proceed.Apr 25 2022, 2:52 PM
swh/core/db/db_utils.py
82–83

This could be raised for any number of reasons, eg. a syntax error in the query

swh/core/db/db_utils.py
82–83

and when it errors, then the contextmanager's __enter__ would return None, causing another error when used

swh/core/db/db_utils.py
82–83

This could be raised for any number of reasons, eg. a syntax error in the query

But that part is actually used as before.

and when it errors, then the contextmanager's enter would return None, causing another error when used

so any particular hint as to how improve this at all?

swh/core/db/db_utils.py
82–83

But that part is actually used as before.

It's not. Before, logger.exception("Failed to connect to %s`", db_or_conninfo) was in an except` block which only covered db = connect_to_conninfo(db_or_conninfo).

Now, the except block covers the whole lifetime of the context manager. It should be like this:

if isinstance(db_or_conninfo, pgconnection):
    yield db_or_conninfo
else:
    if "=" not in db_or_conninfo and "//" not in db_or_conninfo:
        # Database name
        db_or_conninfo = f"dbname={db_or_conninfo}"

    try:
        db psycopg2.connect(db_or_conninfo)
    except psycopg2.Error:
        logger.exception("Failed to connect to `%s`", db_or_conninfo)
    else:
        yield db

so any particular hint as to how improve this at all?

you need to check for None on every call to the context manager, and return early if it is

swh/core/db/db_utils.py
82–83

yes, right.
I thought of something like this yesterday evening but got side-tracked by other things, thx!

The last try except, i'd write it more concisely like this though:

try:
    yield psycopg2.connect(db_or_conninfo)
except psycopg2.Error:
    logger.exception("Failed to connect to `%s`", db_or_conninfo)

Adapt according to discussion

you should move connect_to_conninfo's try block in the else block

Build is green

Patch application report for D7644 (id=27710)

Rebasing onto 95709cffdd...

Current branch diff-target is up to date.
Changes applied before test
commit 297019774eaa1f428c0b11f4a48728aa3c651a12
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Apr 25 13:25:36 2022 +0200

    db_utils: Make connect_to_conninfo use through contextmanager
    
    This allows to reduce the boilerplate regarding initial connection failures.

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/405/ for more details.

swh/core/db/db_utils.py
82–83

as discussed with val, his suggestion and mine are different.
With mine, the exception is forwarded by the yield, but we don't want that here.
So i kept his better suggestion instead.

This revision is now accepted and ready to land.Apr 26 2022, 12:03 PM