Differential D340

Migrate away from temporary tables for read queries
ClosedPublic
Actions

Authored by olasd on Jun 4 2018, 8:24 PM.

Details

Reviewers

ardumont

Group Reviewers

Reviewers

Maniphest Tasks

T1073: Remove temporary table usage for read-only queries

Commits

rDSTOCa16796047b33: Migrate away from temporary tables for read queries
R65:a16796047b33: Migrate away from temporary tables for read queries
rDSTOa16796047b33: Migrate away from temporary tables for read queries

Summary

To do so, we import a function from a recent version of psycopg2,
execute_values, which can execute queries efficiently with a list of values.

We also scale the cursor back from having things in SQL functions towards having
things inside the db.py database "backend". This will make it easier to iterate,
as we won't have to deploy function changes to twenty different databases.

After these changes, testing the web UI on a physical replica works.

Close T1073

Test Plan

Local integration tests are happy; Navigating the frontend backed by
a physical replica seems to be okay now.

Diff Detail

Repository

rDSTO Storage manager

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

olasd created this revision.Jun 4 2018, 8:24 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJun 4 2018, 8:24 PM

Harbormaster completed remote builds in B1243: Diff 1090.Jun 4 2018, 8:24 PM

Cool stuff.

We also scale the cursor back from having things in SQL functions towards having

things inside the db.py database "backend". This will make it easier to iterate,
as we won't have to deploy function changes to twenty different databases.

Quite!

So, now the db module layer has finally a justified existence ;)