Page MenuHomeSoftware Heritage

common/origin_save: Speedup save requests list loading
ClosedPublic

Authored by anlambert on Thu, Jun 25, 2:58 PM.

Details

Summary

When loading save requests list, the web application will try to find
a visit date for each submitted origin save request. This process is
repeated while a visit date has not been found yet.

To speedup the loading of the requests lst, stop trying to find a visit
date one month after a save request submission as those requests to storage
are expensive and associated loading task surely ended up with errors.

Diff Detail

Repository
rDWAPPS Web applications
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

anlambert created this revision.Thu, Jun 25, 2:58 PM

Could you add a test?

Build is green

Patch application report for D3354 (id=11899)

Rebasing onto 608f13c1cb...

Current branch diff-target is up to date.
Changes applied before test
commit 40d4ba6bb0a23ae2b5961c7b2aa95e562419e2aa
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Thu Jun 25 14:52:50 2020 +0200

    common/origin_save: Speedup save requests list loading
    
    When loading save requests list, the web application will try to find
    a visit date for each submitted origin save request. This process is
    repeated while a visit date has not been found yet.
    
    To speedup the loading of the requests lst, stop trying to find a visit
    date one month after a save request submission as those requests to storage
    are expensive and associated loading task surely ended up with errors.

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/226/ for more details.

Could you add a test?

Diff updated. Jeez, it was so painful to write as mocking is required here.

Build is green

Patch application report for D3354 (id=11902)

Rebasing onto 608f13c1cb...

Current branch diff-target is up to date.
Changes applied before test
commit 5d9995f3beb2a03621f76b304ffecdd915bdfbb2
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Thu Jun 25 14:52:50 2020 +0200

    common/origin_save: Speedup save requests list loading
    
    When loading save requests list, the web application will try to find
    a visit date for each submitted origin save request. This process is
    repeated while a visit date has not been found yet.
    
    To speedup the loading of the requests lst, stop trying to find a visit
    date one month after a save request submission as those requests to storage
    are expensive and associated loading task surely ended up with errors.

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/227/ for more details.

ardumont accepted this revision.Fri, Jun 26, 9:57 AM
This revision is now accepted and ready to land.Fri, Jun 26, 9:57 AM

Build is green

Patch application report for D3354 (id=11917)

Rebasing onto 608f13c1cb...

Current branch diff-target is up to date.
Changes applied before test
commit 04170796f41ba734729d5d4555b5dc2494a3bf1c
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Thu Jun 25 14:52:50 2020 +0200

    common/origin_save: Speedup save requests list loading
    
    When loading save requests list, the web application will try to find
    a visit date for each submitted origin save request. This process is
    repeated while a visit date has not been found yet.
    
    To speedup the loading of the requests lst, stop trying to find a visit
    date one month after a save request submission as those requests to storage
    are expensive and associated loading task surely ended up with errors.

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/229/ for more details.