Page MenuHomeSoftware Heritage

counters: Aggregate values for origin and revision graphs
ClosedPublic

Authored by vsellier on Apr 13 2021, 3:10 PM.

Details

Summary
  • Add the static values for origin and revision historical data
  • Add missing points previously added by the webapp

Related to T3232

Test Plan
diff origin/production/pergamon.softwareheritage.org current/pergamon.softwareheritage.org
*******************************************
  File[/etc/bind/keys/local-update] =>
   parameters =>
     content =>
      @@ -2,4 +2,4 @@
       key local-update {
       	algorithm hmac-sha256;
      -	secret "PXCZFyOTX4rbuA/m4K6C1ajqhamei8GlnKpqXRfGMkcVTLhv/WK7qJ0eBq13WJs/33scbYqjzivywkr06b3fMA==";
      +	secret "4wuqkA2nm9wqYbhAVk8OEKigJkQdc1hyk2IiWTbKO3f1x97lhZ4Otq+Bua5XYfYENPle8Eo/mWZl5aArvE7eOQ==";
       };
*******************************************
  File[/etc/bind/rndc.key] =>
   parameters =>
     content =>
      @@ -2,4 +2,4 @@
       key rndc-key {
       	algorithm hmac-md5;
      -	secret "ZMUI2JsPalFY8LsxS2UDc86r72wlumglWcaBSQhdEcahfSGHlZnuRwkvMiYDORE/xXwj8QJCv69+VwTzA43xnQ==";
      +	secret "nLHMR3WOMRFm5uoxaXKT3/aX9b0XnzVsUR8YKC5dZNH5mFVRPmVmE8jpoLP1NEdbxDlXriNqqIvIw5sZxrV2Ug==";
       };
*******************************************
  File[/usr/local/bin/export_archive_counters.py] =>
   parameters =>
     content =>
      @@ -105,9 +105,7 @@
           result = {}
           hist_data = history_data(history_data_file)
      -    # for content, we retrieve existing data and merges with the new one
      -    content_data = get_timestamp_history(server, port, 'content')
      -    result['content'] = hist_data['content'] + content_data
      -    for label in ['origin', 'revision']:
      -        result[label] = get_timestamp_history(server, port, label)
      +    for label in ['content', 'origin', 'revision']:
      +        content_data = get_timestamp_history(server, port, label)
      +        result[label] = hist_data.get(label) + content_data
      _
           print(json.dumps(result))
*******************************************
  File[/usr/local/share/swh-data/history-counters.munin.json] =>
   parameters =>
     content =>
      @@ -1 +1,3 @@
-------------- too long to be added here --------------
*******************************************
*** End octocatalog-diff on pergamon.softwareheritage.org

Diff Detail

Repository
rSPSITE puppet-swh-site
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

vsellier created this revision.

Well, i'll trust you for the datapoints heh ;)

I have one question inline.

site-modules/profile/files/stats_exporter/export_archive_counters.py
115

Do we still need that script?

I thought this got merged within the counter server part.

What did i miss?

site-modules/profile/files/stats_exporter/export_archive_counters.py
115

You're right, as soon the webapp will use swh-counters in production, this script will not be needed anymore.
This diff is just an temporary solution to be able to release D5490 without impacting the production rendering

This revision is now accepted and ready to land.Apr 13 2021, 3:30 PM