Page MenuHomeSoftware Heritage

Cleanup the old counters environment
Closed, MigratedEdits Locked

Description

The counters are in production since a few months now and no issues were detected.

The old counters stack can be decommissioned.

After a discussion, we (sysadms) have decided to keep the counter computation in database and the export to prometheus to have a idea of the drift of the hyperloglogs counters.

However, some tasks are still needed:

  • Reduce the refresh of the database counter to once a week or month to reduce the pressure on the database
  • Cleanup the puppet code relative the plumbing around stats.softwareheritage.org (script, crons, historical data)
  • Cleanup apache the configuration to remove the unused stats.softwarehritage.org vhost
  • Remove the files on pergamon

Event Timeline

vsellier triaged this task as Normal priority.Jun 29 2021, 11:20 AM
vsellier created this task.
vsellier changed the task status from Open to Work in Progress.Aug 5 2021, 5:37 PM
vsellier moved this task from Backlog to in-progress on the System administration board.

Pergamon manual cleanup after D6064 is apply:

  • Remove /var/www/stats.export.softwareheritage.org directory
  • Remove apache vhosts:
    • /etc/apache2/sites-enabled/25-stats.export.softwareheritage.org_non-ssl.conf
    • /etc/apache2/sites-enabled/25-stats.export.softwareheritage.org_ssl.conf
    • /etc/apache2/sites-available/25-stats.export.softwareheritage.org_non-ssl.conf
    • /etc/apache2/sites-available/25-stats.export.softwareheritage.org_ssl.conf
  • check crontab removal: export_archive_counters
  • remove '/usr/local/bin/export_archive_counters.py'
  • remove '/usr/local/share/swh-data' directory

For the counters frequency, I have still not found which cron/command regularly is responsible of lauching the refresh

For the counters frequency, I have still not found which cron/command regularly is responsible of lauching the refresh

Maybe looking around what calls the following functions could help.

$ psql service=swh

09:33:13 softwareheritage@belvedere:5432=> \df swh_update_counter*
                                      List of functions
+--------+----------------------------------+------------------+---------------------+------+
| Schema |               Name               | Result data type | Argument data types | Type |
+--------+----------------------------------+------------------+---------------------+------+
| public | swh_update_counter               | void             | object_type text    | func |
| public | swh_update_counter_bucketed      | void             |                     | func |
| public | swh_update_counters_from_buckets | trigger          |                     | func |
+--------+----------------------------------+------------------+---------------------+------+
(3 rows)

For the counters frequency, I have still not found which cron/command regularly is responsible of lauching the refresh

Maybe looking around what calls the following functions could help.

I found it, not through code search though, just that hit me it could be there:

so cron installed on belvedere for the user postgres.

postgres@belvedere:~$ crontab -l | grep update-counter
29 */2  *   *   *     /usr/bin/chronic /usr/bin/flock -xn /srv/softwareheritage/postgres/swh-update-counter.lock /usr/bin/psql -p 5433 softwareheritage -c "select swh_update_counter(object_type) from object_counts where single_update = true order by last_update limit 1"
  • D6064 landed
  • manual cleanup:
    • the apache vhost was removed by puppet
    • /var/www/stats.export.softwareheritage.org directory removed
    • the crontab was removed by puppet
    • /usr/local/bin/export_archive_counters.py file removed
    • /usr/local/share/swh-data directory removed
  • the refresh of the database counter is now scheduled each monday at 6:29 AM
postgres@belvedere:~$ crontab -l | grep counter
29 6  *   *  mon     /usr/bin/chronic /usr/bin/flock -xn /srv/softwareheritage/postgres/swh-update-counter.lock /usr/bin/psql -p 5433 softwareheritage -c "select swh_update_counter(object_type) from object_counts where single_update = true order by last_update limit 1"
vsellier moved this task from in-progress to done on the System administration board.