Page MenuHomeSoftware Heritage

System administrationFolder
ActivePublic

Members

  • This project does not have any members.

Watchers

  • This project does not have any watchers.

Details

Description

general system administration tasks, not specific to any product

Recent Activity

Today

olasd added a comment to T1414: Set up an inventory app.

As software is mostly declared in puppet, I think the main areas that could be improved would be

  • hardware inventory
  • network topology
  • puppet reports integration
Tue, Dec 11, 2:11 PM · System administration, Sprint 2018 12
olasd added a subtask for T1434: Refactor prometheus SQL exporter configuration generation to use latest version from elephant shed: T1436: Integrate swh-storage metrics in prometheus.
Tue, Dec 11, 1:42 PM · System administration
olasd added a parent task for T1437: Rewrite the munin stats export for the website to use prometheus: T1436: Integrate swh-storage metrics in prometheus.
Tue, Dec 11, 1:17 PM · System administration
olasd triaged T1437: Rewrite the munin stats export for the website to use prometheus as Normal priority.
Tue, Dec 11, 1:17 PM · System administration
olasd added a subtask for T1355: Move the object counter from munin to prometheus: T1436: Integrate swh-storage metrics in prometheus.
Tue, Dec 11, 1:16 PM · System administration

Yesterday

olasd added a parent task for T1355: Move the object counter from munin to prometheus: T1356: Kill munin.
Mon, Dec 10, 10:58 PM · System administration
olasd added a subtask for T1356: Kill munin: T1355: Move the object counter from munin to prometheus.
Mon, Dec 10, 10:58 PM · Sprint 2018 12, System administration
olasd closed T1434: Refactor prometheus SQL exporter configuration generation to use latest version from elephant shed as Resolved by committing rSPSITEcd93710eaf95: Update update-prometheus-sql-exporter-config from elephant shed.
Mon, Dec 10, 10:51 PM · System administration
olasd closed T1434: Refactor prometheus SQL exporter configuration generation to use latest version from elephant shed, a subtask of T1355: Move the object counter from munin to prometheus, as Resolved.
Mon, Dec 10, 10:51 PM · System administration
olasd changed the status of T1434: Refactor prometheus SQL exporter configuration generation to use latest version from elephant shed from Open to Work in Progress.
Mon, Dec 10, 2:18 PM · System administration

Fri, Dec 7

ftigeot added a comment to T1372: Compare Rsnapshot / BorgBackup / Backuppc.

Borgbackup is unable to pull data from remote hosts to a central location.

I do not understand this assertion.

Fri, Dec 7, 10:50 AM · System administration

Thu, Dec 6

zack added a comment to T1414: Set up an inventory app.

what kind of inventory we want to do with this? hardware? software? both?

Thu, Dec 6, 1:47 AM · System administration, Sprint 2018 12

Wed, Dec 5

vlorentz triaged T1360: Install a sentry server as Normal priority.
Wed, Dec 5, 5:31 PM · System administration
vlorentz triaged T1359: Add sentry support in every swh running service as Normal priority.
Wed, Dec 5, 5:31 PM · System administration
vlorentz triaged T1358: Setup a sentry service as Normal priority.
Wed, Dec 5, 5:31 PM · System administration
vlorentz triaged T1412: refactor systemd swh services (puppet) as Normal priority.
Wed, Dec 5, 5:11 PM · System administration, Sprint 2018 12
vlorentz triaged T1414: Set up an inventory app as Normal priority.
Wed, Dec 5, 5:11 PM · System administration, Sprint 2018 12
vlorentz raised the priority of T1356: Kill munin from Normal to High.
Wed, Dec 5, 5:10 PM · Sprint 2018 12, System administration

Tue, Dec 4

douardda added a comment to T1372: Compare Rsnapshot / BorgBackup / Backuppc.

Borgbackup is unable to pull data from remote hosts to a central location.

Tue, Dec 4, 5:33 PM · System administration
ftigeot changed the status of T1372: Compare Rsnapshot / BorgBackup / Backuppc, a subtask of T1282: Revisit backups, from Open to Work in Progress.
Tue, Dec 4, 2:41 PM · System administration
ftigeot changed the status of T1372: Compare Rsnapshot / BorgBackup / Backuppc from Open to Work in Progress.

There is a huge difference between Borgbackup and Rsnapshot + Backuppc: Borgbackup is unable to pull data from remote hosts to a central location.
Its working model is based on Borgbackup running locally and storing data to a local filesystem.

Tue, Dec 4, 2:41 PM · System administration
olasd triaged T1427: Memcached on moma is eating all the memory as High priority.
Tue, Dec 4, 2:08 PM · System administration
ftigeot added a comment to T1392: Add a new hypervisor.

New hypervisor hardware has been racked in our bay at Rocquencourt.
The machine's iDrac management interface is accessible on the management network, under the name swh7-adm.inria.fr (details on the wiki).

Tue, Dec 4, 11:56 AM · System administration
ftigeot closed T1404: Resolve disk full issue on somerset:/srv/softwareheritage/postgres as Resolved.

Service postgresql@10-indexer.service has been restarted on somerset and database replication is once again operating normally.
Postgres wal files are being removed as expected on the master, slowly freeing disk space.

Tue, Dec 4, 11:31 AM · System administration
douardda added a project to T1412: refactor systemd swh services (puppet): System administration.
Tue, Dec 4, 10:52 AM · System administration, Sprint 2018 12
douardda renamed T1414: Set up an inventory app from Set up a an inventory app to Set up an inventory app.
Tue, Dec 4, 10:47 AM · System administration, Sprint 2018 12
douardda added a project to T1356: Kill munin: Sprint 2018 12.
Tue, Dec 4, 10:32 AM · Sprint 2018 12, System administration
douardda created T1414: Set up an inventory app.
Tue, Dec 4, 10:31 AM · System administration, Sprint 2018 12
lingueess added a comment to T1338: Change BBUs on orsay.

Batteries for PERC H700 adapters have the part number U8735 and/or NU209.

Tue, Dec 4, 3:43 AM · System administration

Mon, Dec 3

ftigeot added a comment to T1404: Resolve disk full issue on somerset:/srv/softwareheritage/postgres.

Some no longer useful dump files were removed by seirl@, freeing some space on somerset:/srv/softwareheritage/postgres .

Mon, Dec 3, 3:19 PM · System administration
ftigeot added a comment to T1404: Resolve disk full issue on somerset:/srv/softwareheritage/postgres.

somerset:softwareheritage-indexer is the master database for dbreplica1:softwareheritage-indexer.

Mon, Dec 3, 3:17 PM · System administration
ftigeot added a parent task for T1395: Enlarge disk on dbreplica1: T1404: Resolve disk full issue on somerset:/srv/softwareheritage/postgres.
Mon, Dec 3, 3:11 PM · System administration
ftigeot added a subtask for T1404: Resolve disk full issue on somerset:/srv/softwareheritage/postgres: T1395: Enlarge disk on dbreplica1.
Mon, Dec 3, 3:11 PM · System administration
ftigeot changed the status of T1404: Resolve disk full issue on somerset:/srv/softwareheritage/postgres from Open to Work in Progress.
Mon, Dec 3, 3:10 PM · System administration
ftigeot closed T1395: Enlarge disk on dbreplica1 as Resolved.

The pvmove command was done this morning.

Mon, Dec 3, 3:07 PM · System administration

Wed, Nov 28

olasd added a comment to T1395: Enlarge disk on dbreplica1.

Target size: 8 TB striped across 8 disks (from 2 TB on a single disk).

Wed, Nov 28, 3:04 PM · System administration
olasd triaged T1395: Enlarge disk on dbreplica1 as Unbreak Now! priority.
Wed, Nov 28, 2:35 PM · System administration

Tue, Nov 27

ftigeot added a parent task for T1372: Compare Rsnapshot / BorgBackup / Backuppc: T1282: Revisit backups.
Tue, Nov 27, 4:45 PM · System administration
ftigeot added a subtask for T1282: Revisit backups: T1372: Compare Rsnapshot / BorgBackup / Backuppc.
Tue, Nov 27, 4:45 PM · System administration
ftigeot changed the status of T1392: Add a new hypervisor from Open to Work in Progress.
Tue, Nov 27, 4:42 PM · System administration
olasd triaged T1387: Keep track of exported original artifacts as Wishlist priority.
Tue, Nov 27, 12:04 PM · System administration

Fri, Nov 23

olasd added a comment to T1382: Survey of the data stored in uffizi:/srv/storage/space and banco:/srv/storage/space..

ncdu on several million inodes is *slow*, this is going to take a while...

Fri, Nov 23, 3:51 PM · System administration
olasd changed the status of T1382: Survey of the data stored in uffizi:/srv/storage/space and banco:/srv/storage/space. from Open to Work in Progress.
Fri, Nov 23, 3:50 PM · System administration
olasd changed the status of T1382: Survey of the data stored in uffizi:/srv/storage/space and banco:/srv/storage/space., a subtask of T1371: Move the raw imported data off uffizi /srv/storage/space, which is getting full, from Open to Work in Progress.
Fri, Nov 23, 3:50 PM · System administration
olasd triaged T1382: Survey of the data stored in uffizi:/srv/storage/space and banco:/srv/storage/space. as High priority.
Fri, Nov 23, 3:50 PM · System administration
olasd closed T1381: Survey of cold storage options as Resolved.

In the current (fairly time sensitive) situation, the most sensible option seems to be Azure Cool storage in the West Europe region, minding the exit cost of storing data there.

Fri, Nov 23, 3:47 PM · System administration
olasd closed T1381: Survey of cold storage options, a subtask of T1371: Move the raw imported data off uffizi /srv/storage/space, which is getting full, as Resolved.
Fri, Nov 23, 3:47 PM · System administration
ftigeot added a comment to T1338: Change BBUs on orsay.

At least some of the batteries for PERC H800 adapters use part number KR174 and/or M164C.
Some information leads me to believe they could also be used with PERC H700 adapters.

Fri, Nov 23, 3:20 PM · System administration
ftigeot lowered the priority of T979: Migrate TLS certificates away from the *.softwareheritage.org wildcards from High to Wishlist.

I did some experiments with Letsencrypt but other things were more urgent during the September-October 2018 period and in the end a wildcard Digicert certificate was used again instead.

Fri, Nov 23, 3:04 PM · System administration
olasd updated the task description for T1381: Survey of cold storage options.
Fri, Nov 23, 2:56 PM · System administration