Page MenuHomeSoftware Heritage

Inventory of the swh infrastructure
Closed, MigratedEdits Locked

Description

In order to prepare the networks refactoring to isolate the different environments (staging, production and possibly an additional admin vlan), an exhaustive list of all the running services is needed.

An inventory app will be used to have a live and up to date vision of the infrastructure.
After a quick poc, netbox seems to do the job by supporting, hardware, virtualisation, networks, services, .... and being configurable by api.

List of the tasks to perform :

  • Add the support of netbox in puppet
  • Provision and install a server to run netbox
  • Reference the physical and virtual servers
  • Reference the current networks and ips
  • Reference the running services

The next step (refactoring) will be described in another task after the analysis of the current state.

Event Timeline

vsellier changed the task status from Open to Work in Progress.Sep 21 2020, 5:54 PM
vsellier claimed this task.
vsellier triaged this task as Normal priority.
vsellier created this task.
vsellier added a revision: Restricted Differential Revision.Sep 24 2020, 9:59 AM

The service is up and runnig at https://inventory.internal.softwareheritage.org
I will add the admin password on the credentials.

The initial import of the puppet fact was done with the quick and dirty import script[1]. Each imported ressource is tagged with the tag `puppet-import` to keep the information for later.

Now the time to the manual work of categorization is coming.

[1]: https://forge.softwareheritage.org/source/snippets/browse/master/sysadmin/netbox-importer/

Netbox is up and used since several weeks now.
The backup is correctly configured:

root@bojimans:/etc/borgmatic# borgmatic info --archive latest
borg@banco.internal.softwareheritage.org:/srv/borg/repositories/bojimans.internal.softwareheritage.org: Displaying summary info for archives
Archive name: bojimans.internal.softwareheritage.org-2020-11-03T12:41:02.069548
Archive fingerprint: f8d0932e85043e61f59b21856a2cd871336d2b7e7a3e7d6e681cd4333f091581
Comment:
Hostname: bojimans
Username: root
Time (start): Tue, 2020-11-03 12:41:03
Time (end): Tue, 2020-11-03 12:41:10
Duration: 7.19 seconds
Number of files: 62391
Command line: /usr/bin/borg create --exclude-from /tmp/tmpo2f1n9xq --exclude-caches --exclude-if-present .nobackup 'borg@banco.internal.softwareheritage.org:/srv/borg/repositories/bojimans.internal.softwareheritage.org::bojimans.internal.softwareheritage.org-{now:%Y-%m-%dT%H:%M:%S.%f}' /
Utilization of maximum supported archive size: 0%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:                1.84 GB            938.96 MB              2.12 MB
All archives:               64.97 GB             32.95 GB              1.06 GB
                       Unique chunks         Total chunks
Chunk index:                   61324              2163683
root@bojimans:~# borgmatic mount --archive latest --mount-point /tmp/bck
root@bojimans:/tmp/bck/opt# du --apparent-size -schP {/tmp/bck,}/opt/netbox* {/tmp/bck,}/var/lib/netbox {/tmp/bck,}/var/lib/postgresql/
17	/tmp/bck/opt/netbox
141M	/tmp/bck/opt/netbox-2.9.3
17	/opt/netbox
156M	/opt/netbox-2.9.3
0	/tmp/bck/var/lib/netbox
16K	/var/lib/netbox
75M	/tmp/bck/var/lib/postgresql/
75M	/var/lib/postgresql/
446M	total

the difference of size return by `du` on the netbox directory seems due to the computation of the size on the fuse fs

root@bojimans:~# mount | grep /tmp/bck
borgfs on /tmp/bck type fuse (ro,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions)

There is no visible differences on the 2 directories :

root@bojimans:~# diff -r {/tmp/bck,}/opt/netbox-2.9.3/
root@bojimans:~#