Page MenuHomeSoftware Heritage
Create Task
Maniphest T2620

Inventory of the swh infrastructure
Closed, MigratedEdits Locked
Actions

Description

In order to prepare the networks refactoring to isolate the different environments (staging, production and possibly an additional admin vlan), an exhaustive list of all the running services is needed.

An inventory app will be used to have a live and up to date vision of the infrastructure.
After a quick poc, netbox seems to do the job by supporting, hardware, virtualisation, networks, services, .... and being configurable by api.

List of the tasks to perform :

  • Add the support of netbox in puppet
  • Provision and install a server to run netbox
  • Reference the physical and virtual servers
  • Reference the current networks and ips
  • Reference the running services

The next step (refactoring) will be described in another task after the analysis of the current state.

Revisions and Commits

Restricted Differential RevisionRestricted Diffusion Commit
rSENV Puppet Environment
D4119rSENV7e7fc6651790 vagrantfile: update the netbox vm's name to match its real name
rSPPRIVC Private Puppet Data, censored
D4027rSPPRIVC3096c8805ca2 Add netbox passwords
D4027rSPPRIVC4699252ae7cb Add vsellier's password
rSPSITE puppet-swh-site
D4025rSPSITEe3866b5d00c8 Add netbox inventory tool support

Related Objects

Event Timeline

vsellier changed the task status from Open to Work in Progress.Sep 21 2020, 5:54 PM
vsellier claimed this task.
vsellier triaged this task as Normal priority.
vsellier created this task.
vsellier added a revision: D4025: Add netbox inventory tool support.Sep 24 2020, 9:41 AM
vsellier added a revision: Restricted Differential Revision.Sep 24 2020, 9:59 AM
vsellier added a revision: D4027: Add netbox password.Sep 24 2020, 10:03 AM
vsellier mentioned this in rSENV5fe9303121a6: Declare puppet/redis repository on swh forge.Sep 24 2020, 2:54 PM
vsellier added a revision: D4119: vagrantfile: update the netbox vm's name to match its real name.Oct 1 2020, 6:55 PM
vsellier added a commit: rSENV7e7fc6651790: vagrantfile: update the netbox vm's name to match its real name.Oct 1 2020, 7:29 PM
vsellier added a commit: Restricted Diffusion Commit.Oct 1 2020, 7:36 PM
vsellier added a commit: rSPPRIVC4699252ae7cb: Add vsellier's password.
vsellier added a commit: rSPPRIVC3096c8805ca2: Add netbox passwords.
vsellier added a commit: rSPSITEe3866b5d00c8: Add netbox inventory tool support.
vsellier updated the task description. (Show Details)Oct 1 2020, 7:39 PM
vsellier updated the task description. (Show Details)Oct 2 2020, 4:42 PM
vsellier added a comment.Oct 2 2020, 4:59 PM

The service is up and runnig at https://inventory.internal.softwareheritage.org
I will add the admin password on the credentials.

The initial import of the puppet fact was done with the quick and dirty import script[1]. Each imported ressource is tagged with the tag `puppet-import` to keep the information for later.

Now the time to the manual work of categorization is coming.

[1]: https://forge.softwareheritage.org/source/snippets/browse/master/sysadmin/netbox-importer/

vsellier closed this task as Resolved.Nov 3 2020, 3:01 PM

Netbox is up and used since several weeks now.
The backup is correctly configured:

root@bojimans:/etc/borgmatic# borgmatic info --archive latest
borg@banco.internal.softwareheritage.org:/srv/borg/repositories/bojimans.internal.softwareheritage.org: Displaying summary info for archives
Archive name: bojimans.internal.softwareheritage.org-2020-11-03T12:41:02.069548
Archive fingerprint: f8d0932e85043e61f59b21856a2cd871336d2b7e7a3e7d6e681cd4333f091581
Comment:
Hostname: bojimans
Username: root
Time (start): Tue, 2020-11-03 12:41:03
Time (end): Tue, 2020-11-03 12:41:10
Duration: 7.19 seconds
Number of files: 62391
Command line: /usr/bin/borg create --exclude-from /tmp/tmpo2f1n9xq --exclude-caches --exclude-if-present .nobackup 'borg@banco.internal.softwareheritage.org:/srv/borg/repositories/bojimans.internal.softwareheritage.org::bojimans.internal.softwareheritage.org-{now:%Y-%m-%dT%H:%M:%S.%f}' /
Utilization of maximum supported archive size: 0%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:                1.84 GB            938.96 MB              2.12 MB
All archives:               64.97 GB             32.95 GB              1.06 GB
                       Unique chunks         Total chunks
Chunk index:                   61324              2163683
root@bojimans:~# borgmatic mount --archive latest --mount-point /tmp/bck
root@bojimans:/tmp/bck/opt# du --apparent-size -schP {/tmp/bck,}/opt/netbox* {/tmp/bck,}/var/lib/netbox {/tmp/bck,}/var/lib/postgresql/
17	/tmp/bck/opt/netbox
141M	/tmp/bck/opt/netbox-2.9.3
17	/opt/netbox
156M	/opt/netbox-2.9.3
0	/tmp/bck/var/lib/netbox
16K	/var/lib/netbox
75M	/tmp/bck/var/lib/postgresql/
75M	/var/lib/postgresql/
446M	total

the difference of size return by `du` on the netbox directory seems due to the computation of the size on the fuse fs

root@bojimans:~# mount | grep /tmp/bck
borgfs on /tmp/bck type fuse (ro,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions)

There is no visible differences on the 2 directories :

root@bojimans:~# diff -r {/tmp/bck,}/opt/netbox-2.9.3/
root@bojimans:~#
vsellier updated the task description. (Show Details)Nov 3 2020, 3:02 PM
vsellier mentioned this in T1414: Set up an inventory app.Jan 5 2021, 2:43 PM
gitlab-migration changed the task status from Resolved to Migrated.Oct 19 2022, 5:58 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

About · Developer information · Legal