Page MenuHomeSoftware Heritage

vsellier (Vincent Sellier)
UserAdministrator

User Details

User Since
Mar 21 2019, 4:14 PM (108 w, 2 d)
Roles
Administrator

Recent Activity

Fri, Apr 16

vsellier accepted D5549: Deploy check save code now end to end check.

just one question inline regarding the icinga test name.
Otherwise, looks good

Fri, Apr 16, 6:00 PM
vsellier accepted D5541: Add end to end save code now check.

lgtm

Fri, Apr 16, 3:52 PM
vsellier closed T3254: Outboarding of tenma as Resolved.
Fri, Apr 16, 2:23 PM · System administration
vsellier added a comment to T3254: Outboarding of tenma.
  • he was not in the list of members of the swh-team (thanks @ardumont for adding me to the owner of the list)
Fri, Apr 16, 2:23 PM · System administration
vsellier updated subscribers of T3254: Outboarding of tenma.
  • unix credentials disabled
  • user tenma removed from the groups Staff and Reviewers in phabricator
  • VPN credential revoked:
root@louvre:~# cd /etc/openvpn/keys
root@louvre:/etc/openvpn/keys# ls
dh1024.pem  dh2048.pem	easyrsa  old  pki  vars  x509-types
root@louvre:/etc/openvpn/keys# ./easyrsa revoke tenma
Fri, Apr 16, 11:51 AM · System administration
vsellier placed T2743: Investigate why logging is inconsistent up for grabs.
Fri, Apr 16, 11:48 AM · Vault, Object storage, Core & foundations
vsellier placed T1410: Kill implicit configuration: new configuration scheme up for grabs.
Fri, Apr 16, 11:48 AM · Core & foundations
vsellier placed T2885: investigate the persistent seemingly benign errors on cooking progress up for grabs.
Fri, Apr 16, 11:48 AM · Vault
vsellier removed a member for Staff: Tanya.
Fri, Apr 16, 11:44 AM
vsellier removed a member for Reviewers: tenma.
Fri, Apr 16, 11:43 AM
vsellier removed a member for Staff: tenma.
Fri, Apr 16, 11:41 AM
vsellier committed rSPPRIVC8abf1d0534b8: refresh compared to uncensored data (authored by vsellier).
refresh compared to uncensored data
Fri, Apr 16, 11:39 AM
vsellier added a revision to T3254: Outboarding of tenma: D5542: Remove tenma's access.
Fri, Apr 16, 11:31 AM · System administration
vsellier requested review of D5542: Remove tenma's access.
Fri, Apr 16, 11:31 AM
vsellier changed the status of T3254: Outboarding of tenma from Open to Work in Progress.
Fri, Apr 16, 10:42 AM · System administration
vsellier moved T3203: docs: Document the firewall installation and procedures from Backlog to Weekly backlog on the System administration board.
Fri, Apr 16, 10:13 AM · Documentation, System administration
vsellier moved T3242: Decommission ClearlyDefined resources from Backlog to in-progress on the System administration board.
Fri, Apr 16, 10:13 AM · System administration
vsellier moved T3243: Replace /dev/sdb and /dev/sdc on storage1.staging from Backlog to in-progress on the System administration board.
Fri, Apr 16, 10:12 AM · System administration, Staging environment
vsellier accepted D5541: Add end to end save code now check.

I'm not a big fan of the random pickup of repo because it adds entropy on the result, but otherwise it's great to be able to monitor this 👍

Fri, Apr 16, 9:35 AM

Thu, Apr 15

vsellier closed T2939: Replace out of order disks on db1.staging and storage1.staging as Resolved.

closing this issue as the quota of disks replacement will be over after T3243.
The disk on db1 looks stable for the moment. It will be removed from the zfs pool in case of problems.

Thu, Apr 15, 6:14 PM · System administration
vsellier closed D5539: Outboarding of tg1999.
Thu, Apr 15, 5:57 PM
vsellier committed rSPSITE73f28d682007: Outboarding of tg1999 (authored by vsellier).
Outboarding of tg1999
Thu, Apr 15, 5:57 PM
vsellier added a comment to T3242: Decommission ClearlyDefined resources.

The compression and the transfer of the dump from clearly-defined server to saam is in progress.

ssh clearly-defined.internal.staging.swh.network "pbzip2 -c -9 /srv/softwareheritage/clearlydefined/clearcode-dump-2021-01-05/clearcode_backup-20210105-2111.dump" | pv | cat > clearcode_backup-20210105-2111.dump.bz2

When it will be done, and if the compression ratio is good, the compressed archive will be added to the public annex in a dataset/clearly-defined directory

Thu, Apr 15, 5:49 PM · System administration
vsellier updated the task description for T3242: Decommission ClearlyDefined resources.
Thu, Apr 15, 5:43 PM · System administration
vsellier added a comment to T3242: Decommission ClearlyDefined resources.

According to https://intranet.softwareheritage.org/wiki/Outboarding:

  • tg19999 unix account disabled

*TG1999 removed from interns members in the forge, no tasks were assigned to the user

  • He was not on the #swh-team channel
  • He was not a member of the swh-team ml
  • VPN certificated revoked
root@louvre:/etc/openvpn/keys# ./easyrsa revoke tg1999
Thu, Apr 15, 5:42 PM · System administration
vsellier removed a member for Interns: TG1999.
Thu, Apr 15, 5:32 PM
vsellier added a revision to T3242: Decommission ClearlyDefined resources: D5539: Outboarding of tg1999.
Thu, Apr 15, 5:26 PM · System administration
vsellier requested review of D5539: Outboarding of tg1999.
Thu, Apr 15, 5:26 PM
vsellier changed the status of T3242: Decommission ClearlyDefined resources from Open to Work in Progress.

thanks, I will move the dump in a secure place where it will be backuped.

Thu, Apr 15, 3:05 PM · System administration
vsellier added a comment to T3243: Replace /dev/sdb and /dev/sdc on storage1.staging.

Email sent to the dsi to launch the replacement.

Thu, Apr 15, 3:03 PM · System administration, Staging environment
vsellier added a comment to T3243: Replace /dev/sdb and /dev/sdc on storage1.staging.

In preparation of the disk replacement, their leds must be activated to make the emplacement identifiable:

  • Ensure all the led are off
root@storage1:~# ls /dev/sd* | grep -e "[a-z]$" | xargs -n1 -t -i{} ledctl normal={} 
ledctl normal=/dev/sda 
ledctl normal=/dev/sdb 
ledctl normal=/dev/sdc 
ledctl normal=/dev/sdd 
ledctl normal=/dev/sde 
ledctl normal=/dev/sdf 
ledctl normal=/dev/sdg 
ledctl normal=/dev/sdh 
ledctl normal=/dev/sdi 
ledctl normal=/dev/sdj 
ledctl normal=/dev/sdk 
ledctl normal=/dev/sdl 
ledctl normal=/dev/sdm 
ledctl normal=/dev/sdn
  • light on
root@storage1:~# ledctl locate=/dev/sdb
root@storage1:~# ledctl locate=/dev/sdc
Thu, Apr 15, 2:31 PM · System administration, Staging environment
vsellier triaged T3251: Count authors from revisions and releases as Normal priority.
Thu, Apr 15, 1:13 PM · Monitoring, Web app
vsellier added a comment to T2912: Next generation archive counters.

I would really like to keep the author counter: how complex is it to add it?

Thu, Apr 15, 1:03 PM · Roadmap 2021, System administration, Monitoring, Web app
vsellier added a comment to T2912: Next generation archive counters.

Staging webapp[1] and webapp1 on production [2] are now configured to use swh-counters to display the historical values and the live object counts.

Thu, Apr 15, 11:40 AM · Roadmap 2021, System administration, Monitoring, Web app
vsellier closed T3250: Deploy the counters in production (step1 webapp1 only), a subtask of T2912: Next generation archive counters, as Resolved.
Thu, Apr 15, 11:32 AM · Roadmap 2021, System administration, Monitoring, Web app
vsellier closed T3250: Deploy the counters in production (step1 webapp1 only) as Resolved.

Deployment done on staging and production. The new counters are currently only activated on webapp1

Thu, Apr 15, 11:32 AM · System administration, Monitoring, Web app
vsellier closed D5531: counters: Configure staging and webapp1 to use swh-counters.
Thu, Apr 15, 11:11 AM
vsellier committed rSPSITE6fbdb75919c1: counters: Configure staging and webapp1 to use swh-counters (authored by vsellier).
counters: Configure staging and webapp1 to use swh-counters
Thu, Apr 15, 11:11 AM
vsellier added inline comments to D5531: counters: Configure staging and webapp1 to use swh-counters.
Thu, Apr 15, 11:06 AM
vsellier updated the test plan for D5531: counters: Configure staging and webapp1 to use swh-counters.
Thu, Apr 15, 10:33 AM
vsellier updated the summary of D5531: counters: Configure staging and webapp1 to use swh-counters.
Thu, Apr 15, 10:30 AM
vsellier updated the diff for D5531: counters: Configure staging and webapp1 to use swh-counters.

fix a typo on the commit description

Thu, Apr 15, 10:29 AM
vsellier requested review of D5531: counters: Configure staging and webapp1 to use swh-counters.
Thu, Apr 15, 10:23 AM
vsellier added a revision to T3250: Deploy the counters in production (step1 webapp1 only): D5531: counters: Configure staging and webapp1 to use swh-counters.
Thu, Apr 15, 10:23 AM · System administration, Monitoring, Web app

Wed, Apr 14

vsellier changed the status of T3250: Deploy the counters in production (step1 webapp1 only) from Open to Work in Progress.
Wed, Apr 14, 7:12 PM · System administration, Monitoring, Web app
vsellier closed T3231: Make the source of the object counts configurable, a subtask of T2912: Next generation archive counters, as Resolved.
Wed, Apr 14, 6:40 PM · Roadmap 2021, System administration, Monitoring, Web app
vsellier closed T3231: Make the source of the object counts configurable as Resolved.
  • version 0.1.296(swh2) released and deployed on all the webapp
Wed, Apr 14, 6:40 PM · System administration, Monitoring, Web app
vsellier accepted D5526: conf/loader: Declare save code now queues to consume from.
Wed, Apr 14, 5:38 PM
vsellier accepted D5486: Declare new service worker to consume save code now queues.

lgtm

Wed, Apr 14, 5:36 PM
vsellier closed D5524: Add the configuration for the source of object counters.
Wed, Apr 14, 4:23 PM
vsellier committed rDWAPPS05d55040f31c: Add the configuration for the source of object counters (authored by vsellier).
Add the configuration for the source of object counters
Wed, Apr 14, 4:23 PM
vsellier updated the diff for D5524: Add the configuration for the source of object counters.

rename the object variable

Wed, Apr 14, 4:13 PM
vsellier requested review of D5524: Add the configuration for the source of object counters.
Wed, Apr 14, 3:12 PM
vsellier updated the task description for T3231: Make the source of the object counts configurable.
Wed, Apr 14, 3:11 PM · System administration, Monitoring, Web app
vsellier added a revision to T3231: Make the source of the object counts configurable: D5524: Add the configuration for the source of object counters.
Wed, Apr 14, 3:05 PM · System administration, Monitoring, Web app
vsellier closed D5518: Make the project usable in the webapp tests.
Wed, Apr 14, 11:18 AM
vsellier committed rDCNT0e5d551ff59b: Make the project usable in the webapp tests (authored by vsellier).
Make the project usable in the webapp tests
Wed, Apr 14, 11:18 AM
vsellier updated the diff for D5518: Make the project usable in the webapp tests.

rebase

Wed, Apr 14, 11:17 AM
vsellier updated the diff for D5518: Make the project usable in the webapp tests.

take review feedbacks in consideration

Wed, Apr 14, 11:16 AM
vsellier requested review of D5518: Make the project usable in the webapp tests.
Wed, Apr 14, 10:19 AM
vsellier added a revision to T3231: Make the source of the object counts configurable: D5518: Make the project usable in the webapp tests.
Wed, Apr 14, 10:18 AM · System administration, Monitoring, Web app

Tue, Apr 13

vsellier closed D5502: Add a method get several counter values at once.
Tue, Apr 13, 7:09 PM
vsellier committed rDCNT38eab384f275: Add a method get several counter values at once (authored by vsellier).
Add a method get several counter values at once
Tue, Apr 13, 7:09 PM
vsellier closed T3232: remove hardcoded historical values from the webapp, a subtask of T2912: Next generation archive counters, as Resolved.
Tue, Apr 13, 6:59 PM · Roadmap 2021, System administration, Monitoring, Web app
vsellier closed T3232: remove hardcoded historical values from the webapp as Resolved.

swh-web v0.0.295 is released and deployed on staging and production.

Tue, Apr 13, 6:59 PM · Monitoring, Web app
vsellier added a comment to T3232: remove hardcoded historical values from the webapp.

The puppet script generating the aggragated data is updated and was run to refresh the data.
The webapp can be released with this diff now

Tue, Apr 13, 6:02 PM · Monitoring, Web app
vsellier removed a task from D5502: Add a method get several counter values at once: T3232: remove hardcoded historical values from the webapp.
Tue, Apr 13, 6:01 PM
vsellier removed a revision from T3232: remove hardcoded historical values from the webapp: D5502: Add a method get several counter values at once.
Tue, Apr 13, 6:01 PM · Monitoring, Web app
vsellier updated the summary of D5502: Add a method get several counter values at once.
Tue, Apr 13, 6:01 PM
vsellier added a revision to T3231: Make the source of the object counts configurable: D5502: Add a method get several counter values at once.
Tue, Apr 13, 6:01 PM · System administration, Monitoring, Web app
vsellier updated the diff for D5502: Add a method get several counter values at once.

Reference the right task

Tue, Apr 13, 6:01 PM
vsellier closed D5494: counters: Aggregate values for origin and revision graphs.
Tue, Apr 13, 5:44 PM
vsellier committed rSPSITE7ec10abc3cb9: counters: Aggregate values for origin and revision graphs (authored by vsellier).
counters: Aggregate values for origin and revision graphs
Tue, Apr 13, 5:44 PM
vsellier requested review of D5502: Add a method get several counter values at once.
Tue, Apr 13, 5:44 PM
vsellier added a revision to T3232: remove hardcoded historical values from the webapp: D5502: Add a method get several counter values at once.
Tue, Apr 13, 5:43 PM · Monitoring, Web app
vsellier added inline comments to D5494: counters: Aggregate values for origin and revision graphs.
Tue, Apr 13, 3:24 PM
vsellier triaged T3242: Decommission ClearlyDefined resources as Normal priority.
Tue, Apr 13, 3:13 PM · System administration
vsellier closed D5490: counters: Move the hardcoded points to the static historical file.
Tue, Apr 13, 3:12 PM
vsellier committed rDWAPPS78a038b13927: counters: Move the hardcoded points to the static historical file (authored by vsellier).
counters: Move the hardcoded points to the static historical file
Tue, Apr 13, 3:12 PM
vsellier added a revision to T3232: remove hardcoded historical values from the webapp: D5494: counters: Aggregate values for origin and revision graphs.
Tue, Apr 13, 3:10 PM · Monitoring, Web app
vsellier requested review of D5494: counters: Aggregate values for origin and revision graphs.
Tue, Apr 13, 3:10 PM
vsellier added a comment to T3232: remove hardcoded historical values from the webapp.

one additional point before releasing this, the puppet script making the aggregation need to be improved as it only merge the data for the content graph :
https://forge.softwareheritage.org/source/puppet-swh-site/browse/production/site-modules/profile/files/stats_exporter/export_archive_counters.py$109

Tue, Apr 13, 12:50 PM · Monitoring, Web app
vsellier changed the status of T3231: Make the source of the object counts configurable, a subtask of T2912: Next generation archive counters, from Open to Work in Progress.
Tue, Apr 13, 12:14 PM · Roadmap 2021, System administration, Monitoring, Web app
vsellier changed the status of T3231: Make the source of the object counts configurable from Open to Work in Progress.
Tue, Apr 13, 12:14 PM · System administration, Monitoring, Web app
vsellier requested review of D5490: counters: Move the hardcoded points to the static historical file.
Tue, Apr 13, 12:14 PM
vsellier added a revision to T3232: remove hardcoded historical values from the webapp: D5490: counters: Move the hardcoded points to the static historical file.
Tue, Apr 13, 12:07 PM · Monitoring, Web app
vsellier added a comment to T3232: remove hardcoded historical values from the webapp.

The P1005 convert the data added by the webapp to json data that can be added to the /usr/local/share/swh-data/history-counters.munin.json file

Tue, Apr 13, 11:21 AM · Monitoring, Web app
vsellier added a comment to P1005 Convert webapp historical data to historical json.

this is the result to add to the current historical data:

{"revision": [[1441065600000, 0], [1467331200000, 594305600], [1473811200000, 644628800], [1479945600000, 704845952], [1494374400000, 780882048], [1506384000000, 853277241], [1516752000000, 943061517], [1518480000000, 946216028], [1521936000000, 980390191], [1538611200000, 1126348335], [1548547200000, 1248389319], [1554681600000, 1293870115], [1561593600000, 1326776432], [1563926400000, 1358421267], [1569110400000, 1379380527], [1569715200000, 1385477933], [1577836800000, 1414420369], [1580947200000, 1428955761], [1586217600000, 1590436149], [1589673600000, 1717420203], [1590537600000, 1744034936]], "origin": [[1441065600000, 0], [1467331200000, 22777052], [1473811200000, 25258776], [1479945600000, 53488904], [1494374400000, 58257484], [1506384000000, 65546644], [1516752000000, 71814787], [1518480000000, 81655813], [1521936000000, 83797945], [1538611200000, 85202432], [1548547200000, 88288721], [1554681600000, 88297714], [1561593600000, 89301694], [1563926400000, 89601149], [1569110400000, 90231104], [1569715200000, 90487661], [1577836800000, 91400586], [1580947200000, 91512130], [1586217600000, 107875943], [1589673600000, 121172621], [1590537600000, 123781438]], "content": [[1441065600000, 0]]}
Tue, Apr 13, 11:17 AM
vsellier created P1005 Convert webapp historical data to historical json.
Tue, Apr 13, 11:17 AM
vsellier changed the status of T3232: remove hardcoded historical values from the webapp, a subtask of T2912: Next generation archive counters, from Open to Work in Progress.
Tue, Apr 13, 9:49 AM · Roadmap 2021, System administration, Monitoring, Web app
vsellier changed the status of T3232: remove hardcoded historical values from the webapp from Open to Work in Progress.
Tue, Apr 13, 9:49 AM · Monitoring, Web app

Mon, Apr 12

vsellier changed the status of T3243: Replace /dev/sdb and /dev/sdc on storage1.staging, a subtask of T3236: staging: Disk error on storage1, from Open to Work in Progress.
Mon, Apr 12, 7:31 PM · System administration, Staging environment
vsellier changed the status of T3243: Replace /dev/sdb and /dev/sdc on storage1.staging from Open to Work in Progress.

The disks are removed from the zfs pool. The replacement be done

Mon, Apr 12, 7:31 PM · System administration, Staging environment
vsellier closed T3236: staging: Disk error on storage1 as Resolved.
Mon, Apr 12, 7:30 PM · System administration, Staging environment
vsellier added a comment to T3236: staging: Disk error on storage1.

The mirror is removed fro the pool:

root@storage1:~# zpool list
NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
data  21.8T  2.50T  19.3T        -         -    20%    11%  1.00x    ONLINE  -
Mon, Apr 12, 7:30 PM · System administration, Staging environment
vsellier closed T3228: Free 4 of 5 remaining ips still used on vlan210 as Resolved.

All the 3 vms are reconfigured. the ips are released.

Mon, Apr 12, 6:11 PM · System administration
vsellier added a comment to T3228: Free 4 of 5 remaining ips still used on vlan210.
  • puppet disabled on all the 3 nodes (pergamon, moma, tate)
  • old /etc/network/interface file backuped
cp /etc/network/interfaces T3228-interfaces
  • configuration changed on promox to swtich eth0 from vlan210 to vlan1300
  • applying puppet configuration
  • restart
  • Former vlan1300 interface removed on proxmox
Mon, Apr 12, 5:55 PM · System administration
vsellier updated the task description for T3228: Free 4 of 5 remaining ips still used on vlan210.
Mon, Apr 12, 5:28 PM · System administration
vsellier updated the task description for T3228: Free 4 of 5 remaining ips still used on vlan210.
Mon, Apr 12, 5:28 PM · System administration
vsellier closed D5479: network: Remove network interface on deprecated VLAN210 network.
Mon, Apr 12, 5:18 PM