Page MenuHomeSoftware Heritage
Feed Advanced Search

Jan 21 2016

olasd updated the task description for T267: Migrate databases to postgres 9.5.
Jan 21 2016, 12:08 PM

Jan 18 2016

olasd updated the task description for T267: Migrate databases to postgres 9.5.
Jan 18 2016, 10:58 PM
olasd updated the task description for T267: Migrate databases to postgres 9.5.
Jan 18 2016, 6:17 PM
olasd changed the status of T267: Migrate databases to postgres 9.5 from Open to Work in Progress.
Jan 18 2016, 5:59 PM
olasd changed the status of T267: Migrate databases to postgres 9.5, a subtask of T262: Upgrade postgresql to 9.5, from Open to Work in Progress.
Jan 18 2016, 5:59 PM
olasd created T267: Migrate databases to postgres 9.5.
Jan 18 2016, 5:59 PM
ardumont added a comment to T266: Setup pgbouncer on prado.

With your changes regarding secure connection:

Jan 18 2016, 4:59 PM
olasd added a comment to T266: Setup pgbouncer on prado.

Added opportunistic TLS for client and server connections:

Jan 18 2016, 4:57 PM
ardumont added a comment to T266: Setup pgbouncer on prado.

Configuration changed in ~/.pg_service.conf and ~/.pg_pass (for the port)

Jan 18 2016, 4:57 PM
olasd changed the status of T266: Setup pgbouncer on prado, a subtask of T262: Upgrade postgresql to 9.5, from Open to Work in Progress.
Jan 18 2016, 4:37 PM
olasd changed the status of T266: Setup pgbouncer on prado from Open to Work in Progress.

pgbouncer has been setup on prado (port 6432) and should proxy all connections to postgres 9.4.

Jan 18 2016, 4:37 PM
olasd created T266: Setup pgbouncer on prado.
Jan 18 2016, 4:37 PM
zack changed the status of T7: backup: object storage — 2nd copy after first large batch import from Open to Work in Progress.
Jan 18 2016, 4:23 PM
zack added a comment to T7: backup: object storage — 2nd copy after first large batch import.

Now that the first batch import (github + snapshot.debian.org + gnu.org) is done and we won't be importing other sources for a while, a full object store backup from uffizi to banco has now started.

Jan 18 2016, 4:22 PM
zack claimed T7: backup: object storage — 2nd copy after first large batch import.
Jan 18 2016, 3:25 PM
zack renamed T7: backup: object storage — 2nd copy after first large batch import from backup: object storage (AKA file content) to backup: object storage — 2nd copy after first large batch import.
Jan 18 2016, 3:25 PM
olasd closed T265: Increase storage space on uffizi as Resolved.
louvre# for i in `seq 0 9` a b c d e f; do lvextend -L +5T vg-data/$i; done
Jan 18 2016, 3:18 PM
olasd created T265: Increase storage space on uffizi.
Jan 18 2016, 3:12 PM

Jan 12 2016

zack added a comment to T262: Upgrade postgresql to 9.5.

I'd be in favor of working with three clusters then:

Jan 12 2016, 4:22 PM
olasd added a comment to T262: Upgrade postgresql to 9.5.

That sounds sensible. We should be able to export a LV from our spinning rust storage to prado for that "mirrors" postgresql cluster.

Jan 12 2016, 4:18 PM
zack added a comment to T262: Upgrade postgresql to 9.5.

Agreed, pgbouncer would be nice (and IIRC also the choice that Dimitri recommended to us a while back).

Jan 12 2016, 3:45 PM
olasd added a comment to T262: Upgrade postgresql to 9.5.

The latest pgbouncer release (1.7) supports TLS, hba and unix peer authentification so looks like a good candidate. I submitted a bug to Debian to ask for an update of the package : https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=810816

Jan 12 2016, 3:30 PM
olasd created T262: Upgrade postgresql to 9.5.
Jan 12 2016, 2:39 PM

Dec 17 2015

zack added a comment to T237: test postgres db restore.
Dec 17 2015, 12:25 PM

Dec 15 2015

zack closed T237: test postgres db restore as Resolved.

This is now done. It worked well, but it's really slow.

Dec 15 2015, 10:03 AM

Dec 14 2015

olasd closed T229: louvre: restart the swhdata{1,2,3} multipath devices to pick up config changes as Resolved.

Done during the 2015-12 reboot

Dec 14 2015, 4:07 PM

Dec 12 2015

zack added a comment to T238: barman@banco:~$ ssh postgres@db-public: connection refused on port 10022.
In T238#2777, @olasd wrote:

Added DNAT configuration to louvre:/etc/network/interfaces.

Dec 12 2015, 5:53 PM
olasd added a comment to T237: test postgres db restore.
Starting local restore for server swh using backup 20151204T074046
Destination directory: /srv/storage/0/barman-backup-restore-test-T237/
Copying the base backup.
Copying required WAL segments.
Generating archive status files
Identify dangerous settings in destination directory.
Dec 12 2015, 4:25 PM
olasd closed T238: barman@banco:~$ ssh postgres@db-public: connection refused on port 10022 as Resolved.

Added DNAT configuration to louvre:/etc/network/interfaces.

Dec 12 2015, 2:07 PM
zack created T238: barman@banco:~$ ssh postgres@db-public: connection refused on port 10022.
Dec 12 2015, 10:56 AM

Dec 10 2015

zack changed the status of T237: test postgres db restore from Open to Work in Progress.
Dec 10 2015, 12:12 PM
zack claimed T237: test postgres db restore.

this is now ongoing on banco, in a screen session of user barman

Dec 10 2015, 12:11 PM

Dec 9 2015

olasd closed T231: banco: prepare object storage as Resolved.
  • foreach $i in 0..f
    • Created 10TB logical volume vg-swhbackup/backup$i
    • Created XFS filesystem on /dev/mapper/vg--swhbackup-backup$i
    • Created directory /srv/storage/space/$i
    • Mounted /dev/mapper/vg--swhbackup-backup$i on /srv/storage/space/$i
    • foreach $j in 0..f
      • Created directories /srv/storage/space/$i/$i$j and /srv/softwareheritage/objects/$i$j
      • Mounted directory /srv/storage/space/$i/$i$j on /srv/softwareheritage/objects/$i$j
Dec 9 2015, 8:09 PM
olasd closed T232: banco: prepare barman storage as Resolved.

Done (lv name: vg-swhbackup/barman, size = 20TB)

Dec 9 2015, 7:07 PM
olasd closed T230: banco: make the raid device + lvm volume group on top of the three logical volumes, a subtask of T231: banco: prepare object storage, as Resolved.
Dec 9 2015, 7:06 PM
olasd closed T230: banco: make the raid device + lvm volume group on top of the three logical volumes as Resolved.

Done (vg name: vg-swhbackup).

Dec 9 2015, 7:06 PM
olasd closed T230: banco: make the raid device + lvm volume group on top of the three logical volumes, a subtask of T232: banco: prepare barman storage, as Resolved.
Dec 9 2015, 7:06 PM
olasd closed T228: banco: restart the swhbackup1 multipath device to pick up multipath config changes as Resolved.

Done.

Dec 9 2015, 7:06 PM
olasd closed T228: banco: restart the swhbackup1 multipath device to pick up multipath config changes, a subtask of T230: banco: make the raid device + lvm volume group on top of the three logical volumes, as Resolved.
Dec 9 2015, 7:06 PM
olasd closed T11: cleanup network setup for louvre as Resolved.

Turns out the /etc/network/interfaces syntax for the bond interface was wrong T_T. Fixed.

Dec 9 2015, 3:34 PM

Dec 7 2015

zack created T237: test postgres db restore.
Dec 7 2015, 10:54 PM

Dec 3 2015

olasd added a subtask for T232: banco: prepare barman storage: T230: banco: make the raid device + lvm volume group on top of the three logical volumes.
Dec 3 2015, 2:18 PM
olasd added a parent task for T230: banco: make the raid device + lvm volume group on top of the three logical volumes: T232: banco: prepare barman storage.
Dec 3 2015, 2:18 PM
olasd created T232: banco: prepare barman storage.
Dec 3 2015, 2:18 PM
olasd added a subtask for T231: banco: prepare object storage: T230: banco: make the raid device + lvm volume group on top of the three logical volumes.
Dec 3 2015, 2:17 PM
olasd added a parent task for T230: banco: make the raid device + lvm volume group on top of the three logical volumes: T231: banco: prepare object storage.
Dec 3 2015, 2:17 PM
olasd created T231: banco: prepare object storage.
Dec 3 2015, 2:17 PM
olasd added a subtask for T230: banco: make the raid device + lvm volume group on top of the three logical volumes: T228: banco: restart the swhbackup1 multipath device to pick up multipath config changes.
Dec 3 2015, 2:16 PM
olasd added a parent task for T228: banco: restart the swhbackup1 multipath device to pick up multipath config changes: T230: banco: make the raid device + lvm volume group on top of the three logical volumes.
Dec 3 2015, 2:16 PM
olasd created T230: banco: make the raid device + lvm volume group on top of the three logical volumes.
Dec 3 2015, 2:15 PM
olasd created T229: louvre: restart the swhdata{1,2,3} multipath devices to pick up config changes.
Dec 3 2015, 2:10 PM
olasd created T228: banco: restart the swhbackup1 multipath device to pick up multipath config changes.
Dec 3 2015, 2:09 PM

Nov 5 2015

zack created T189: centralized syslog.
Nov 5 2015, 3:56 PM · General

Oct 29 2015

olasd closed T167: anacron logrotate failure on prado: /var/log/syslog.1.xz: File exists as Resolved.

Actually, I ran logrotate -f /etc/logrotate.conf and it ran okay.

Oct 29 2015, 8:06 AM
olasd claimed T167: anacron logrotate failure on prado: /var/log/syslog.1.xz: File exists.

I removed the culprit file (which was empty). I suppose it's a side effect of the out of disk issue of last week.

Oct 29 2015, 8:05 AM
zack created T167: anacron logrotate failure on prado: /var/log/syslog.1.xz: File exists.
Oct 29 2015, 7:59 AM

Oct 21 2015

olasd closed T113: deploy SSL cert for intranet.softwareheritage.org as Resolved.

Deployed via puppet during the migration of T18.

Oct 21 2015, 12:24 PM
olasd closed T18: migrate intranet.softwareheritage.org (mediawiki) to SWH infrastructure as Resolved.

Deployed via puppet and DNS updated.

Oct 21 2015, 12:24 PM
olasd closed T18: migrate intranet.softwareheritage.org (mediawiki) to SWH infrastructure, a subtask of T113: deploy SSL cert for intranet.softwareheritage.org, as Resolved.
Oct 21 2015, 12:24 PM

Oct 20 2015

zack closed T114: provide data dumps and conffile of the current intranet.softwareheritage.org deployment as Resolved.

this has been scripted on the current hosting site, dumps can be generated whenever we want them to be available

Oct 20 2015, 3:59 PM
zack closed T114: provide data dumps and conffile of the current intranet.softwareheritage.org deployment, a subtask of T18: migrate intranet.softwareheritage.org (mediawiki) to SWH infrastructure, as Resolved.
Oct 20 2015, 3:59 PM
zack created T114: provide data dumps and conffile of the current intranet.softwareheritage.org deployment.
Oct 20 2015, 11:15 AM
zack added a subtask for T113: deploy SSL cert for intranet.softwareheritage.org: T18: migrate intranet.softwareheritage.org (mediawiki) to SWH infrastructure.
Oct 20 2015, 11:15 AM
zack added a parent task for T18: migrate intranet.softwareheritage.org (mediawiki) to SWH infrastructure: T113: deploy SSL cert for intranet.softwareheritage.org.
Oct 20 2015, 11:15 AM
zack created T113: deploy SSL cert for intranet.softwareheritage.org.
Oct 20 2015, 11:11 AM

Oct 19 2015

olasd closed T4: SSL certificate for *.softwareheritage.org as Resolved.

Signed certificates and private keys for *.softwareheritage.org and *.internal.softwareheritage.org are now available in rPWD.

Oct 19 2015, 3:14 PM · Restricted Project

Oct 16 2015

zack reassigned T4: SSL certificate for *.softwareheritage.org from zack to olasd.

Status update: Nicolas has submitted the CSR(s), waiting for DigiCert to ack.

Oct 16 2015, 12:16 PM · Restricted Project

Oct 15 2015

olasd added a member for System administrators: ardumont.
Oct 15 2015, 1:52 PM
zack lowered the priority of T4: SSL certificate for *.softwareheritage.org from Normal to Low.
Oct 15 2015, 10:34 AM · Restricted Project

Oct 14 2015

zack added a project to T91: puppetize backup of the public SWH website: Language-Puppet.
Oct 14 2015, 4:23 PM · Puppet recipes, Language-Puppet
zack created T91: puppetize backup of the public SWH website.
Oct 14 2015, 4:23 PM · Puppet recipes, Language-Puppet
zack closed T86: backup the swh public website off-gandi as Resolved.
Oct 14 2015, 4:22 PM
zack added a comment to T86: backup the swh public website off-gandi.

This is now done. Some details:

Oct 14 2015, 4:22 PM
olasd closed T60: deploy webapp at http://base.softwareheritage.org as Resolved.

And apache through to rSPPROFa11a325c.

Oct 14 2015, 12:10 PM · Web app
olasd added a comment to T60: deploy webapp at http://base.softwareheritage.org.

uWSGI has been deployed through rSPPROFb76ea39d4

Oct 14 2015, 10:44 AM · Web app

Oct 13 2015

olasd claimed T60: deploy webapp at http://base.softwareheritage.org.
Oct 13 2015, 3:49 PM · Web app

Oct 12 2015

zack created T86: backup the swh public website off-gandi.
Oct 12 2015, 3:49 PM
olasd closed T69: backup: log dar output to file as Resolved by committing rSPDARea479fcd9dd9: dar::backup: add logfile and silence output.
Oct 12 2015, 10:44 AM
olasd added a comment to T15: add systemd service file for forge IRC bot.

The systemd service file has been added, but it doesn't go well with the other phabricator daemons that are running...

Oct 12 2015, 10:20 AM

Oct 9 2015

olasd claimed T57: git commit IRC notifications on #swh-devel.
Oct 9 2015, 3:19 PM · General
olasd added a comment to T57: git commit IRC notifications on #swh-devel.
Oct 9 2015, 3:15 PM · General
olasd added a comment to T57: git commit IRC notifications on #swh-devel.

Added another setting "notification.types" : ["task", "cmit"]

Oct 9 2015, 3:13 PM · General
olasd added a comment to T57: git commit IRC notifications on #swh-devel.

Added a setting "notifications.verbosity" in the bot config file. The default disabled notifications...

Oct 9 2015, 3:06 PM · General
olasd added a comment to T13: setup inbound email for forge.

We can configure "application emails" to route inbound email to phabricator applications. We can then use Herald rules to route those addresses properly.

Oct 9 2015, 2:40 PM
olasd closed T13: setup inbound email for forge as Resolved.

Pending DNS propagation, this should be done.

Oct 9 2015, 2:15 PM
olasd closed T84: Add parameters for relaying to puppet postfix as Resolved.

done in rSPPFIX840582c1d52c and rSPPFIXf97729a999d3

Oct 9 2015, 12:39 PM
olasd closed T84: Add parameters for relaying to puppet postfix, a subtask of T13: setup inbound email for forge, as Resolved.
Oct 9 2015, 12:39 PM
olasd created T84: Add parameters for relaying to puppet postfix.
Oct 9 2015, 11:47 AM
zack closed T83: open inbound port 25 to forge.softwareheritage.org, a subtask of T13: setup inbound email for forge, as Invalid.
Oct 9 2015, 11:03 AM
zack closed T83: open inbound port 25 to forge.softwareheritage.org as Invalid.
Oct 9 2015, 11:03 AM
zack created T83: open inbound port 25 to forge.softwareheritage.org.
Oct 9 2015, 10:59 AM
ardumont added a comment to T60: deploy webapp at http://base.softwareheritage.org.

swh's puppet state of art - https://forge.softwareheritage.org/diffusion/SENV/

Oct 9 2015, 9:49 AM · Web app

Oct 7 2015

zack removed a project from T77: continuous integration to run test suite and static code checks: Restricted Project.
Oct 7 2015, 9:41 AM · Phabricator
zack created T77: continuous integration to run test suite and static code checks.
Oct 7 2015, 9:25 AM · Phabricator

Oct 6 2015

zack created T69: backup: log dar output to file.
Oct 6 2015, 9:54 AM

Oct 5 2015

olasd closed T2: mail forwarding on all VMs as Resolved by committing rSPPROFd2553fd7edf5: profile::base: add mail aliases.
Oct 5 2015, 12:30 PM
olasd closed T63: Fix backups failing when a file changes as Resolved by committing rSPDARd1b5b9618f3b: swh-dar-backup: allow dar to backup "moving" files.
Oct 5 2015, 11:55 AM
zack closed T6: backup: postgres DB as Resolved.
Oct 5 2015, 9:38 AM

Oct 4 2015

olasd created T63: Fix backups failing when a file changes.
Oct 4 2015, 2:23 PM

Oct 2 2015

zack added a project to T60: deploy webapp at http://base.softwareheritage.org: Web app.
Oct 2 2015, 10:58 PM · Web app