- User Since
- Sep 7 2015, 3:25 PM (197 w, 2 d)
Support old unittest.mock
Fallback to fsync if fdatasync isn't available
Rebase; update client_id to match class hierarchy
Tue, Jun 18
(I'll pass on the underlying limitation of being forced to link to a release object from wikidata, which feels a bit artificial but is out of scope for this task)
I don't think that extras_require is going to work.
Mon, Jun 17
After processing the logs of the backfilling process to make sure to redo all the ranges that were interrupted in various database migrations, I'm now confident that this task is complete: we have a full mirror of all contents on Azure, which is kept up to date by the main archive storage backend writing synchronously to it.
FWIW https://grafana.softwareheritage.org/d/jScG7g6mk/objstorage-object-counts?orgId=1 is a (not great) implementation of this.
With the post-hoc moderation of Save Code Now requests, do we really need a captcha? Isn't the base rate limiting enough?
Fri, Jun 14
Thu, Jun 13
Fri, Jun 7
- The main archive currently synchronously writes all contents to Azure as well as the local storage (the gap is strictly closing)
- all partitions from uffizi have been copied to azure and mass-injected (except for partition 8 which only got partially mass injected)
- after this process, it looks like azure is missing 10% of all objects (excluding partition 8), which are all on banco
- I've started a procedure to copy the missing objects from banco directly. Estimated time to completion ~ 1 month
- The same procedure has been started to copy the missing objects from partition 8 on uffizi. Estimated time to completion ~ 15 days
Thu, Jun 6
Tue, Jun 4
Once we have the (released) dists figured out, we can also consider -as a second step- having the packagist lister submit tasks for the upstream repositories mentioned in the source key as well, as an extra data source.
Some part of me wonders if it wouldn't make sense to unify the other way around, as the argument names you picked in the in-memory storage are way more sensible, but we've not succeeded in making a change in this area without breaking some (or all) clients...
Mon, Jun 3
Thanks a lot to @hboutemy for your valuable insights on sources in the Maven central repository, and for the pointer to Reproducible Builds on the JVM.
Tue, May 28
As mentioned in D1516 I don't think DNS provides the proper granularity for this; somerset is a "database replica" server only coincidentally; all its databases that aren't the replica of the main database are actually primaries.
I'm not convinced this is such a good idea; this machine is way more than a "db replica" server (it only has one replica, most its databases are actually primary) and I don't think DNS provides the appropriate granularity level to record this information.
Wed, May 22
Thanks for this change!
Thanks for this first pass at a Maven lister.
May 20 2019
May 16 2019
Obviously needs a core release, but looks good to me.
Please add click in requirements-test.txt :-)
Considering the size of that database, and the fact that we don't have any provisions to automatically spin up a new database server, I think it would make more sense to repatriate it on our main postgres setup, rather than movig it to a new machine on azure.
May 15 2019
Expanding on what Dirk Eddelbuettel posted on IRC when we talked about that, a minimal R script to fetch the current package information would be:
May 14 2019
So that took a few tries in puppet, but adding new brokers to the kafka deployment should now be seamless.
Looks good, thanks!
(looks like Herald can't easily trigger on a comment, because that'd be too easy)
So, according to the fine manual (https://secure.phabricator.com/book/phabricator/article/differential_land/) the "Land Revision" button on the Phabricator user interface depends on setting up:
- Drydock repository automation : https://secure.phabricator.com/book/phabricator/article/drydock_repository_automation/
- Which depends on setting up Drydock working copies : https://secure.phabricator.com/book/phabricator/article/drydock_working_copies/
- Which depends on setting up Drydock hosts : https://secure.phabricator.com/book/phabricator/article/drydock_hosts/
- Which depends on Almanac : https://secure.phabricator.com/book/phabricator/article/almanac/
We're already using LiveDNS for the softwareheritage.org zone, so DNS switchover time should not be an issue (all the more so considering the softwareheritage.org -> www.softwareheritage.org redirect is half-broken already).