An implementation of this is now available in rDSCH.
- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Advanced Search
Feb 19 2016
Feb 17 2016
It currently breaks on *completely* empty messages, but the patch seems fairly simple.
Feb 11 2016
We did some f2f thinking about this, concentrating on the "origin update" part of the mechanism. The shortcoming of our mechanism is that it's completely specific to updating our origins, and we can do something better...
Feb 9 2016
This is now supported.
Feb 8 2016
No need of making estimates anymore, as we know know.
After GitHub + Debian snapshot + GNU we have the following:
Feb 4 2016
We now have a backup of all the contents that were stored on uffizi at the end of our first batch import.
Jan 29 2016
Jan 28 2016
This schema change is now done in production.
Jan 27 2016
full SQL code with the new schemata for origin_visit, occurrence_history and occurrence. Those three tables are implicitly relevant only for the "Software Heritage" authority.
Jan 26 2016
Reading the dulwich code a bit further, it turns out that git commits can have more header attributes than we initally expected.
Reading the dulwich code a bit further, it turns out that git commits can have more header attributes than we initally expected.
Dulwich seems to handle some of those special cases just fine.
full SQL code with the new schemata for origin_visit, occurrence_history and occurrence. Those three tables are implicitly relevant only for the "Software Heritage" authority.
Roberto Di Cosmo (via mobile/cell)
Le 26 janv. 2016 07:47, "olasd (Nicolas Dandrimont)" <
forge@softwareheritage.org> a écrit :
This has now been deployed in swh.storage v0.0.30: occurrences and releases can now point to arbitrary objects.
Jan 25 2016
Jan 24 2016
Full benchmark data are available at: https://intranet.softwareheritage.org/index.php?title=User:StefanoZacchiroli/Disk_array_benchmark
For completeness, here are the slowdown benchmarks for prado's SSD disks (bottom line: the slow down seems to be present there too, but "only" of the order of 20% or so):
Should be fixed.
Jan 23 2016
Jan 22 2016
The main bottleneck turned out to be seek time, that for 1.6B files really adds up.
By looking at bonnie++ output and doing some math, we have concluded that transfer slowness is essentially dominated by seek time.
This has now been done.
Still thinking about this .