In D6424#172094, @dachary wrote:I don't think we should be storing fields defined as size_t on disk, as I believe that's architecture-dependent: I see that you have defined and use ntohq / htonq to, if I understand correctly, make sure that the endianness of the data stored on disk is always in network byte order (although currently we're just always swapping the byte orders, AFAICT). If we end up using an architecture with a 32 (or 128) bit size_t, or even rarer, a big endian architecture, we'll have a bad time.
I think we should:
store explicit uint64_t values on disk, with a specified byte order.Done.
use off_t instead of size_t on the C side, for types that store file offsets (which should be a signed 64 bits value when compiled with -D_FILE_OFFSET_BITS=64, even on 32 bits systems). Unfortunately that means wasting a bit on offset storage, but I think we can live with that.I'm not sure what would be the upside so I kept it as u_int64_t. Do you have a specific scenario in mind where it would create a problem?
- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Feed All Stories
All Stories
All Stories
Nov 9 2021
Nov 9 2021
Build is green
Build is green
Update commit msg
Use empty string as author instead of None, it's actually unclear how to make the model handle None authors with non-None dates
abandonning this, as there isn't a good way to serialize manifests with a date but no author
abandonning this, as there isn't a good way to serialize manifests with a date but no author
douardda committed rDSTO850a7553b6d5: Add support for a redis-based reporting for invalid mirrorred objects (authored by douardda).
Add support for a redis-based reporting for invalid mirrorred objects
douardda committed rDSTO04bd15a0bca8: Refactor fixer.fix_objects() to extract the inner object_fixers dict (authored by douardda).
Refactor fixer.fix_objects() to extract the inner object_fixers dict
Remove now useless fixers
douardda committed rDSTO55eed77b519c: Add a --type option to 'swh storage replay' (authored by douardda).
Add a --type option to 'swh storage replay'
Moving towards event notifications and stream processing instead of polling sounds worthwhile, before the amount of polling becomes more important than managing the event notification mechanism. For the two systems you've mentioned, I think we're really, really far away from that, but it's still worth considering a way to do event-driven notifications properly, so we don't have to rush it through.
Build is green
Build was aborted
swh-public-ci added a comment to D6571: Add support for a redis-based reporting for invalid mirrorred objects.
Build is green
Harbormaster failed remote builds in B24961: Diff 24086 for D6570: Remove now useless revision date checker in fixer!
Build has FAILED
Build is green
makes sense
Can you add a test that the manifest renders properly (with the date) in that case?
douardda updated the diff for D6571: Add support for a redis-based reporting for invalid mirrorred objects.
rebase
rebase
rebase
a couple of remarks inline.
Declare kibana vm
Harbormaster failed remote builds in B24959: Diff 24084 for D6629: Package loader: Uniformize author and message!
Build has FAILED
Build is green
changes suggested by vlorentz
swh-public-ci added a comment to D6527: Update extrinsic-metadata-specification.rst to match the current implementation.
Build is green
stsp committed rDLDCVSd28a4b21c56a: handle Attic-only RCS files over CVS pserver (authored by stsp).
handle Attic-only RCS files over CVS pserver
Build is green
rebased patch
stsp committed rDLDCVSd72f15f24dbd: improve test coverage of file additions and deletions (authored by stsp).
improve test coverage of file additions and deletions
Build is green
rebased patch
vlorentz committed rDSTO0262f1c191cc: Update extrinsic-metadata-specification.rst to match the current implementation (authored by vlorentz).
Update extrinsic-metadata-specification.rst to match the current implementation
stsp committed rDLDCVSca23bc13c21b: display file state in progress logging output (authored by stsp).
display file state in progress logging output
vlorentz updated the diff for D6527: Update extrinsic-metadata-specification.rst to match the current implementation.
rebase
vlorentz committed rDMOD2ffe5dba8401: Give model and swhid objects a nicer repr() (authored by vlorentz).
Give model and swhid objects a nicer repr()
Build is green
rebased patch
stsp committed rDLDCVSf52f0e452132: add support for RCS keyword expansion over pserver protocol (authored by stsp).
add support for RCS keyword expansion over pserver protocol
swh-public-ci added a comment to D6590: add support for RCS keyword expansion over pserver protocol.
Build is green
rebased patch
Looks good
vlorentz requested review of D6625: Refactor package loaders to make the version part of BasePackageInfo.
What's missing, provider_url and domain:
db is almost ready:
As of D6623 the CVS loader is able to convert GNU dino correctly over both rsync and pserver access.
ardumont committed rDLDSVNe36caff0a4f6: SvnLoaderFromRemoteDump: Drop dump when svn repository is mounted (authored by ardumont).
SvnLoaderFromRemoteDump: Drop dump when svn repository is mounted
swh-public-ci added a comment to D6622: SvnLoaderFromRemoteDump: Drop dump when svn repository is mounted.
Build is green
ardumont added inline comments to D6622: SvnLoaderFromRemoteDump: Drop dump when svn repository is mounted.
ardumont updated the diff for D6622: SvnLoaderFromRemoteDump: Drop dump when svn repository is mounted.
Wrap the remove instruction in case failure to remove happens.
ardumont added inline comments to D6622: SvnLoaderFromRemoteDump: Drop dump when svn repository is mounted.
ardumont added inline comments to D6622: SvnLoaderFromRemoteDump: Drop dump when svn repository is mounted.
Build is green
assert self.cvsroot_path instead of casting and squash all changes
swh-public-ci added a comment to D6622: SvnLoaderFromRemoteDump: Drop dump when svn repository is mounted.
Build is green
ardumont updated the diff for D6622: SvnLoaderFromRemoteDump: Drop dump when svn repository is mounted.
Update