Followed the tutorial to run a first experiment which failed https://doc.fed4fire.eu/firstexperiment.html
- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Feed All Stories
All Stories
All Stories
Oct 21 2021
Oct 21 2021
Followed the tutorial https://doc.fed4fire.eu/tools.html
$ time tox -e py3 -- --basetemp=/mnt/pytest -s --shard-size $((100 * 1024)) --object-max-size $((4 * 1024)) -k test_build_speed number of objects = 45973694, total size = 105903024192 baseline 165.74853587150574, write_duration 495.07564210891724, build_duration 24.210500478744507, total_duration 519.2861425876617
@ardumont I believe all your comments were addressed. It is lucky that you did not review the actual logic: while running benchmarks I ran into a bug that lead to a significant refactor (using fread/fwrite instead of mmap and addresses). Not to the point where you'll have to re-read everything from scratch though 馃槄
Build is green
using mmap turned out to be more involved than expected, use
read/write instead
Oct 20 2021
Oct 20 2021
I have created a new Jenkins pipeline job manually to experiment on setting up that new CI build.
Build has FAILED
Build has FAILED
Build has FAILED
ardumont committed rDDOCc4a1d1dc6dfb: Fix empty page link in sysadm instance (authored by ardumont).
Fix empty page link in sysadm instance
ardumont committed rDDOC0bfb31c9aa69: sysadm: Move keycloak documentation to sysadm instance (authored by ardumont).
sysadm: Move keycloak documentation to sysadm instance
Closing here the tests of the workers.
olasd accepted D6504: converters: Fix detection of tree entries with non-standard commit/tree mode..
Thanks!
Build is green
See https://jenkins.softwareheritage.org/job/DDOC/job/build-on-diff/9/ for more details.
Build is green
anlambert committed rCDFJ7d453b55a904: sphinx/Dockerfile: Install pyarcanist in doc build environment (authored by anlambert).
sphinx/Dockerfile: Install pyarcanist in doc build environment
document the build_objstorage() test helper function
anlambert requested review of D6522: sphinx/Dockerfile: Install pyarcanist in doc build environment.
Build is green
Build is green
remove useless statement
remove mistakenly commited mypy.ini file
Build has FAILED
anlambert committed rDWAPPSa8cfc14b4a5c: admin/deposit: Allow users with permission to list their deposits (authored by anlambert).
admin/deposit: Allow users with permission to list their deposits
ardumont updated the task description for T3675: Investigate why cran origins are marked as non-visited in the scheduler metrics.
ardumont triaged T3675: Investigate why cran origins are marked as non-visited in the scheduler metrics as Normal priority.
lgtm
anlambert requested review of D6518: admin/deposit: Allow users with permission to list their deposits.
This is the impact of configuration the add batch insert size. Tested values were 500, 1000 and 2000.
A full repair of a cassandra node was in progress due to a datalost (disk erased by someone else's experiment after the disk reservation expiration)
Improve a bit the seaweedfs backend
douardda committed rDOBJS55ff4b95d306: Improve tests of the seaweedfs backend (authored by douardda).
Improve tests of the seaweedfs backend
Build is green
Harbormaster failed remote builds in B24583: Diff 23694 for D6492: Add support for pathslicing in seaweedfs backend!
Build has FAILED
rebase
remove a (comment) garbage line
Harbormaster failed remote builds in B24580: Diff 23691 for D6517: Improve tests of the seaweedfs backend!
Build has FAILED
Harbormaster failed remote builds in B24581: Diff 23692 for D6492: Add support for pathslicing in seaweedfs backend!
Build has FAILED
rebase
slight simplification as suggested by vlorentz
Build is green
Harbormaster failed remote builds in B24579: Diff 23690 for D6492: Add support for pathslicing in seaweedfs backend!
Build has FAILED
Build has FAILED
respawn jenkins
respawn jenkins
Rebase
Fix reference to infrastructure
ardumont committed rDDOC6cd2086cd84e: Fix dangling reference in the toc tree (authored by ardumont).
Fix dangling reference in the toc tree
Rebase
ardumont committed rDDOCc9c3cd67060a: Cross-reference properly the infrastructure page in sysadm instance (authored by ardumont).
Cross-reference properly the infrastructure page in sysadm instance
anlambert closed T3257: Template the save code now rejection reason to ease operator workflow, a subtask of T3256: Propose reason for rejecting a save code now, as Resolved.
anlambert closed T3257: Template the save code now rejection reason to ease operator workflow as Resolved.
It has been implemented and deployed to production, closing this.
anlambert closed T3256: Propose reason for rejecting a save code now, a subtask of T3252: Better handling of erroneous origins submitted to save code now, as Resolved.
It has been implemented and deployed to production, closing this.
Harbormaster failed remote builds in B24574: Diff 23685 for D6492: Add support for pathslicing in seaweedfs backend!
Build has FAILED
rebase
It's been done for a while. We can see it appearing in the main archive page under cgit.
Remains 6 origins still ongoing...
ardumont renamed T3674: Determine how to reference manually listed/ingested origins in the scheduler metrics from Somehow reference old manually ingested origins in the scheduler metrics to Determine how to reference manually listed/ingested origins in the scheduler metrics.
After further investigation, I can't find any directory that is in a completely bad order; they are either ordered like git does (by adding a / at the end of dir entries) or by assuming a null byte at the end of dir entries.
ardumont updated the task description for T3674: Determine how to reference manually listed/ingested origins in the scheduler metrics.
ardumont closed T3658: Reference bitbucket mercurial origins, a subtask of T3338: Load the archived bitbucket mercurial repositories, as Resolved.
I've opened T3674 to discuss how to properly reference origins that are not the output of listers.
ardumont triaged T3674: Determine how to reference manually listed/ingested origins in the scheduler metrics as Normal priority.
Harbormaster failed remote builds in B24572: Diff 23683 for D6492: Add support for pathslicing in seaweedfs backend!
Build has FAILED
split the diff in 2
ardumont moved T3658: Reference bitbucket mercurial origins from in-progress to deployed/landed/monitoring on the System administration board.
New webapp version deployed [1], we can see the mercurial origins referenced as a discontinued service there.