Duplicate of T3533

lgtm

@ardumont @olasd I'm not sure if it was polite / useful to add you as subscribers to this isolated commit. Please let me know if I should have done otherwise.

@ardumont @olasd I'm not sure if it was polite / useful to add you as subscribers to this isolated commit. Please let me know if I should have done otherwise.

I've read https://docs.softwareheritage.org/sysadm/deployment/howto-debian-packaging.html to figure out what should be done to complete the "Bootstrap Debian packaging" part which does not show in the pool https://debian.softwareheritage.org/pool/main/s/ . I think to understand this is done by a job in https://forge.softwareheritage.org/source/swh-jenkins-jobs/, presumably https://forge.softwareheritage.org/source/swh-jenkins-jobs/browse/master/jobs/swh-packages.yaml$196 but... it's not happening

I pushed https://forge.softwareheritage.org/D6700 for inclusion in the documentation and from the output of

For the record, the documentation is disabled for now (see 1 and 2). I'll try to get it to build.

The basic auth is working with the default remote implementation if the credentials are specified in the url like https://user:password@objstorage_url

Let's consider this is done since 2 of 3 bullets have been canceled (for now)

But then Fed4Fire killed the experiment prematurely, ignoring the Grid5000 extension. I'll have another go at it.

When trying to extend the duration of an experiment (slice in the Fed4Fire parlance), an error occurs.

The CLI is actually more complicated because it requires input that are difficult to figure out:

@olasd thanks for adding the dependencies 🎉 D6545

The https://jfed.ilabt.imec.be/downloads/ CLI may be easier to use than the graphical client when repeatin experiments.

It's at https://forge.softwareheritage.org/source/swh-jenkins-dockerfiles/browse/master/base-buster/Dockerfile$39

Contribution to the Grid5000 documentation for Fed4Fire.

Using Export As ansible, I unzipped the result.

Created another experiment (with 25min lifetime only) and it's going better:

Linked Grid5000 & Fed4Fire accounts.

I wanted to terminate the experiment but it looks like it must expire (although Grid5000 has the option to terminate a job).

The Grid5000 machines are found in the "

https://www.grid5000.fr/w/Fed4FIRE is the better documentation to use Grid5000 via Fed4Fire

@olasd I'd like to add dependencies to the CI job running swh-perfecthash (valgrind) so that it can verify the C implementation is clean. Would you be so kind as to point me in the right direction? I looked in https://forge.softwareheritage.org/source/swh-jenkins-jobs but could not find the keyword cmph and concluded it must be in another repository.

There is a monitor that shows all testbeds, among which is grid5000:

Followed the tutorial to run a first experiment which failed https://doc.fed4fire.eu/firstexperiment.html

Followed the tutorial https://doc.fed4fire.eu/tools.html

There is an error on mmap which was not detected, therefore no information on why it failed. This was fixed.

• time tox -e py3 -- --basetemp=/mnt/pytest -s --shard-size $((100 * 1024)) --object-max-size $((4 * 1024)) -k test_build_speed number of objects = 45973118 baseline 163.73826217651367, write_duration 300.58917450904846, build_duration 26.01908826828003, total_duration 326.6082627773285

Running benchmarks directly on grid5000

• oarsub -I -l "{cluster='dahu'}/host=1,walltime=1" -t deploy
• kadeploy3 -f $OAR_NODE_FILE -e debian11-x64-base -k
• ssh root@$(tail -1 $OAR_NODE_FILE)
• mkfs.ext4 /dev/sdb1
• mount /dev/sdb1 /mnt
• apt-get install -y python3-venv libcmph-dev gcc git
• git clone https://git.easter-eggs.org/biceps/swh-perfecthash/
• python3 -m venv bench
• source bench/bin/activate
• pip install -r requirements.txt -r requirements-test.txt
• cd swh-perfecthash
• tox -e py3
• time tox -e py3 -- --basetemp=/mnt/pytest -s --shard-size $((100 * 1024)) --object-max-size $((100 * 1024)) -k test_build_speed
• rm -fr /mnt/pytest

/opt/jFed/jFed-Experimenter works but I'll have to wait on the approval of the account before proceeding further.

Created a project in https://portal.fed4fire.eu/ with the intention of using grid5000. It is pending approval from an administrator (see T3670).

This is a duplicate of T3634

The draft implementation is available publicly at https://git.easter-eggs.org/biceps/swh-perfecthash/-/tree/wip-hash while D6424 is under review.

The implementation of the benchmarks is prepared at:

For the record I created a "draft" repository for contributions to https://forge.softwareheritage.org/source/swh-perfecthash/ at https://git.easter-eggs.org/biceps/swh-perfecthash. It is only meant to be a publicly available sandbox.

[FTR] See https://forge.softwareheritage.org/source/swh-perfecthash/

@olasd these are the failed dependencies you told me to expect, right? The missing package is ... libcmph-dev.