I think now we that we have proper error messages in the frontend for repositories that are too big, we can open new tasks for specific Vault failures.

I don't think so, I'll reopen if we see the bug happening again.

Is this still an issue?

I agree we must be careful with not bloating swh.core, but the current subject, it really makes sense to me to put this basic db access wrapper as a core functionality.

In the current state though, for the build to be green again, that'd need another swh-storage tag (and publish to pypi though).

The commit mentioned should fix it.
In the current state though, for the build to be green again, that'd need another swh-storage tag (and publish to pypi though).

Or a fourth solution:

Hypothesis is a test requirements from storage.
The vault uses the storage's fixture (which now requires as test hypothesis but not for everything)
Hypothesis is not used by the vault so that fails here.

build reference: https://jenkins.softwareheritage.org/job/DVAU/job/tests/16/console

Now with a new session I can relaunch a failed directory, but in an old session it keeps the error in the vault table view.

I just had two vault successes:

• d83b7dda887dc790f7207608474650d4344b8df9
• cf31343fd5744d56d7126b6cb99e96d29f5c0376

Now this returned with internal server error: fb13b51abbcfd13de85d9ba8d070a23679576cd7
and this one is done: 4a2875f69e68bb60a9e76414439c073fd2b20732

It would also be possible to add a parameter to force the recooking of an object, which would be more flexible to do a variety of tests on random "known" objects. This might require some thought about whether that would be a public option or not, and if not, how we would restrict its access.

For the new error where download stays at "new" for a long time, here are my 2 tests:
4a2875f69e68bb60a9e76414439c073fd2b20732
fb13b51abbcfd13de85d9ba8d070a23679576cd7

AFAICT the vault sets the number of retries to 0 when scheduling tasks; A least-effort improvement would be to set that to a non-zero number.

Directories I tried and failed:

• b1ed8c3c380b35c0a6546a04e526259d600c7a31
• 8fd378d35eb2c80a24d5a29e66b994ed2a275592

rDSTO329d0f920895f22aa6c34cd317b8ce1674e869af is in fact a work-around for this

[ updated the message above in-place to add 3 and 4 ]

We want to test the end-to-end functionalities of the vault. Various use cases should hence be tested:

Maybe git-bundle(1) is an even better idea. I'm not sure if the format is documented somewhere though.

Adding IRC discussion:

10:14:39  +ardumont | seirl: how are we supposed to run the vault tests?
10:14:58  +ardumont | like i mentioned a while ago, that's not running ok for me
10:15:16  +ardumont | i don't see the swh-vault in the modules to run for the rebuild-testdata routine
10:15:53  +ardumont | (routine in charge of dropping, creating, resetting, initializing, etc... the db and creating the dumps for the tests)
10:17:30  +ardumont | and indeed, i don't see any Makefile in swh-vault/sql/ which could help in plugging to that routine
10:19:06  +ardumont | to be clear, i'd like to align the vault if that's ok
10:19:25  +ardumont | i'd like to be able to run `make test` from $SWH_ENVIRONMENT_HOME (that's what i call it ;)
10:19:45      +zack | ardumont: yep, that remains definitely a goal, thanks for working on this!                                                                                                                  10:20:42      
...

D310 fixed the remaining failures!

with D308 applied:

with a5ab9358de023c4bd0ecab62d0c8ce3b1d1f15fc applied, less errors:

In D306#6243, @seirl wrote:

The response of batch_progress looks like this:

In [51]: c.batch_progress(4)
Out[51]: 
{'bundles': [{'id': 3,
   'obj_id': '7d4aecffc20478ea6807b9649b25b71e22ebbcb6',
   'obj_type': 'revision_gitfast',
   'progress_message': None,
   'status': 'done'},
  {'id': 4,
   'obj_id': '355873d2daf160b736409a359da9e9ca4d714570',
   'obj_type': 'revision_gitfast',
   'progress_message': None,
   'status': 'done'},
  {'id': 5,
   'obj_id': '612dd81b51a443a19e6a2c17f67ef46ea4c2c123',
   'obj_type': 'revision_gitfast',
   'progress_message': None,
   'status': 'done'}],
 'done': 3,
 'failed': 0,
 'new': 0,
 'pending': 0,
 'total': 3}

The response of batch_progress looks like this:

So, that was a beast of a puppet refactoring, but the end result is: all our RPC servers are now nicely tucked behind an instance of nginx.

Here is a pcap of the issue observed while a Vault cooker was calling set_progress on the Vault backend. The BadStatusLine packet that only contains "\r\n" is packet 306.

The BadStatusLine error reared back its ugly head on the vault backend. It was reproducible. Putting a nginx in front of the vault backend fixed it. I guess it's time to puppetize the configuration of a http frontend to all our RPC servers.

Now put_bundle is part of the try/catch block.

Type problem...

Stil happens after rDVAU3c5d89a9c747