Complete the test (i had forgotten to)

I'm more and more inclined towards using the web client instead.
See D6211

Build is green

• rebase
• directory_entry_add_batch: Reduce churn of prepared statements

In D6139#160624, @vsellier wrote:

After analyze, the batch mode uses huge prepared statements (PS). It seems with the concurrency of the replayers, they are evicted from the PS cache before they can be executed. The client silently reprepare the statement and retry fall in a prepared/evicted loop.

I wonder if we should add a "Pull Request" tab on snapshot views. Seems like a rabbit hole to implement it correctly for all forges though...

According to the documentation of the cassandra concurrent api[1], it seems the concurrency can by specified as an argument of the execute_concurrent_with_args method. The default is 100, but it could be interesting to check with higher or lower values

These are more results with different number of replayers.
Each line represents a server with 20 directory replayers, the renages are for one-by-one, concurrent, batch

• 1 node

• 2 nodes

ACK. These were the ones I was indeed thinking of. All good then :-)

In D6208#160587, @olasd wrote:

I'm not too sure about the flat-out removal of a lot of these logging statements (rather than just turning them into logging statements with deferred formatting), but the changes to the flush() logic at least look sound.

Good idea to add this documentation!
I have added a few comments.

I'm not too sure about the flat-out removal of a lot of these logging statements (rather than just turning them into logging statements with deferred formatting), but the changes to the flush() logic at least look sound.

Build is green

• Add new RabbitMQ-based client/server API
• Split set method's requests into several queues on server side
• Remove get queues and have client read from ProvenanceStorage directly
• Add support for relation_add to the RabbitMQ server
• Refactor server to use multiple sub-processes instead of threads
• Rework ProvenanceStorageRabbitMQWorker to handle connection loss
• Remove old client/server storage based on swh.core.api.RPCClient
• Improve connection error handling on both client and server side

Build is green

rebase

Build is green

Build is green

rebase

Update:

• rebase
• address @vlorentz comments
• fix a couple of spotted issues
• add sphinx documentation
• add modals displayable from the SWHIDs tab explaining how to integrate iframe views in external web page

Build is green

LGTM thanks

The link of a collaborative document has been created for publication of a blog post:

rebase

Build is green

Looks good to me.

Remove unneeded lines

Looks good to me.

Build is green