Note that the PR for repo2docker has been merged, and another PR for binderhub is currently in progress.

add mininal test

In D4923#125398, @douardda wrote:

In D4923#125371, @vlorentz wrote:

(requesting changes to get it out of my review queue)

That's not a valid reason! A valid reason is "I agree with olasd's comments, fix them (plz)"...

typo

In D4923#125371, @vlorentz wrote:

(requesting changes to get it out of my review queue)

rebas

rebase

few requests, please:

• add tests with this commit; every introduced function should have at least one test.
• add doctrings to your new functions,
• improve the commit message (see https://chris.beams.io/posts/git-commit/ ); with the current one, I have no idea what exactly is done in this commit, and more importantly, why this is needed for.

Let's consider it as done.

set the DeprecationWarning category in journal_data

remove stuff added mistakenly, and properly deprecate journal_data instead of breaking swh-storage

Back on this, the plan is now to make swh-journal not depend on the actual model definition, which is currently mostly due to the presence of the journal_data.py in swh-journal. So the plan is to move this file in swh-model so it's kept up to date with swh-model, even if it's mostly used for testing other packages (like swh-journal).

And once again, this "cache" behavior makes the simulator unable to run "forever" (it will eat RAM). Maybe it's an assumed design choice, but please document it somewhere.

Something I don't understand: why do you need to keep both _visit_times and latest_snapshots in "caches" when a snapshot is derived from this visit time (and visit type and origin)?

Isn't there some inherent limitation with this lister_process (gradually eating RAM) that should be documented (maybe)?

Note that I still think there should be something in docs/simulator.rst also...

In D4909#123949, @vlorentz wrote:

We're not claiming this is a realistic model. We only tried to do something that isn't completely naive, and exercises simple edge cases. Making it realistic is hard, and will probably be most of @olasd's work this week.

In D4909#123805, @vlorentz wrote:

Yes, but that's not inconsistent as we can discover origins that we didn't know about.

I'm really not sure to understand what the simulated model looks like in the end. Do I get it right that, including this diff:

rebased

s/-H/-P/

thanks. I think however, given its purpose, this example code should be heavily commented: each constant (eg. MyPageType) and each method should be commented (not docstrings but comments exaplaining what the method/variable is used for).

Not very fond of this "one week => dead" embedded in there, but meh.

ok, but it would have been nice to have an explanation of why this is necessary in the commit message.

In D4920#123571, @vlorentz wrote:

In D4920#123533, @douardda wrote:

why not (cli option), but why (keep it deterministic)?

1. reproducibility, so we can run the simulator twice with different code, and be sure that differences in behavior are not caused by randomness

rebased

rebased

In D4920#123548, @douardda wrote:

In D4920#123533, @douardda wrote:

In D4920#123465, @vlorentz wrote:

I'd like to keep the simulator deterministic. What about adding a CLI option with a seed?

why not (cli option), but why (keep it deterministic)?

Also, a given seed will not be enough here: there is also the maxts = int(utcnow().timestamp()) that will kill the deterministic property...

In D4920#123533, @douardda wrote:

In D4920#123465, @vlorentz wrote:

I'd like to keep the simulator deterministic. What about adding a CLI option with a seed?

why not (cli option), but why (keep it deterministic)?

In D4920#123465, @vlorentz wrote:

I'd like to keep the simulator deterministic. What about adding a CLI option with a seed?

rebased

rebase on D4916

rebased

kill unnedded dependency on D4920

with the commit...

type + vorentz' comment

lgtm

rebased

rebased

In D4891#122658, @ardumont wrote:

This makes me wonder if we shouldn't add an explicit failed status too, while we're at it, for explicit failures that couldn't generate a partial snapshot.

I'd be fine with that indeed.

I think we entertain the idea with @douardda and @vsellier

lgtm

Looks ok to me. I'd like however to have a description of implemented metrics in the commit message (and in the documentation, but this may come later)

rebased