Page MenuHomeSoftware Heritage

"save code now"
Closed, MigratedEdits Locked

Description

The web app should grow a submission interface to allow people to request archiving of a specific origin.

As a first approximation a form similar to Internet's Archive "save page" one could be added. Differently from IA, however, we will not save the page "now", but rather "ASAP", adding the relevant job to our update queue.

Right now we can easily support this for Git (even though the question about the job priority is open…), but ideally this should be something generic, with either auto-detection of the right kind of job, or the ability for the user to specify the kind of origin (git, svn, tar, etc).

Related Objects

Event Timeline

olasd changed the visibility from "All Users" to "Public (No Login Required)".May 13 2016, 5:09 PM
rdicosmo added a parent task: Unknown Object (Maniphest Task).May 30 2016, 9:50 AM
jbertran lowered the priority of this task from Normal to Low.Aug 17 2016, 3:12 PM
zack renamed this task from "save origin now" form to "save code now".Jun 27 2018, 8:11 AM
zack raised the priority of this task from Low to High.
zack edited projects, added General; removed Web app.

I've generalized the title of this task, will add sub-tasks for the specific features that are still missing to complete this.

anlambert changed the status of subtask T1121: save code now API entry point from Open to Work in Progress.Jul 3 2018, 11:16 AM
anlambert changed the status of subtask T1120: save code now moderation UI from Open to Work in Progress.Jul 9 2018, 10:53 AM

Does adding a supported forge (e.g gitlab instance) considered a possible save now request?

That would be great.

Does adding a supported forge (e.g gitlab instance) considered a possible save now request?

Yeah, that would be cool. Nice idea.

We certainly want to allow users to suggest saving instances of forges we already support (e.g., GitLab). And reusing the same interface and moderation process for save origin now totally makes sense.

I'm not sure how easy it would be, because the underlying backends are quite different. E.g., you don't "schedule" the addition of an entire forge as a single task, so the monitoring would be different. But there are certainly some pieces, at least at the UI level, that we might want to reuse.

@anlambert maybe think about this additional use case and how it might fit in, but let's not block the save origin now work due to this.

E.g., you don't "schedule" the addition of an entire forge as a single task,

Yes, there are 2 tasks for now (incremental, full) but if we also hide that detail within T1157... Then that could be a win, i think ;)

E.g., you don't "schedule" the addition of an entire forge as a single task,

Yes, there are 2 tasks for now (incremental, full) but if we also hide that detail within T1157... Then that could be a win, i think ;)

But that's listing, not loading. It's not clear to me that a user that added a forge would be interested in knowing when we're done adding its origins, that's just an implementation detail. The user will want to know when we have archived all of it at least once, which is complicated to define. It might be enough to give visibility to when the listing is done, but it'll certainly require a different user-facing explanation than saving an origin.

But that's listing, not loading. It's not clear to me that a user that added a forge would be interested in knowing when we're done adding its origins, that's just an implementation detail. The user will want to know when we have archived all of it at least once, which is complicated to define. It might be enough to give visibility to when the listing is done, but it'll certainly require a different user-facing explanation than saving an origin.

If we implement this today, we should be clear from the start that we can answer the user only the 'done listing?' question (for now).

And then iterate to improve on this.
Ultimately, implementing the missing tool (cross-referencing lister's db, scheduler, archive) which will be able to answer the real important one ('done ingesting?').

require a different user-facing explanation than saving an origin.

Nitpicking here, it's not just saving origins, it's also the scheduling of each origin ingestion (without this, it's pointless indeed).

Cheers,

anlambert changed the status of subtask T1119: save code now submission form from Open to Work in Progress.Jul 24 2018, 11:14 AM

this is now done (YAY :-)), closing the task to reflect current status

zack claimed this task.
gitlab-migration changed the status of subtask T1120: save code now moderation UI from Resolved to Migrated.
gitlab-migration changed the status of subtask T1121: save code now API entry point from Resolved to Migrated.