@moranegg I've went the script way in the end (what val initially proposed).

This is possible with the main API:

also i just realize i should have mentioned this earlier @KShivendu.
To reproduce the issue, no need for any loader or whatever else,
just ipython in your venv:

Possibly a composition of requests adapter [1] and urllib.request.urlretrieve [2] should
or could be enough instead.

This could possibly be done using https://pypi.org/project/requests-ftp/

Ipol's editor has agreed that this solution will be good for their use case.

I think so, thanks

@vlorentz should we close this one?

@sdruskat Good news about the GitHub support.

Just to say I'm sorry I haven't made any progress here lately. Will get to it once CFF 1.2.0 is out the door. Has become a priority since GitHub and Zenodo now support it :).

closed by https://forge.softwareheritage.org/D5825

In T3157#66041, @jayeshv wrote:

This will simplify branches view implementation. You should create a diff to handle it.

Yes. Let me check it's side effects as well. The 'archive.lookup_snapshot' is called from multiple places.

This will simplify branches view implementation. You should create a diff to handle it.

In T3157#66039, @jayeshv wrote:

@anlambert Thank you. I will try to do some refactoring here.

@anlambert Thank you. I will try to do some refactoring here.

There is no need to use JavaScript here from my point of view.

The ideal fix would be to refactor the code to move the entire presentation logic to the template engine. (We already have a check in the template to show "The list of branches is empty" message in case the branch list is empty. But, i think this case will never occur with the current server code)

More answers after further reading and analysis.

This will eventually cause a 404 error when the user search for a non existing branch name in the UI. I don't think this is the intended behavior.

@anlambert The server code seems to have an issue when branches are filtered by name.
"swh.web.common.archive.lookup_snapshot" function is raising an error if the number of searched branches is 0.
raise
This will eventually cause a 404 error when the user search for a non existing branch name in the UI. I don't think this is the intended behavior.

1. Should we add a filter UI component in the page or just a query param is enough?

2: If we have to add a UI component, do we have a design or example for that? (maybe a generic one for both branches and Releases, the "search branches" widget in github looks nice)

@anlambert I have a few questions regarding this task.
1: Should we add a filter UI component in the page or just a query param is enough?
2: If we have to add a UI component, do we have a design or example for that? (maybe a generic one for both branches and Releases, the "search branches" widget in github looks nice)
3: What about the API /snapshot/<id>, should we add the possibility of this filter there as well?
4: The branches seem to be randomly sorted over Date. Is that for a reason or something we should address?

The 2nd url [2] targetted from val is ok for my part (not the first one [1] though, it goes 404),
here is the content from the [2]nd url inlined just in case:

That's an unrelated error, could you open a task for this?

Hey @vlorentz, the sentry link isn't working (or maybe isn't publically accessible).

Discussed for 2021W18 or 2021W19 during today's call.

Discussed for 2021W18 or 2021W19 during today's call.

If we replaced the whole code with just this:

Do you some more tests or this task can be declared as resolved?

We need more loud warnings of doc build issues (eg. broken references) first.

Is there a reason for not doing it?

In T2265#63605, @anlambert wrote:

In T2265#63475, @zack wrote:

I'm confused about the status of this task. I've just rebuilt the docs for docs.s.o and it says "build succeeded, 83 warnings.". So is the fix for this not yet "deployed" somehow? Also, why is the build succeeding even with all those warnings? Because as long as that's the case, we will for sure keep reintroducing warnings as time goes by.

I'm confused about the status of this task. I've just rebuilt the docs for docs.s.o and it says "build succeeded, 83 warnings.". So is the fix for this not yet "deployed" somehow? Also, why is the build succeeding even with all those warnings? Because as long as that's the case, we will for sure keep reintroducing warnings as time goes by.

\o/ https://jenkins.softwareheritage.org/job/DDOC/job/dev/5528/

Great. In addition to the content of the free form field, the standard answer should contain proper boilerplate reminding what is expected in a Save Code Now request, along the lines of what is written in the "Help" tab of https://archive.softwareheritage.org/save/

Hey @hm, I don't see any typo in https://forge.softwareheritage.org/source/swh-lister/browse/master/docs/run_a_new_lister.rst$50-51. If you do, please make a revision to fix the same.

Sure! My apologies @Hakimb, but it's thank to your work that we have realized what was the right fate for this task.

Right, I suppose we can close the task then?

@seirl, @vlorentz: I see your point, and I agree. We should never have used /nested/paths for this API.
Maybe we should just reconsider this and, one @Hakimb is ready with a new traversal language proposal, we can map it to a better REST API that uses query parameters, and deal properly with 4xx return codes.

@zack We talked about this on IRC with @vlorentz, I think this issue is invalid. We chose to have the source and destination nodes as part of the URI in the API. Semantically, it makes sense that accessing the path without these path fragments would return a 404: it's not a missing argument but an invalid path. If we had a ?src= and a &dst= arguments instead, then having a 400 error would make sense, but in our case the semantics are really weird.

Schema image is now properly displayed: https://docs.softwareheritage.org/devel/swh-storage/sql-storage.html#sql-storage