Page MenuHomeSoftware Heritage

Create a lister for x.org
Closed, MigratedEdits Locked

Description

We need archive the contents present on https://www.x.org/releases/

The lister has to be one shot because the packages are no longer updated and as mentioned in https://www.x.org/wiki/, there is only one directory which is updated. That is https://www.x.org/releases/individual/, as it is also mentioned

further development work is being done in conjunction with the freedesktop.org

However, the packages that are updated on https://www.x.org/releases/individual/ are also present on the Gitlab page of freedesktop.org (https://gitlab.freedesktop.org/xorg). As Gitlab lister is already present, so we can skip https://www.x.org/releases/individual/
Hence this would be one-shot.

Any API or some index file is not available for this website so the best approach would be the scrape https://www.x.org/releases/ and get the tarball links. These can directly be fed to loader-tar to ingest.

Event Timeline

nahimilega created this task.
nahimilega created this object in space S1 Public.

I don't like the idea of this lister.

Fist, X.org itself is developed in a Git repo hosted on GitLab (the Freedesktop instance). So the easiest way to get its code archived is to just add that GitLab to our list of crawled instances (which will also give us other relevant projects, so let's just do that).

Once we have that, the marginal value of having X.org releases listed on x.org/releases becomes essentially zero.

We will eventually need a lister for "random lists of tarballs linked from a web page", but that is a more general concern, which will probably require more thought than a dedicated X.org lister.

@zack I agree, to the fact that archiving https://www.x.org/releases/individual/ is virtually not required because it is a git repo. However, I was concerned about archiving tarballs of other projects which are only present on https://www.x.org/releases/ like x.org/releases/X11R6.8.0/.
However, as you mention about

a lister for "random lists of tarballs linked from a web page."

.then we do not need a dedicated X.org lister.

nahimilega claimed this task.