A priori, except for svn repositories, there are no information about repositories nature in the metadata (.json file) nor in filename of the repositories that are currently being downloaded by worker01.
So we may need an intermediary step to determine that.
Note: Also, those files are currently archives (.zip, .gz) files (so not homogeneous).
| R61 Fetcher Googlecode | |||
| R61:aa01eda8b927 Fetch the project.json per project: | |||
| Status | Assigned | Task | ||
|---|---|---|---|---|
| Unknown Object (Maniphest Task) | ||||
| Migrated | gitlab-migration | T367 ingest Google Code repositories | ||
| Migrated | gitlab-migration | T372 Discriminate repositories nature (hg, svn, git) - code.google.com |
The type of repository can be extracted using the main API of the Google Code Archive. It's something extra that we should do in addition to the file download, but it'd be much better than applying heuristics do the download files (no matter how trivial they would be).
See the repoType field.
In the process, it would be good to also archive the project.json files, as those will be important in the future for us too.