Page MenuHomeSoftware Heritage
Feed Advanced Search

Jan 11 2021

anarcat added a comment to T1799: ingest Tor git repositories.

great, thanks! :)

Jan 11 2021, 5:30 PM · Archive coverage
anarcat added a comment to T1799: ingest Tor git repositories.

was this completed? i suspect this issue can be closed if the gitweb is being regularly crawled.

Jan 11 2021, 4:51 PM · Archive coverage
anarcat updated the task description for T2954: ingest gitlab.torproject.org.
Jan 11 2021, 4:51 PM · Archive coverage
anarcat created T2954: ingest gitlab.torproject.org.
Jan 11 2021, 4:50 PM · Archive coverage
anarcat added a comment to T1800: gitweb lister.

i wonder if this can be closed... was this ever implemented? and it's true that our gitweb is really a cgit instance... i wonder if *anyone* still runs gitweb with any significantly large number of repos, considering how much faster cgit is.

Jan 11 2021, 4:47 PM · Lister

Jun 28 2019

anarcat added a comment to D1610: swh.lister.cgit.

sorry folks, the volume is just too high for me to follow here. i'll assume you can ping me if you need help with testing or a final review or something. :) thanks for the hard work!

Jun 28 2019, 4:07 PM

Jun 17 2019

anarcat added a comment to T1659: rewrite the CGit lister as a proper lister.

not that I get a vote in this, but i'd say convert to python. depending on xmllint is very brittle... i already had to tweak the thing once to make it work at all, and the pipeline is kind of nasty. i think you will have to import some HTML parser at some point anyways, so you might as well bite that bullet now.

Jun 17 2019, 7:23 PM · CGit lister

Jun 12 2019

anarcat added a comment to T1799: ingest Tor git repositories.

got it, holding off. i'll let you handle this from here on! keep in mind that tor might switch to gitlab in the future, so might have to redo that process eventually.

Jun 12 2019, 11:17 PM · Archive coverage
anarcat added a comment to T1799: ingest Tor git repositories.

so the clone URLs are not exactly the same as the "gitweb" (AKA cgit) repo, so this requires further hacking... i tried this:

Jun 12 2019, 11:01 PM · Archive coverage
anarcat added a comment to T1659: rewrite the CGit lister as a proper lister.

i couldn't find the time to work through the developer setup and the lister tutorial, so I used the shell script to generate a list of projects for tor gitweb.

Jun 12 2019, 10:20 PM · CGit lister
anarcat renamed T1799: ingest Tor git repositories from ingest Tor gitweb repositories to ingest Tor git repositories.
Jun 12 2019, 9:32 PM · Archive coverage
anarcat edited subtasks for T1798: ingest Tor project source code (meta task), added: T1799: ingest Tor git repositories; removed: T1659: rewrite the CGit lister as a proper lister.
Jun 12 2019, 9:32 PM · Archive coverage
anarcat added a parent task for T1799: ingest Tor git repositories: T1798: ingest Tor project source code (meta task).
Jun 12 2019, 9:32 PM · Archive coverage
anarcat removed a parent task for T1659: rewrite the CGit lister as a proper lister: T1798: ingest Tor project source code (meta task).
Jun 12 2019, 9:32 PM · CGit lister
anarcat edited subtasks for T1799: ingest Tor git repositories, added: T1659: rewrite the CGit lister as a proper lister; removed: T1800: gitweb lister.
Jun 12 2019, 9:31 PM · Archive coverage
anarcat removed a parent task for T1800: gitweb lister: T1799: ingest Tor git repositories.
Jun 12 2019, 9:31 PM · Lister
anarcat added a parent task for T1659: rewrite the CGit lister as a proper lister: T1799: ingest Tor git repositories.
Jun 12 2019, 9:31 PM · CGit lister
anarcat edited subtasks for T1798: ingest Tor project source code (meta task), added: T1659: rewrite the CGit lister as a proper lister; removed: T1799: ingest Tor git repositories.
Jun 12 2019, 9:30 PM · Archive coverage
anarcat removed a parent task for T1799: ingest Tor git repositories: T1798: ingest Tor project source code (meta task).
Jun 12 2019, 9:30 PM · Archive coverage
anarcat added a parent task for T1659: rewrite the CGit lister as a proper lister: T1798: ingest Tor project source code (meta task).
Jun 12 2019, 9:30 PM · CGit lister