Page MenuHomeSoftware Heritage

GSoC 2019Tag
ActivePublic

Watchers

  • This project does not have any watchers.
  • View All

Details

Description

Stuff related to Software Heritage participation into Google Summer of Code program, 2019 edition.

Recent Activity

Tue, Nov 23

borisbaldassari added a comment to T1724: Maven Central repository support.

Hi there,

Tue, Nov 23, 9:21 AM · GSoC 2019, Archive coverage

Mon, Nov 22

ardumont updated the task description for T1724: Maven Central repository support.
Mon, Nov 22, 11:56 AM · GSoC 2019, Archive coverage
ardumont updated the task description for T1724: Maven Central repository support.
Mon, Nov 22, 11:56 AM · GSoC 2019, Archive coverage
ardumont renamed T1724: Maven Central repository support from Maven Central repository Lister to Maven Central repository support.
Mon, Nov 22, 11:54 AM · GSoC 2019, Archive coverage

Oct 2 2021

borisbaldassari added a revision to T1724: Maven Central repository support: D6396: Implement maven jar source files loader.
Oct 2 2021, 9:56 PM · GSoC 2019, Archive coverage
borisbaldassari added a revision to T1724: Maven Central repository support: D6395: lister: Add new maven lister.
Oct 2 2021, 9:21 PM · GSoC 2019, Archive coverage
borisbaldassari added a revision to T1724: Maven Central repository support: D6394: gitlab: Allow listing of instances providing multiple vcs_type [PLEASE DELETE ME].
Oct 2 2021, 9:06 PM · GSoC 2019, Archive coverage
borisbaldassari added a revision to T1724: Maven Central repository support: D6393: maven-lister: initialise lister.maven-lister: update following review on D6133. [PLEASE DELETE ME].
Oct 2 2021, 8:49 PM · GSoC 2019, Archive coverage

Aug 30 2021

borisbaldassari added a revision to T1724: Maven Central repository support: D6159: maven jar-loader: Initalise files, add archive format..
Aug 30 2021, 10:22 AM · GSoC 2019, Archive coverage

Aug 29 2021

borisbaldassari added a revision to T1724: Maven Central repository support: D6158: maven jar-loader: Initalise files..
Aug 29 2021, 10:21 PM · GSoC 2019, Archive coverage

Aug 27 2021

ardumont added a revision to T1724: Maven Central repository support: D6133: maven-lister: initialise lister..
Aug 27 2021, 2:57 PM · GSoC 2019, Archive coverage

Jun 21 2021

borisbaldassari added a comment to T1724: Maven Central repository support.

Updates:

  • A ticket has been submitted in the Sonatype JIRA to let them know we will fetch maven poms and src jars soon.
  • An email has been sent on the maven-dev mailing list with a few kind answers, mainly stating to let Sonatype know through a JIRA issue.
  • Hervé Bouthemy provided some precious insights about the best way to use the poms; it seems we can get a near-complete list of maven repositories worldwide by parsing some pom arguments and following dependencies up. It should probably not be used directly by the lister (which should provide only the list of src jars and scm attributes to the loaders), but we can output it somewhere to feed the lister manually.
Jun 21 2021, 8:59 PM · GSoC 2019, Archive coverage

Jun 9 2021

borisbaldassari added a comment to T1724: Maven Central repository support.

Update for the Maven Indexer prototype: it works! (finally)

Jun 9 2021, 9:03 AM · GSoC 2019, Archive coverage

Jun 8 2021

borisbaldassari added a comment to T1724: Maven Central repository support.

Some more information about the maven indexer. Beware people it's a bit dirty, and you're not going to like it infra-wise.

Jun 8 2021, 9:26 AM · GSoC 2019, Archive coverage
borisbaldassari added a comment to T1724: Maven Central repository support.

So, to sum up the options we have.. Basically we "just" need all artifacts coordinates. From there for each artifact we can:

Jun 8 2021, 8:40 AM · GSoC 2019, Archive coverage

Jun 5 2021

borisbaldassari added a comment to T1724: Maven Central repository support.

Few more cents in the bucket..

  • scrapping is explicitly forbidden, see https://repo1.maven.org/terms.html -- however making contact first will help us go through most of the abuse-limiting rules I guess.
  • regarding fasten, there are indeed some bits that could be useful. However most of our difficulties are in getting a list of projects, whereas this information is already provided by the user in the case of fasten. So, interesting and useful, but not a game changer regarding the difficult part of our job.
Jun 5 2021, 2:51 PM · GSoC 2019, Archive coverage

Mar 17 2021

rdicosmo added a comment to T1724: Maven Central repository support.

After recent exchanges with @hboutemy and Charles Sabourdin, here is a clarification of the scope of this task.
We need a Maven repository lister that addresses the following issues:

Mar 17 2021, 10:40 AM · GSoC 2019, Archive coverage

Mar 11 2021

rdicosmo added a comment to T1724: Maven Central repository support.

@hboutemy : I wonder if you are aware that we have now in place a grant program that allows to fund development of listers like this one.
All the information is available at https://www.softwareheritage.org/grants and you can mail me for more info if needed.

Mar 11 2021, 8:32 PM · GSoC 2019, Archive coverage

Nov 17 2020

anlambert closed T1477: Improve swh-web design for mobile browsing as Resolved.

Implemented during GSOC 2019, closing this.

Nov 17 2020, 5:03 PM · GSoC 2019, Web app
anlambert closed T1768: Add end to end tests for the frontend part of swh-web as Resolved.

Let's close that task when we reach 80% of code coverage

Nov 17 2020, 5:01 PM · GSoC 2019, Web app

Sep 12 2019

rdicosmo added a comment to T1724: Maven Central repository support.

Here is an interesting update on the issue of listing Maven Central. Great people at the FASTEN EU project are analyzing software dependencies and for that they are working on a tool to download projects from various sources, including Maven.
The tool is here: https://github.com/fasten-project/source-populate
It appears to be more about downloading a known project source than listing the content of a repository, but we could try and share efforts in this space.

Sep 12 2019, 5:46 PM · GSoC 2019, Archive coverage

Aug 7 2019

anlambert closed T1909: Enable to manually select the programming language to highlight as Resolved.

Closed by rDWAPPS393deaa5b18c602a27a550836d7e05f049986804

Aug 7 2019, 5:00 PM · GSoC 2019, Web app

Aug 6 2019

anlambert added a parent task for T1768: Add end to end tests for the frontend part of swh-web: T1411: reach a minimum of 80% SLOC coverage across all components.
Aug 6 2019, 11:32 AM · GSoC 2019, Web app
anlambert updated the task description for T1768: Add end to end tests for the frontend part of swh-web.
Aug 6 2019, 11:32 AM · GSoC 2019, Web app

Jul 11 2019

anlambert updated the task description for T1909: Enable to manually select the programming language to highlight.
Jul 11 2019, 11:38 PM · GSoC 2019, Web app
anlambert closed T1900: Improve image and pdf contents rendering as Resolved.

Resolved in a4816b7bb313

Jul 11 2019, 11:06 AM · GSoC 2019, Web app

Jul 10 2019

anlambert assigned T1909: Enable to manually select the programming language to highlight to kalpitk.
Jul 10 2019, 5:27 PM · GSoC 2019, Web app

Jul 9 2019

anlambert assigned T1900: Improve image and pdf contents rendering to kalpitk.
Jul 9 2019, 11:38 AM · GSoC 2019, Web app

Jun 12 2019

nahimilega closed T1709: implement an R-cran lister as Resolved by committing rDLSa9a37a85bf9e: swh.lister.cran.
Jun 12 2019, 4:05 PM · GSoC 2019, Archive coverage
kalpitk closed D1568: Add End to end tests: Code Highlighting.
Jun 12 2019, 12:23 PM · GSoC 2019
vlorentz accepted D1568: Add End to end tests: Code Highlighting.
Jun 12 2019, 11:03 AM · GSoC 2019
kalpitk updated subscribers of T1768: Add end to end tests for the frontend part of swh-web.
Jun 12 2019, 9:28 AM · GSoC 2019, Web app
kalpitk added a comment to T1768: Add end to end tests for the frontend part of swh-web.

I plan to add the following e2e tests-

  1. Test basic webapp functionalities like 'sidebar', 'back-to-top'
  2. Test home page displays positive stats for directories, authors,...
  3. Test the origin-search with different combinations of checkboxes
  4. Test basic functionality of directory view.
  5. Test file being displayed (for some known format, maybe .txt)
  6. Test error being displayed when invalid sha1 or unknown origin url
Jun 12 2019, 9:20 AM · GSoC 2019, Web app

Jun 11 2019

swh-public-ci added a comment to D1568: Add End to end tests: Code Highlighting.

Build is green
See https://jenkins.softwareheritage.org/job/DWAPPS/job/tox/486/ for more details.

Jun 11 2019, 7:31 PM · GSoC 2019
kalpitk updated the summary of D1568: Add End to end tests: Code Highlighting.
Jun 11 2019, 7:29 PM · GSoC 2019
kalpitk added a project to D1568: Add End to end tests: Code Highlighting: GSoC 2019.
Jun 11 2019, 7:28 PM · GSoC 2019

Jun 8 2019

anlambert added a project to T1768: Add end to end tests for the frontend part of swh-web: GSoC 2019.
Jun 8 2019, 2:34 AM · GSoC 2019, Web app

Jun 7 2019

anlambert closed T1786: Improve popovers display on small screens, a subtask of T1477: Improve swh-web design for mobile browsing , as Resolved.
Jun 7 2019, 6:30 PM · GSoC 2019, Web app
anlambert closed T1786: Improve popovers display on small screens as Resolved.

Closed by d355d8973205

Jun 7 2019, 6:30 PM · GSoC 2019, Web app
anlambert merged task T1770: Add end to end tests into T1768: Add end to end tests for the frontend part of swh-web.
Jun 7 2019, 6:11 PM · GSoC 2019, Web app

Jun 5 2019

anlambert renamed T1770: Add end to end tests from Add end to end tests using Selenium to Add end to end tests .
Jun 5 2019, 11:38 AM · GSoC 2019, Web app
anlambert updated the task description for T1786: Improve popovers display on small screens.
Jun 5 2019, 11:31 AM · GSoC 2019, Web app
anlambert triaged T1786: Improve popovers display on small screens as Normal priority.
Jun 5 2019, 11:30 AM · GSoC 2019, Web app

Jun 4 2019

hboutemy added a comment to T1724: Maven Central repository support.

I think there's value in making the Maven lister support more Maven repositories than just Maven Central, even if we focus on Maven Central as the first proof of concept.

makes sense. IMHO, it's good to have one issue per repository, since every repository will have its specific topics

Jun 4 2019, 8:00 AM · GSoC 2019, Archive coverage

Jun 3 2019

olasd added a comment to T1724: Maven Central repository support.

Thanks a lot to @hboutemy for your valuable insights on sources in the Maven central repository, and for the pointer to Reproducible Builds on the JVM.

Jun 3 2019, 4:18 PM · GSoC 2019, Archive coverage

Jun 2 2019

hboutemy added a comment to T1724: Maven Central repository support.

if you search for source-release or src, yes, you'll find archivable sources
the only issue is that you'll not find many content...

Jun 2 2019, 11:08 PM · GSoC 2019, Archive coverage
nahimilega added a comment to T1724: Maven Central repository support.

I renamed the issue title to "Maven Central repository Lister" if the intent is to focus on this repository https://maven.apache.org/repository/index.html

Jun 2 2019, 2:39 PM · GSoC 2019, Archive coverage
hboutemy added a comment to T1724: Maven Central repository support.

I renamed the issue title to "Maven Central repository Lister" if the intent is to focus on this repository https://maven.apache.org/repository/index.html

Jun 2 2019, 12:21 PM · GSoC 2019, Archive coverage
hboutemy renamed T1724: Maven Central repository support from Maven Lister to Maven Central repository Lister.
Jun 2 2019, 12:10 PM · GSoC 2019, Archive coverage

May 31 2019

nahimilega added a comment to T1724: Maven Central repository support.

Creating Your Own Mirror
The size of the central repository is increasing steadily To save us bandwidth and you time, mirroring the entire central repository is >not allowed. (Doing so will get you automatically banned) Instead, we suggest you setup a repository manager as a proxy.

May 31 2019, 7:21 PM · GSoC 2019, Archive coverage