diff --git a/talks-public/2020-09-18-EsLibre/2020-09-18-EsLibre.org b/talks-public/2020-09-18-EsLibre/2020-09-18-EsLibre.org index af1218f..5170e25 100644 --- a/talks-public/2020-09-18-EsLibre/2020-09-18-EsLibre.org +++ b/talks-public/2020-09-18-EsLibre/2020-09-18-EsLibre.org @@ -1,446 +1,425 @@ #+COLUMNS: %40ITEM %10BEAMER_env(Env) %9BEAMER_envargs(Env Args) %10BEAMER_act(Act) %4BEAMER_col(Col) %10BEAMER_extra(Extra) %8BEAMER_opt(Opt) #+TITLE: Software Heritage: a common infrastructure to preserve our Software Commons # #+AUTHOR: Roberto Di Cosmo # #+EMAIL: roberto@dicosmo.org @rdicosmo @swheritage #+BEAMER_HEADER: \date[18/09/2020]{18/09/2020, \emph{EsLibre}} #+BEAMER_HEADER: \title[(CC-BY 4.0) Preserving the Software Commons]{Software Heritage: a common infrastructure to preserve our Software Commons} #+BEAMER_HEADER: \author[Roberto Di Cosmo]{Roberto Di Cosmo\\Inria and Universit\'e de Paris} # #+BEAMER_HEADER: \setbeameroption{show notes on second screen} #+BEAMER_HEADER: \setbeameroption{hide notes} #+KEYWORDS: software heritage legacy preservation knowledge mankind technology #+LATEX_HEADER: \usepackage{tcolorbox} #+LATEX_HEADER: \definecolor{links}{HTML}{2A1B81} #+LATEX_HEADER: \hypersetup{colorlinks,linkcolor=,urlcolor=links} # # prelude.org contains all the information needed to export the main beamer latex source # use prelude-toc.org to get the table of contents # #+INCLUDE: "../../common/modules/prelude-toc.org" :minlevel 1 #+INCLUDE: "../../common/modules/169.org" # +LaTeX_CLASS_OPTIONS: [aspectratio=169,handout,xcolor=table] #+LATEX_HEADER: \usepackage{bbding} #+LATEX_HEADER: \DeclareUnicodeCharacter{66D}{\FiveStar} # # If you want to change the title logo it's here # # +BEAMER_HEADER: \titlegraphic{\includegraphics[width=0.7\textwidth]{SWH-logo}} # aspect ratio can be changed, but the slides need to be adapted # - compute a "resizing factor" for the images (macro for picblocks?) # # set the background image # # https://pacoup.com/2011/06/12/list-of-true-169-resolutions/ # #+BEAMER_HEADER: \pgfdeclareimage[height=90mm,width=160mm]{bgd}{swh-world-169.png} #+BEAMER_HEADER: \setbeamertemplate{background}{\pgfuseimage{bgd}} #+LATEX: \addtocounter{framenumber}{-1} * Software Source Code: precious part of our Knowledge ** Software source code: a precious part of our heritage #+INCLUDE: "../../common/modules/source-code-different-short.org::#softwareisdifferent" :only-contents t :minlevel 3 ** Software Commons, Open Science #+INCLUDE: "../../common/modules/foss-commons.org::#commonsdef" :only-contents t +#+BEAMER: \pause *** *Source code*: part of our commons\hfill ... pillar of Open Science! /(would require another talk)/ #+BEAMER: \pause *** Precious, endangered /executable/ and /human readable/ knowledge key people *passing away*, platforms (GoogleCode, Gitorious, etc.) closing down ...\\ ** Source code: /special/ and /endangered/ :noexport: *** /Executable/ and /human readable/ knowledge \hfill copyright law :noexport: /“Programs must be written for people to read, and only incidentally for machines to execute.”/\\ \hfill Harold Abelson #+BEAMER: \pause *** Software /evolves/ over time - projects may last decades - the /development history/ is key to its /understanding/ #+BEAMER: \pause *** Complexity :B_picblock: :PROPERTIES: :BEAMER_env: picblock :BEAMER_OPT: pic=python3-matplotlib.pdf, width=.6\linewidth :END: - /millions/ of lines of code - large /web of dependencies/ + easy to break, difficult to maintain + /research software/ a thin top layer - sophisticated /developer communities/ *** :B_ignoreheading: :PROPERTIES: :BEAMER_env: ignoreheading :END: #+BEAMER: \pause *** Precious, endangered /executable/ and /human readable/ knowledge key people *passing away*, platforms (GoogleCode, Gitorious, etc.) closing down ...\\ \hfill no organised effort to catalog and archive it * Meet Software Heritage ** Software Heritage in a nutshell \hfill www.softwareheritage.org #+BEAMER: \transdissolve #+INCLUDE: "../../common/modules/swh-goals-oneslide-vertical.org::#goals" :only-contents t :minlevel 3 ** Largest software archive, principled \hfill \url{http://bit.ly/swhpaper} #+latex: \begin{center} #+ATTR_LATEX: :width 0.5\linewidth file:SWH-as-foundation-slim.png #+latex: \end{center} #+BEAMER: \pause #+latex: \centering #+ATTR_LATEX: :width \extblockscale{.6\linewidth} file:2020-09-08-growth.png #+BEAMER: \pause *** Technology :PROPERTIES: :BEAMER_col: 0.34 :BEAMER_env: block :END: - transparency and FOSS - replicas all the way down *** Content (billions!) :PROPERTIES: :BEAMER_col: 0.32 :BEAMER_env: block :END: - *intrinsic identifiers* - facts and provenance *** Organization :PROPERTIES: :BEAMER_col: 0.33 :BEAMER_env: block :END: - non-profit - multi-stakeholder ** A peek under the hood #+BEAMER: \begin{center} #+BEAMER: \mode{\only<1>{\includegraphics[width=\extblockscale{1\textwidth}]{swh-dataflow-merkle-listers.pdf}}} #+BEAMER: \only<2-3>{\includegraphics[width=\extblockscale{1\textwidth}]{swh-dataflow-merkle.pdf}} #+BEAMER: \end{center} #+BEAMER: \pause #+BEAMER: \pause /Global development history/ permanently archived in a /unique/ git-like Merkle DAG - *~400 TB* (uncompressed) blobs, *~20 B* nodes, *~280 B* edges # - *GitHub*, Gitlab.com, Bitbucket, /Gitorious/, /GoogleCode/, GNU, PyPi, Debian, NPM... ** Software Heritage Identifiers (SWHID) \hfill [[https://docs.softwareheritage.org/devel/swh-model/persistent-identifiers.html][link to full docs]] # #+INCLUDE: "../../common/modules/swh-id-syntax.org::#swh-id-syntax" :only-contents t :minlevel 3 #+LATEX: \centering%\forcebeamerstart #+LATEX: \mode{\only<1>{\includegraphics[width=\linewidth]{SWHID-v1.4_1.png}}} #+LATEX: \mode{\only<2>{\includegraphics[width=\linewidth]{SWHID-v1.4_2.png}}} #+LATEX: \only<3->{\includegraphics[width=\linewidth]{SWHID-v1.4_3.png}} #+LATEX: %\forcebeamerend *** An emerging standard :B_block: :PROPERTIES: :BEAMER_act: <4-> :BEAMER_COL: .6 :BEAMER_env: block :END: - in Linux Foundation's [[https://spdx.github.io/spdx-spec/appendix-VI-external-repository-identifiers/#persistent-id][SPDX 2.2]] - IANA registered, WikiData property [[https://www.wikidata.org/wiki/Property:P6138][P6138]] *** Examples: :B_block: :PROPERTIES: :BEAMER_act: <5-> :BEAMER_COL: .4 :BEAMER_env: block :END: - [[https://archive.softwareheritage.org/swh:1:cnt:64582b78792cd6c2d67d35da5a11bb80886a6409;origin=https://github.com/virtualagc/virtualagc;lines=245-261/][Apollo 11 AGC excerpt]], - [[https://archive.softwareheritage.org/swh:1:cnt:bb0faf6919fc60636b2696f32ec9b3c2adb247fe;origin=https://github.com/id-Software/Quake-III-Arena;lines=549-572/][Quake III rsqrt]] * Demo time! * Building for the long term ... ** An international, non profit initiative... :PROPERTIES: :CUSTOM_ID: support :END: *** Sharing the vision :B_block: :PROPERTIES: :CUSTOM_ID: endorsement :BEAMER_COL: .5 :BEAMER_env: block :END: #+LATEX: \begin{center}{\includegraphics[width=\extblockscale{.4\linewidth}]{unesco_logo_en_285}}\end{center} #+LATEX: \vspace{-0.8cm} #+LATEX: \begin{center}\vskip 1em \includegraphics[width=\extblockscale{1.4\linewidth}]{support.pdf}\end{center} #+latex: \small And many more ...\\ #+latex:\mbox{}~~~~~~~\tiny\url{www.softwareheritage.org/support/testimonials} +#+BEAMER: \pause *** Donors, members, sponsors :B_block: :PROPERTIES: :CUSTOM_ID: sponsors :BEAMER_COL: .5 :BEAMER_env: block :END: #+LATEX: \begin{center}\includegraphics[width=\extblockscale{.4\linewidth}]{inria-logo-new}\end{center} #+LATEX: \begin{center} # #+LATEX: \includegraphics[width=\extblockscale{.2\linewidth}]{sponsors-levels.pdf} #+LATEX: \colorbox{white}{\includegraphics[width=\extblockscale{1.4\linewidth}]{sponsors.pdf}} #+LATEX: \end{center} # - sponsoring / partnership :: \hfill \url{sponsorship.softwareheritage.org} *** :B_ignoreheading: :PROPERTIES: :BEAMER_env: ignoreheading :END: *** Research collaboration :B_picblock:noexport: :PROPERTIES: :BEAMER_COL: .5 :BEAMER_env: picblock :BEAMER_OPT: pic=Qwant_Logo, leftpic=true :END: source code search engine *** See more :noexport: \hfill\tiny\url{http:://www.softwareheritage.org/support/testimonials} *** Global network :B_picblock:noexport: :PROPERTIES: :BEAMER_COL: .5 :BEAMER_env: picblock :BEAMER_OPT: pic=fossid, leftpic=true, width=.3\linewidth :END: - first *independent mirror* - increased reliability ** ... creating a mirror network ... *** Thomas Jefferson, February 18, 1791 :B_block: :PROPERTIES: :BEAMER_ACT: :BEAMER_env: block :END: #+latex: {\em ...let us save what remains: not by vaults and locks which fence them from the public eye and use in consigning them to the waste of time, but by such a multiplication of copies, as shall place them beyond the reach of accident. #+latex: } #+BEAMER: \pause *** Welcoming ENEA :B_block: :PROPERTIES: :BEAMER_env: picblock :BEAMER_OPT: pic=LogoENEAcompletoENG.png, leftpic=true, width=.7\linewidth :END: - first *institutional* mirror - increased resilience - - *AI infrastructure* for researchers + - *infrastructure* for researchers - stepping stone to \endgraf - \hfill an European joint effort + \hfill an international joint effort ** ... raising awareness about Software Source Code *** :B_column:BMCOL: :PROPERTIES: :BEAMER_col: .53 :BEAMER_env: column :END: #+ATTR_LATEX: :width .7\linewidth file:UNESCOParisCallMeeting.png UNESCO, Inria, Software Heritage invite\\ [[https://en.unesco.org/news/experts-call-greater-recognition-software-source-code-heritage-sustainable-development][40 international experts meet in Paris]] ... #+BEAMER: \pause *** :B_column:BMCOL: :PROPERTIES: :BEAMER_col: .5 :BEAMER_env: column :END: #+ATTR_LATEX: :width .65\linewidth file:paris_call_ssc_cover.jpg [[https://en.unesco.org/foss/paris-call-software-source-code][Their call is published on Feb 2019]] \pause *** :B_ignoreheading: :PROPERTIES: :BEAMER_env: ignoreheading :END: *** :PROPERTIES: :BEAMER_COL: 1.06 :BEAMER_env: block :END: It's an important /policy tool/, already referenced and used ... \hfill /yes, you can sign it!/\\ \vspace{10pt} \hfill https://en.unesco.org/foss/paris-call-software-source-code \hfill\mbox{} * ... and firefighting too! ** It's urgent to expand the archive *** Saving 250.000 endangered repositories... - summer 2019: BitBucket announce Mercurial VCS phase out - fall 2019: Software Heritage teams up with Octobus (funded by NLNet, thanks!) - july 2020: BitBucket erases /250.000/ repositories - august 2020: [[https://bitbucket-archive.softwareheritage.org][bitbucket-archive.softwareheritage.org]] is live #+BEAMER: \pause *** ... preserving the web of knowledge \hfill (Tweet [[https://twitter.com/gabrielaltay/status/1300218789762662401][is here]] ) :B_picblock: :PROPERTIES: :BEAMER_env: picblock :BEAMER_OPT: pic=bitbucket_swh_praise.png, width=.6\linewidth, leftpic=true :END: \\ *Bottomline*\\ /explicit deposit/ is important, ...\\ \mbox{}\hfill ... and we must promote it...\hfill\mbox{}\\ \mbox{}\hfill ... but will never be enough.\\ \mbox{}\\ \mbox{}\hfill /(think also of all software dependencies!)/ ** You can help *** Experts are needed ... - can contribute /listers/ and /loaders/?\\ - apply for a Sloan funded minigrant! - - http://bit.ly/swhgrants - - rolling basis + - can contribute /listers/ and /loaders/? apply for a Sloan funded minigrant! + + http://bit.ly/swhgrants (rolling basis) + - knowledgeable in international organizations? UNESCO seeks contributions! + + https://careers.unesco.org/job/Paris-Consultant-on-Software-Heritage-CIMID/519826502/ #+BEAMER: \pause *** ... and everybody can help - adopt :: SWH in your work - save :: relevant source code - contribute :: SWH is free software - advocate :: spread the word ** Come in, we're open! \hfill www.softwareheritage.org #+BEGIN_EXPORT latex \begin{center} \includegraphics[width=.6\linewidth]{SWH-logo.pdf} \end{center} \begin{center} {\large \url{www.softwareheritage.org} \hspace{4em} \url{@swheritage}} \end{center} #+END_EXPORT #+LATEX: {\begin{center}\Large Everybody is concerned, everybody can help build\end{center}} *** The Library of Alexandria of code :B_picblock: :PROPERTIES: :BEAMER_env: picblock :BEAMER_COL: 0.42 :BEAMER_OPT: pic=clock-spring-forward.png,width=.45\linewidth,leftpic=true :END: - recover the past - structure the future *** A CERN for Software :B_picblock: :PROPERTIES: :BEAMER_env: picblock :BEAMER_COL: 0.5 :BEAMER_OPT: pic=atacama-telescope.jpg,width=.5\linewidth,leftpic=true :END: - build better software + for industry + for society as a whole * Appendix :B_appendix: :PROPERTIES: :BEAMER_env: appendix :END: ** \vfill \centerline{\Huge Appendix} \vfill * SWHIDs by the example ** A word on the trust model for systems of identifiers \vspace{-5pt} *** Two general classes of systems of identifiers - intrinsic :: /computed/ from the object /(no registry required, fully decentralised)/\\ /(e.g.: chemical notation, music notation, hashes, SWHIDs)/\pause - extrinsic :: /assigned/ by an authority /(need a registry)/\\ /(e.g.: passport number, DOI, ARK, RRID, etc.)/\pause \mbox{}\hfill See [[https://www.softwareheritage.org/2020/07/09/intrinsic-vs-extrinsic-identifiers/][the dedicated blog post]] for more details #+BEAMER: \pause *** Trust model, extrinsic (e.g. DOIs) :B_block: :PROPERTIES: :BEAMER_COL: .5 :BEAMER_env: block :END: #+ATTR_LATEX: :width \linewidth file:doi-vs-pid-1.pdf #+BEAMER: \pause *** Trust model, intrinsic (e.g. SWHIDs) :B_block: :PROPERTIES: :BEAMER_env: block :BEAMER_COL: .45 :END: #+ATTR_LATEX: :width .8\linewidth file:doi-vs-pid-3.pdf *** Trust model for DOIs with checksums :B_block:noexport: :PROPERTIES: :BEAMER_COL: .5 :BEAMER_env: block :END: #+ATTR_LATEX: :width \linewidth file:doi-vs-pid-2.pdf *** :B_ignoreheading:noexport: :PROPERTIES: :BEAMER_env: ignoreheading :END: ** Software Heritage Identifiers (SWHID) \hfill [[https://docs.softwareheritage.org/devel/swh-model/persistent-identifiers.html][link to full docs]] # #+INCLUDE: "../../common/modules/swh-id-syntax.org::#swh-id-syntax" :only-contents t :minlevel 3 #+LATEX: \centering%\forcebeamerstart #+LATEX: \mode{\only<1>{\includegraphics[width=\linewidth]{SWHID-v1.4_1.png}}} #+LATEX: \mode{\only<2>{\includegraphics[width=\linewidth]{SWHID-v1.4_2.png}}} #+LATEX: \only<3->{\includegraphics[width=\linewidth]{SWHID-v1.4_3.png}} #+LATEX: %\forcebeamerend *** An emerging standard :B_block: :PROPERTIES: :BEAMER_act: <4-> :BEAMER_COL: .6 :BEAMER_env: block :END: - in Linux Foundation's [[https://spdx.github.io/spdx-spec/appendix-VI-external-repository-identifiers/#persistent-id][SPDX 2.2]] - IANA registered, WikiData property [[https://www.wikidata.org/wiki/Property:P6138][P6138]] *** Examples: :B_block: :PROPERTIES: :BEAMER_act: <5-> :BEAMER_COL: .4 :BEAMER_env: block :END: - [[https://archive.softwareheritage.org/swh:1:cnt:64582b78792cd6c2d67d35da5a11bb80886a6409;origin=https://github.com/virtualagc/virtualagc;lines=245-261/][Apollo 11 AGC excerpt]], - [[https://archive.softwareheritage.org/swh:1:cnt:bb0faf6919fc60636b2696f32ec9b3c2adb247fe;origin=https://github.com/id-Software/Quake-III-Arena;lines=549-572/][Quake III rsqrt]] ** A worked example #+LATEX: \centering\forcebeamerstart #+LATEX: \only<1>{\colorbox{white}{\includegraphics[width=\extblockscale{\linewidth}]{git-merkle/merkle_1.pdf}}} #+LATEX: \only<2>{\colorbox{white}{\includegraphics[width=\extblockscale{\linewidth}]{git-merkle/contents.pdf}}} #+LATEX: \only<3>{\colorbox{white}{\includegraphics[width=\extblockscale{\linewidth}]{git-merkle/merkle_2_contents.pdf}}} #+LATEX: \only<4>{\colorbox{white}{\includegraphics[width=\extblockscale{\linewidth}]{git-merkle/directories.pdf}}} #+LATEX: \only<5>{\colorbox{white}{\includegraphics[width=\extblockscale{\linewidth}]{git-merkle/merkle_3_directories.pdf}}} #+LATEX: \only<6>{\colorbox{white}{\includegraphics[width=\extblockscale{\linewidth}]{git-merkle/revisions.pdf}}} #+LATEX: \only<7>{\colorbox{white}{\includegraphics[width=\extblockscale{\linewidth}]{git-merkle/merkle_4_revisions.pdf}}} #+LATEX: \only<8>{\colorbox{white}{\includegraphics[width=\extblockscale{\linewidth}]{git-merkle/releases.pdf}}} #+LATEX: \only<9>{\colorbox{white}{\includegraphics[width=\extblockscale{\linewidth}]{git-merkle/merkle_5_releases.pdf}}} #+LATEX: \only<10>{\colorbox{white}{\includegraphics[width=\extblockscale{\linewidth}]{git-merkle/snapshots.pdf}}} #+LATEX: \forcebeamerend * News ** Milestones :noexport: #+INCLUDE: "../../common/modules/swh-key-dates.org::#keydates" :minlevel 3 :only-contents t ** News : archiving /public/ code #+latex: \begin{center} #+ATTR_LATEX: :width 0.7\linewidth file:codeetalab.png #+latex: \end{center} #+BEAMER: \pause https://code.etalab.gouv.fr ** News : SWHAP *** Paris Call on Software Source Code “[We call to] support efforts to gather and preserve the artifacts and narratives of the history of computing, while the earlier creators are still alive” #+BEAMER: \pause *** SWHAP : an important step forward - detailed guidelines to *curate* landmark legacy source code and *archive* it on Software Heritage - intense cooperation with *Università di Pisa* and *UNESCO* - open to all, we'll promote it worldwide *** https://www.softwareheritage.org/swhap -** News : ENEA mirror -*** Thomas Jefferson, February 18, 1791 :B_block: - :PROPERTIES: - :BEAMER_ACT: - :BEAMER_env: block - :END: -#+latex: {\em - ...let us save what remains: not by vaults and locks which fence them - from the public eye and use in consigning them to the waste of time, - but by such a multiplication of copies, as shall place them beyond - the reach of accident. -#+latex: } - #+BEAMER: \pause -*** Welcoming ENEA :B_block: - :PROPERTIES: - :BEAMER_env: picblock - :BEAMER_OPT: pic=LogoENEAcompletoENG.png, leftpic=true, width=.7\linewidth - :END: - - first *institutional* mirror - - increased resilience - - *AI infrastructure* for researchers - - stepping stone to \endgraf - \hfill an European joint effort