Page MenuHomeSoftware Heritage

Reference provenance page in annex behind basic auth
ClosedPublic

Authored by ardumont on Jul 24 2019, 5:38 PM.

Details

Summary

Wants to do the opposite of P481 (add the new annex folder) P481 is a diff of
what's in production against what's in swh-site (so wants to remove the manual
installation of that new folder). We want to keep that new folder so i
puppetized it.

As in commits:

  • annex_web: Explicit possible issue in current annex setup
  • annex: Add missing instructions

As far as the documentation or the execution goes, there is no pre-defined dsl
to set those so fallback to use the custom_fragment instead.

Related P481

Test Plan

bin/octocatalog pergamon
Wants to add the new provenance folder as expected:

$ bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details --to update_configuration pergamon
Found host pergamon.softwareheritage.org
Cloning into '/tmp/swh-ocd.FN56kEgZ/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.FN56kEgZ/environments/update_configuration/data/private'...
done.
*** Running octocatalog-diff on host pergamon.softwareheritage.org
I, [2019-07-24T17:31:43.396106 #8801]  INFO -- : Catalogs compiled for pergamon.softwareheritage.org
W, [2019-07-24T17:31:47.510299 #8801]  WARN -- : Resource File[/tmp/ocd-ipc-20190724-8801-1dxcncu/ocd-builddir-20190724-8826-1u4x99f/routes.yaml] appears to depend on catalog compilation directory. Suppressed from results.
W, [2019-07-24T17:31:47.510373 #8801]  WARN -- : Resource File[/tmp/ocd-ipc-20190724-8801-1dxcncu/ocd-builddir-20190724-8823-nzbbb8/routes.yaml] appears to depend on catalog compilation directory. Suppressed from results.
W, [2019-07-24T17:31:47.510533 #8801]  WARN -- : Resource key Ini_setting[puppetdbserver_urls] parameters => path may depend on catalog compilation directory, but there may be differences. This is included in results for now, but please verify.
W, [2019-07-24T17:31:47.510548 #8801]  WARN -- : Resource key Ini_setting[puppetdbserver_urls] parameters => path appears to depend on catalog compilation directory. Suppressed from results.
W, [2019-07-24T17:31:47.510577 #8801]  WARN -- : Resource key Ini_setting[soft_write_failure] parameters => path may depend on catalog compilation directory, but there may be differences. This is included in results for now, but please verify.
W, [2019-07-24T17:31:47.510588 #8801]  WARN -- : Resource key Ini_setting[soft_write_failure] parameters => path appears to depend on catalog compilation directory. Suppressed from results.
I, [2019-07-24T17:31:47.510751 #8801]  INFO -- : Diffs computed for pergamon.softwareheritage.org
diff origin/production/pergamon.softwareheritage.org current/pergamon.softwareheritage.org
*******************************************
  Apache::Vhost[annex.softwareheritage.org_ssl] =>
   parameters =>
     directories =>
      - [{"path"=>"/srv/softwareheritage/annex/webroot", "require"=>"all granted", "options"=>["Indexes", "FollowSymLinks", "MultiViews"]}, {"path"=>".*/\\.git/?$", "provider"=>"directorymatch", "require"=>"all denied"}]
      + [{"path"=>"/srv/softwareheritage/annex/webroot", "require"=>"all granted", "options"=>["Indexes", "FollowSymLinks", "MultiViews"], "custom_fragment"=>"IndexIgnore private provenance-index"}, {"path"=>".*/\\.git/?$", "provider"=>"directorymatch", "require"=>"all denied"}, {"path"=>"/srv/softwareheritage/annex/webroot/provenance-index", "auth_type"=>"basic", "auth_name"=>"SWH - Password Required", "auth_user_file"=>"/srv/softwareheritage/annex/http_auth_provenance", "auth_require"=>"valid-user", "index_options"=>"FancyIndexing", "custom_fragment"=>"ReadmeName readme.txt"}]
*******************************************
  Concat::Fragment[annex.softwareheritage.org_ssl-directories] =>
   parameters =>
     content =>
      @@ -6,4 +6,5 @@
           AllowOverride None
           Require all granted
      +    IndexIgnore private provenance-index
         </Directory>
      _
      @@ -11,2 +12,12 @@
           Require all denied
         </DirectoryMatch>
      +
      +  <Directory "/srv/softwareheritage/annex/webroot/provenance-index">
      +    IndexOptions FancyIndexing
      +    AllowOverride None
      +    Require valid-user
      +    AuthType basic
      +    AuthName "SWH - Password Required"
      +    AuthUserFile /srv/softwareheritage/annex/http_auth_provenance
      +    ReadmeName readme.txt
      +  </Directory>
*******************************************
  Concat_fragment[annex.softwareheritage.org_ssl-directories] =>
   parameters =>
     content =>
      @@ -6,4 +6,5 @@
           AllowOverride None
           Require all granted
      +    IndexIgnore private provenance-index
         </Directory>
      _
      @@ -11,2 +12,12 @@
           Require all denied
         </DirectoryMatch>
      +
      +  <Directory "/srv/softwareheritage/annex/webroot/provenance-index">
      +    IndexOptions FancyIndexing
      +    AllowOverride None
      +    Require valid-user
      +    AuthType basic
      +    AuthName "SWH - Password Required"
      +    AuthUserFile /srv/softwareheritage/annex/http_auth_provenance
      +    ReadmeName readme.txt
      +  </Directory>
*******************************************
  File[/etc/bind/keys/local-update] =>
   parameters =>
     content =>
      @@ -2,4 +2,4 @@
       key local-update {
        algorithm hmac-sha256;
      - secret "+y13WheleD9OEUGBk9nfXgENgF8pu9peOoGv4PWUwu5N5DUH2+QH3GNtitsgjuXubX+6LLPykln6bmF84zV8aQ==";
      + secret "+sjHABMHP/oYaXYlE+66XomK8omrMGp+lqaJPuaFU9IyDL2d/H01tXzj4TmGXmLZw+n8qdgGpU3jhI2vECq1aQ==";
       };
*******************************************
  File[/etc/bind/rndc.key] =>
   parameters =>
     content =>
      @@ -2,4 +2,4 @@
       key rndc-key {
        algorithm hmac-md5;
      - secret "5oiqmEsGz+WH99azzGwF8O1nRXRPgxdR6oXtk5s5E3WTwSzFUL/JIdSmITx3y9vFeBUmmdt+fPAZphexj+J3Sw==";
      + secret "KYTFjtlZFVptctrrUNbaUidKvxlQltBhXQ8g44CXgpzG/CzqSKMhE6aIT/NWnSJ1zLe/ZJZP1tuXCBLFZmAYDA==";
       };
*******************************************
+ File[/srv/softwareheritage/annex/http_auth_provenance] =>
   parameters =>
      "content": ""
      "ensure": "present"
      "group": "www-data"
      "mode": "0640"
      "owner": "root"
*******************************************
*** End octocatalog-diff on pergamon.softwareheritage.org

Diff Detail

Repository
rSPSITE puppet-swh-site
Branch
update_configuration
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 7009
Build 9865: arc lint + arc unit

Event Timeline

ardumont added inline comments.
site-modules/profile/manifests/annex_web.pp
11

That's already been commit in the private repository.

site-modules/profile/manifests/annex_web.pp
86

As explained, i'm not sure about the production impacts of fixing it.
I don't want to deal with those right now.
So at least, i'm marking it to get back to it at some point.

This technically looks good but from a security point of view, why put the secret "private" and "provenance-index" directories in a publically accessible location ?

They would be better off in a dedicated virtual host, it's easy to miss a directive locking them down.

This revision is now accepted and ready to land.Jul 24 2019, 5:48 PM
site-modules/profile/manifests/annex_web.pp
11

committed*

This revision was automatically updated to reflect the committed changes.