Page MenuHomeSoftware Heritage

Reference provenance page in annex behind basic auth
ClosedPublic

Authored by ardumont on Wed, Jul 24, 5:38 PM.

Details

Summary

Wants to do the opposite of P481 (add the new annex folder) P481 is a diff of
what's in production against what's in swh-site (so wants to remove the manual
installation of that new folder). We want to keep that new folder so i
puppetized it.

As in commits:

  • annex_web: Explicit possible issue in current annex setup
  • annex: Add missing instructions

As far as the documentation or the execution goes, there is no pre-defined dsl
to set those so fallback to use the custom_fragment instead.

Related P481

Test Plan

bin/octocatalog pergamon
Wants to add the new provenance folder as expected:

$ bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details --to update_configuration pergamon
Found host pergamon.softwareheritage.org
Cloning into '/tmp/swh-ocd.FN56kEgZ/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.FN56kEgZ/environments/update_configuration/data/private'...
done.
*** Running octocatalog-diff on host pergamon.softwareheritage.org
I, [2019-07-24T17:31:43.396106 #8801]  INFO -- : Catalogs compiled for pergamon.softwareheritage.org
W, [2019-07-24T17:31:47.510299 #8801]  WARN -- : Resource File[/tmp/ocd-ipc-20190724-8801-1dxcncu/ocd-builddir-20190724-8826-1u4x99f/routes.yaml] appears to depend on catalog compilation directory. Suppressed from results.
W, [2019-07-24T17:31:47.510373 #8801]  WARN -- : Resource File[/tmp/ocd-ipc-20190724-8801-1dxcncu/ocd-builddir-20190724-8823-nzbbb8/routes.yaml] appears to depend on catalog compilation directory. Suppressed from results.
W, [2019-07-24T17:31:47.510533 #8801]  WARN -- : Resource key Ini_setting[puppetdbserver_urls] parameters => path may depend on catalog compilation directory, but there may be differences. This is included in results for now, but please verify.
W, [2019-07-24T17:31:47.510548 #8801]  WARN -- : Resource key Ini_setting[puppetdbserver_urls] parameters => path appears to depend on catalog compilation directory. Suppressed from results.
W, [2019-07-24T17:31:47.510577 #8801]  WARN -- : Resource key Ini_setting[soft_write_failure] parameters => path may depend on catalog compilation directory, but there may be differences. This is included in results for now, but please verify.
W, [2019-07-24T17:31:47.510588 #8801]  WARN -- : Resource key Ini_setting[soft_write_failure] parameters => path appears to depend on catalog compilation directory. Suppressed from results.
I, [2019-07-24T17:31:47.510751 #8801]  INFO -- : Diffs computed for pergamon.softwareheritage.org
diff origin/production/pergamon.softwareheritage.org current/pergamon.softwareheritage.org
*******************************************
  Apache::Vhost[annex.softwareheritage.org_ssl] =>
   parameters =>
     directories =>
      - [{"path"=>"/srv/softwareheritage/annex/webroot", "require"=>"all granted", "options"=>["Indexes", "FollowSymLinks", "MultiViews"]}, {"path"=>".*/\\.git/?$", "provider"=>"directorymatch", "require"=>"all denied"}]
      + [{"path"=>"/srv/softwareheritage/annex/webroot", "require"=>"all granted", "options"=>["Indexes", "FollowSymLinks", "MultiViews"], "custom_fragment"=>"IndexIgnore private provenance-index"}, {"path"=>".*/\\.git/?$", "provider"=>"directorymatch", "require"=>"all denied"}, {"path"=>"/srv/softwareheritage/annex/webroot/provenance-index", "auth_type"=>"basic", "auth_name"=>"SWH - Password Required", "auth_user_file"=>"/srv/softwareheritage/annex/http_auth_provenance", "auth_require"=>"valid-user", "index_options"=>"FancyIndexing", "custom_fragment"=>"ReadmeName readme.txt"}]
*******************************************
  Concat::Fragment[annex.softwareheritage.org_ssl-directories] =>
   parameters =>
     content =>
      @@ -6,4 +6,5 @@
           AllowOverride None
           Require all granted
      +    IndexIgnore private provenance-index
         </Directory>
      _
      @@ -11,2 +12,12 @@
           Require all denied
         </DirectoryMatch>
      +
      +  <Directory "/srv/softwareheritage/annex/webroot/provenance-index">
      +    IndexOptions FancyIndexing
      +    AllowOverride None
      +    Require valid-user
      +    AuthType basic
      +    AuthName "SWH - Password Required"
      +    AuthUserFile /srv/softwareheritage/annex/http_auth_provenance
      +    ReadmeName readme.txt
      +  </Directory>
*******************************************
  Concat_fragment[annex.softwareheritage.org_ssl-directories] =>
   parameters =>
     content =>
      @@ -6,4 +6,5 @@
           AllowOverride None
           Require all granted
      +    IndexIgnore private provenance-index
         </Directory>
      _
      @@ -11,2 +12,12 @@
           Require all denied
         </DirectoryMatch>
      +
      +  <Directory "/srv/softwareheritage/annex/webroot/provenance-index">
      +    IndexOptions FancyIndexing
      +    AllowOverride None
      +    Require valid-user
      +    AuthType basic
      +    AuthName "SWH - Password Required"
      +    AuthUserFile /srv/softwareheritage/annex/http_auth_provenance
      +    ReadmeName readme.txt
      +  </Directory>
*******************************************
  File[/etc/bind/keys/local-update] =>
   parameters =>
     content =>
      @@ -2,4 +2,4 @@
       key local-update {
        algorithm hmac-sha256;
      - secret "+y13WheleD9OEUGBk9nfXgENgF8pu9peOoGv4PWUwu5N5DUH2+QH3GNtitsgjuXubX+6LLPykln6bmF84zV8aQ==";
      + secret "+sjHABMHP/oYaXYlE+66XomK8omrMGp+lqaJPuaFU9IyDL2d/H01tXzj4TmGXmLZw+n8qdgGpU3jhI2vECq1aQ==";
       };
*******************************************
  File[/etc/bind/rndc.key] =>
   parameters =>
     content =>
      @@ -2,4 +2,4 @@
       key rndc-key {
        algorithm hmac-md5;
      - secret "5oiqmEsGz+WH99azzGwF8O1nRXRPgxdR6oXtk5s5E3WTwSzFUL/JIdSmITx3y9vFeBUmmdt+fPAZphexj+J3Sw==";
      + secret "KYTFjtlZFVptctrrUNbaUidKvxlQltBhXQ8g44CXgpzG/CzqSKMhE6aIT/NWnSJ1zLe/ZJZP1tuXCBLFZmAYDA==";
       };
*******************************************
+ File[/srv/softwareheritage/annex/http_auth_provenance] =>
   parameters =>
      "content": ""
      "ensure": "present"
      "group": "www-data"
      "mode": "0640"
      "owner": "root"
*******************************************
*** End octocatalog-diff on pergamon.softwareheritage.org

Diff Detail

Repository
rSPSITE puppet-swh-site
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Wed, Jul 24, 5:38 PM
ardumont edited the test plan for this revision. (Show Details)Wed, Jul 24, 5:41 PM
ardumont added inline comments.
site-modules/profile/manifests/annex_web.pp
11

That's already been commit in the private repository.

ardumont added inline comments.Wed, Jul 24, 5:44 PM
site-modules/profile/manifests/annex_web.pp
83

As explained, i'm not sure about the production impacts of fixing it.
I don't want to deal with those right now.
So at least, i'm marking it to get back to it at some point.

ardumont edited the test plan for this revision. (Show Details)Wed, Jul 24, 5:47 PM
ftigeot accepted this revision.Wed, Jul 24, 5:48 PM

This technically looks good but from a security point of view, why put the secret "private" and "provenance-index" directories in a publically accessible location ?

They would be better off in a dedicated virtual host, it's easy to miss a directive locking them down.

This revision is now accepted and ready to land.Wed, Jul 24, 5:48 PM
ardumont added inline comments.Wed, Jul 24, 5:48 PM
site-modules/profile/manifests/annex_web.pp
11

committed*

ardumont updated this revision to Diff 5968.Wed, Jul 24, 5:49 PM

Plug to production branch

This revision was automatically updated to reflect the committed changes.