Page MenuHomeSoftware Heritage

keycloak: Update deployment with puppet
ClosedPublic

Authored by anlambert on Thu, Oct 8, 9:08 PM.

Details

Summary

That diff contains four commits related to Keycloak deployment and configuration
with puppet:

  • Set some login related realm options to true
  • Add roles to swh-web client (Roles define permissions for a given client).
  • Override direct grant flow for swh-web client by removing the Conditional OTP flow execution. This will prevent users that have configured OTP in their account to get invalid credentials error when trying to generate a bearer token for Web API authentication (experienced by @haltode that morning).
  • Upgrade Keycloak from 8.0.1 to 10.0.2

I tested those changes locally with pupperware and everything worked fine.

Below is the octocatalog diff:

20:14 $ bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details -t staging kelvingrove.internal.softwareheritage.org
Found host kelvingrove.internal.softwareheritage.org
Cloning into '/tmp/swh-ocd.7PWUzRsO/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.7PWUzRsO/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host kelvingrove.internal.softwareheritage.org
I, [2020-10-08T20:14:42.141264 #1878250]  INFO -- : Catalogs compiled for kelvingrove.internal.softwareheritage.org
I, [2020-10-08T20:14:42.349763 #1878250]  INFO -- : Diffs computed for kelvingrove.internal.softwareheritage.org
diff origin/production/kelvingrove.internal.softwareheritage.org current/kelvingrove.internal.softwareheritage.org
*******************************************
+ Archive[keycloak-10.0.2.tar.gz] =>
   parameters =>
      "cleanup": true
      "creates": "/opt/keycloak-10.0.2/bin"
      "ensure": "present"
      "extract": true
      "extract_command": "tar xfz %s --strip-components=1"
      "extract_path": "/opt/keycloak-10.0.2"
      "group": "keycloak"
      "path": "/tmp/keycloak-10.0.2.tar.gz"
      "source": "https://downloads.jboss.org/keycloak/10.0.2/keycloak-10.0.2.tar.gz"
      "user": "keycloak"
*******************************************
- Archive[keycloak-8.0.1.tar.gz]
*******************************************
  Concat::Fragment[config.cli-keycloak] =>
   parameters =>
     target =>
      - /opt/keycloak-8.0.1/config.cli
      + /opt/keycloak-10.0.2/config.cli
*******************************************
+ Concat[/opt/keycloak-10.0.2/config.cli] =>
   parameters =>
      "backup": "puppet"
      "ensure": "present"
      "ensure_newline": false
      "force": false
      "format": "plain"
      "group": "keycloak"
      "mode": "0600"
      "notify": "Exec[jboss-cli.sh --file=config.cli]"
      "order": "alpha"
      "owner": "keycloak"
      "path": "/opt/keycloak-10.0.2/config.cli"
      "replace": true
      "show_diff": false
      "warn": false
*******************************************
- Concat[/opt/keycloak-8.0.1/config.cli]
*******************************************
+ Concat_file[/opt/keycloak-10.0.2/config.cli] =>
   parameters =>
      "backup": "puppet"
      "ensure_newline": false
      "force": false
      "format": "plain"
      "group": "keycloak"
      "mode": "0600"
      "order": "alpha"
      "owner": "keycloak"
      "replace": true
      "show_diff": false
      "tag": "_opt_keycloak-10.0.2_config.cli"
*******************************************
- Concat_file[/opt/keycloak-8.0.1/config.cli]
*******************************************
  Concat_fragment[config.cli-keycloak] =>
   parameters =>
     tag =>
      - _opt_keycloak-8.0.1_config.cli
      + _opt_keycloak-10.0.2_config.cli
     target =>
      - /opt/keycloak-8.0.1/config.cli
      + /opt/keycloak-10.0.2/config.cli
*******************************************
  Exec[create-keycloak-admin] =>
   parameters =>
     command =>
      - /opt/keycloak-8.0.1/bin/add-user-keycloak.sh --user keycloak-admin --password keycloak::admin::password --realm master && touch /opt/keycloak-8.0.1/.create-keycloak-admin-postgresql
      + /opt/keycloak-10.0.2/bin/add-user-keycloak.sh --user keycloak-admin --password keycloak::admin::password --realm master && touch /opt/keycloak-10.0.2/.create-keycloak-admin-postgresql
     creates =>
      - /opt/keycloak-8.0.1/.create-keycloak-admin-postgresql
      + /opt/keycloak-10.0.2/.create-keycloak-admin-postgresql
*******************************************
  Exec[jboss-cli.sh --file=config.cli] =>
   parameters =>
     command =>
      - /opt/keycloak-8.0.1/bin/jboss-cli.sh --file=config.cli
      + /opt/keycloak-10.0.2/bin/jboss-cli.sh --file=config.cli
     cwd =>
      - /opt/keycloak-8.0.1
      + /opt/keycloak-10.0.2
*******************************************
+ Exec[mkdir -p /opt/keycloak-10.0.2/modules/system/layers/keycloak/org/postgresql/main] =>
   parameters =>
      "creates": "/opt/keycloak-10.0.2/modules/system/layers/keycloak/org/postgresql/main"
      "group": "keycloak"
      "path": "/usr/bin:/bin"
      "user": "keycloak"
*******************************************
- Exec[mkdir -p /opt/keycloak-8.0.1/modules/system/layers/keycloak/org/postgresql/main]
*******************************************
  File[/etc/systemd/system/keycloak.service] =>
   parameters =>
     content =>
      @@ -7,5 +7,5 @@
       User=keycloak
       Group=keycloak
      -ExecStart=/opt/keycloak-8.0.1/bin/standalone.sh -b 0.0.0.0 -Djboss.http.port=8080
      +ExecStart=/opt/keycloak-10.0.2/bin/standalone.sh -b 0.0.0.0 -Djboss.http.port=8080
       TimeoutStartSec=600
       TimeoutStopSec=600
*******************************************
+ File[/opt/keycloak-10.0.2/bin/kcadm-wrapper.sh] =>
   parameters =>
      "ensure": "file"
      "group": "keycloak"
      "mode": "0750"
      "owner": "keycloak"
      "show_diff": false
      "content": >>>
#!/bin/bash

KCADM="/opt/keycloak-10.0.2/bin/kcadm.sh"

${KCADM} "$@" --no-config --server http://localhost:8080/auth --realm master --user keycloak-admin --password keycloak::admin::password
<<<
*******************************************
+ File[/opt/keycloak-10.0.2/modules/system/layers/keycloak/org/postgresql/main/module.xml] =>
   parameters =>
      "ensure": "file"
      "group": "keycloak"
      "mode": "0644"
      "owner": "keycloak"
      "content": >>>
<?xml version="1.0" ?>
<module xmlns="urn:jboss:module:1.3" name="org.postgresql">

    <resources>
        <resource-root path="postgresql-jdbc.jar"/>
    </resources>

    <dependencies>
        <module name="javax.api"/>
        <module name="javax.transaction.api"/>
    </dependencies>
</module>
<<<
*******************************************
+ File[/opt/keycloak-10.0.2/modules/system/layers/keycloak/org/postgresql/main/postgresql-jdbc.jar] =>
   parameters =>
      "ensure": "file"
      "group": "keycloak"
      "mode": "0644"
      "owner": "keycloak"
      "source": "/usr/share/java/postgresql.jar"
*******************************************
+ File[/opt/keycloak-10.0.2/modules/system/layers/keycloak/org/postgresql/main] =>
   parameters =>
      "ensure": "directory"
      "group": "keycloak"
      "mode": "0755"
      "owner": "keycloak"
*******************************************
+ File[/opt/keycloak-10.0.2/standalone/configuration/profile.properties] =>
   parameters =>
      "ensure": "file"
      "group": "keycloak"
      "mode": "0644"
      "notify": "Class[Keycloak::Service]"
      "owner": "keycloak"
      "content": >>>
# File managed by Puppet - DO NOT EDIT
<<<
*******************************************
+ File[/opt/keycloak-10.0.2/standalone/configuration] =>
   parameters =>
      "ensure": "directory"
      "group": "keycloak"
      "mode": "0750"
      "owner": "keycloak"
*******************************************
+ File[/opt/keycloak-10.0.2/themes/swh] =>
   parameters =>
      "ensure": "link"
      "target": "/opt/swh-keycloak-theme/swh"
*******************************************
+ File[/opt/keycloak-10.0.2/tmp] =>
   parameters =>
      "ensure": "directory"
      "group": "keycloak"
      "mode": "0755"
      "owner": "keycloak"
*******************************************
+ File[/opt/keycloak-10.0.2] =>
   parameters =>
      "ensure": "directory"
      "group": "keycloak"
      "mode": "0755"
      "owner": "keycloak"
*******************************************
- File[/opt/keycloak-8.0.1/bin/kcadm-wrapper.sh]
*******************************************
- File[/opt/keycloak-8.0.1/modules/system/layers/keycloak/org/postgresql/main/module.xml]
*******************************************
- File[/opt/keycloak-8.0.1/modules/system/layers/keycloak/org/postgresql/main/postgresql-jdbc.jar]
*******************************************
- File[/opt/keycloak-8.0.1/modules/system/layers/keycloak/org/postgresql/main]
*******************************************
- File[/opt/keycloak-8.0.1/standalone/configuration/profile.properties]
*******************************************
- File[/opt/keycloak-8.0.1/standalone/configuration]
*******************************************
- File[/opt/keycloak-8.0.1/themes/swh]
*******************************************
- File[/opt/keycloak-8.0.1/tmp]
*******************************************
- File[/opt/keycloak-8.0.1]
*******************************************
  File[/opt/keycloak] =>
   parameters =>
     target =>
      - /opt/keycloak-8.0.1
      + /opt/keycloak-10.0.2
*******************************************
  File_line[standalone.conf-JAVA_OPTS] =>
   parameters =>
     path =>
      - /opt/keycloak-8.0.1/bin/standalone.conf
      + /opt/keycloak-10.0.2/bin/standalone.conf
*******************************************
  Keycloak_client[swh-web on SoftwareHeritageStaging] =>
   parameters =>
     direct_grant_flow =>
      + direct_grant_no_otp-SoftwareHeritageStaging
     roles =>
      + ["swh.web.api.throtlling_exempted", "swh.web.api.graph"]
*******************************************
  Keycloak_client[swh-web on SoftwareHeritage] =>
   parameters =>
     direct_grant_flow =>
      + direct_grant_no_otp-SoftwareHeritage
     roles =>
      + ["swh.web.api.throtlling_exempted", "swh.web.api.graph"]
*******************************************
+ Keycloak_flow[direct_grant_no_otp on SoftwareHeritageStaging] =>
   parameters =>
      "alias": "direct_grant_no_otp-SoftwareHeritageStaging"
      "description": "Direct grant flow without conditional OTP"
      "ensure": "present"
      "id": "d6a91808-4cad-5e18-a48e-7e48e4281edb"
      "realm": "SoftwareHeritageStaging"
*******************************************
+ Keycloak_flow[direct_grant_no_otp on SoftwareHeritage] =>
   parameters =>
      "alias": "direct_grant_no_otp-SoftwareHeritage"
      "description": "Direct grant flow without conditional OTP"
      "ensure": "present"
      "id": "cff702ba-f497-5298-b244-4b1519bb8799"
      "realm": "SoftwareHeritage"
*******************************************
+ Keycloak_flow_execution[direct-grant-validate-password under direct_grant_no_otp-SoftwareHeritage on SoftwareHeritage] =>
   parameters =>
      "alias": "direct-grant-validate-password-SoftwareHeritage"
      "ensure": "present"
      "flow_alias": "direct_grant_no_otp-SoftwareHeritage"
      "id": "a288cfe6-fa66-585e-8e50-2babc5f764b8"
      "index": 0
      "provider_id": "direct-grant-validate-password"
      "realm": "SoftwareHeritage"
      "requirement": "REQUIRED"
*******************************************
+ Keycloak_flow_execution[direct-grant-validate-password under direct_grant_no_otp-SoftwareHeritageStaging on SoftwareHeritageStaging] =>
   parameters =>
      "alias": "direct-grant-validate-password-SoftwareHeritageStaging"
      "ensure": "present"
      "flow_alias": "direct_grant_no_otp-SoftwareHeritageStaging"
      "id": "39e8b167-5328-5c95-9cd3-d15b6ae85c83"
      "index": 0
      "provider_id": "direct-grant-validate-password"
      "realm": "SoftwareHeritageStaging"
      "requirement": "REQUIRED"
*******************************************
+ Keycloak_flow_execution[direct-grant-validate-username under direct_grant_no_otp-SoftwareHeritage on SoftwareHeritage] =>
   parameters =>
      "alias": "direct-grant-validate-username-SoftwareHeritage"
      "ensure": "present"
      "flow_alias": "direct_grant_no_otp-SoftwareHeritage"
      "id": "d6b5deea-503c-5041-ad59-c9a4ecb21344"
      "index": 0
      "provider_id": "direct-grant-validate-username"
      "realm": "SoftwareHeritage"
      "requirement": "REQUIRED"
*******************************************
+ Keycloak_flow_execution[direct-grant-validate-username under direct_grant_no_otp-SoftwareHeritageStaging on SoftwareHeritageStaging] =>
   parameters =>
      "alias": "direct-grant-validate-username-SoftwareHeritageStaging"
      "ensure": "present"
      "flow_alias": "direct_grant_no_otp-SoftwareHeritageStaging"
      "id": "f77d5d0b-71fc-5c28-aff5-72f32eac709a"
      "index": 0
      "provider_id": "direct-grant-validate-username"
      "realm": "SoftwareHeritageStaging"
      "requirement": "REQUIRED"
*******************************************
  Keycloak_realm[SoftwareHeritageStaging] =>
   parameters =>
     registration_allowed =>
      + true
     reset_password_allowed =>
      + true
     verify_email =>
      + true
*******************************************
  Keycloak_realm[SoftwareHeritage] =>
   parameters =>
     registration_allowed =>
      + true
     reset_password_allowed =>
      + true
     verify_email =>
      + true
*******************************************
  Systemd::Unit_file[keycloak.service] =>
   parameters =>
     content =>
      @@ -7,5 +7,5 @@
       User=keycloak
       Group=keycloak
      -ExecStart=/opt/keycloak-8.0.1/bin/standalone.sh -b 0.0.0.0 -Djboss.http.port=8080
      +ExecStart=/opt/keycloak-10.0.2/bin/standalone.sh -b 0.0.0.0 -Djboss.http.port=8080
       TimeoutStartSec=600
       TimeoutStopSec=600
*******************************************
*** End octocatalog-diff on kelvingrove.internal.softwareheritage.org

Diff Detail

Repository
rSPSITE puppet-swh-site
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

anlambert created this revision.Thu, Oct 8, 9:08 PM
olasd added a subscriber: olasd.Thu, Oct 8, 10:16 PM

Thanks!

I'm a bit concerned about the keycloak upgrade process:

Have you tested the actual upgrade process in pupperware?

olasd added a comment.Fri, Oct 9, 11:31 AM
In D4211#104343, @olasd wrote:

Have you tested the actual upgrade process in pupperware?

Btw, it's okay if the answer is "no"; we'll just make sure to do the deployment in prod under supervision, and manually run a migration script if needed (and then we'll *know* :))

The latest upstream version is 11.0.2, but I guess this is blocked on https://github.com/treydock/puppet-module-keycloak/pull/154; Could you add a comment where the keycloak version is defined?

Yes, there is an issue with flows update when using Keycloak 11.x so I preferred to upgrade to 10.x. What do you mean by adding a comment on the version definition ?

I was looking at upstream docs which say a migration script needs to be executed: https://www.keycloak.org/docs/latest/upgrading/, and I got worried. But the puppet module claims it can handle upgrades, although I don't see it setup the postgres connector to do the database upgrades, so we'll need to double check.

The postgres connector seems to be correctly setup when upgrading, see below:

+ File[/opt/keycloak-10.0.2/modules/system/layers/keycloak/org/postgresql/main/module.xml] =>
   parameters =>
      "ensure": "file"
      "group": "keycloak"
      "mode": "0644"
      "owner": "keycloak"
      "content": >>>
<?xml version="1.0" ?>
<module xmlns="urn:jboss:module:1.3" name="org.postgresql">

    <resources>
        <resource-root path="postgresql-jdbc.jar"/>
    </resources>

    <dependencies>
        <module name="javax.api"/>
        <module name="javax.transaction.api"/>
    </dependencies>
</module>
<<<
*******************************************
+ File[/opt/keycloak-10.0.2/modules/system/layers/keycloak/org/postgresql/main/postgresql-jdbc.jar] =>
   parameters =>
      "ensure": "file"
      "group": "keycloak"
      "mode": "0644"
      "owner": "keycloak"
      "source": "/usr/share/java/postgresql.jar"
*******************************************
+ File[/opt/keycloak-10.0.2/modules/system/layers/keycloak/org/postgresql/main] =>
   parameters =>
      "ensure": "directory"
      "group": "keycloak"
      "mode": "0755"
      "owner": "keycloak"

Have you tested the actual upgrade process in pupperware?

Yes of course. I first ran puppet a first time to get to the same state as we currently are in production and created a couple of users. Then I applied the changes in that diff and ran puppet again. The upgrade was successful and created users still available in the database afterwards.

anlambert edited the summary of this revision. (Show Details)Fri, Oct 9, 11:35 AM
olasd accepted this revision.Fri, Oct 9, 11:38 AM

The latest upstream version is 11.0.2, but I guess this is blocked on https://github.com/treydock/puppet-module-keycloak/pull/154; Could you add a comment where the keycloak version is defined?

Yes, there is an issue with flows update when using Keycloak 11.x so I preferred to upgrade to 10.x. What do you mean by adding a comment on the version definition ?

I've added an inline comment to be clearer.

I was looking at upstream docs which say a migration script needs to be executed: https://www.keycloak.org/docs/latest/upgrading/, and I got worried. But the puppet module claims it can handle upgrades, although I don't see it setup the postgres connector to do the database upgrades, so we'll need to double check.

The postgres connector seems to be correctly setup when upgrading, see below:

+ File[/opt/keycloak-10.0.2/modules/system/layers/keycloak/org/postgresql/main/module.xml] =>
   parameters =>
      "ensure": "file"
      "group": "keycloak"
      "mode": "0644"
      "owner": "keycloak"
      "content": >>>
<?xml version="1.0" ?>
<module xmlns="urn:jboss:module:1.3" name="org.postgresql">

    <resources>
        <resource-root path="postgresql-jdbc.jar"/>
    </resources>

    <dependencies>
        <module name="javax.api"/>
        <module name="javax.transaction.api"/>
    </dependencies>
</module>
<<<
*******************************************
+ File[/opt/keycloak-10.0.2/modules/system/layers/keycloak/org/postgresql/main/postgresql-jdbc.jar] =>
   parameters =>
      "ensure": "file"
      "group": "keycloak"
      "mode": "0644"
      "owner": "keycloak"
      "source": "/usr/share/java/postgresql.jar"
*******************************************
+ File[/opt/keycloak-10.0.2/modules/system/layers/keycloak/org/postgresql/main] =>
   parameters =>
      "ensure": "directory"
      "group": "keycloak"
      "mode": "0755"
      "owner": "keycloak"

I meant that I haven't found anything that looks like https://www.keycloak.org/docs/latest/upgrading/#automatic-relational-database-migration (either the spi setting or the CLI command).

Have you tested the actual upgrade process in pupperware?

Yes of course. I first ran puppet a first time to get to the same state as we currently are in production and created a couple of users. Then I applied the changes in that diff and ran puppet again. The upgrade was successful and created users still available in the database afterwards.

Awesome! Then I think we're good to go.

data/common/common.yaml
2902

Add a comment to this line explaining why we stick to 10.0.2 instead of 11.0.2

This revision is now accepted and ready to land.Fri, Oct 9, 11:38 AM
anlambert updated this revision to Diff 14860.Fri, Oct 9, 12:25 PM

Uodate: Rebase and add comment about Keycloak version

olasd added a comment.Fri, Oct 9, 12:28 PM

Uodate: Rebase and add comment about Keycloak version

Awesome!