Page MenuHomeSoftware Heritage

[POC] Add keycloak service and sample SoftwareHeritage realm
AbandonedPublic

Authored by anlambert on Oct 11 2019, 4:57 PM.

Details

Reviewers
None
Group Reviewers
Reviewers
Summary

That diff enables to test Keycloak integration in swh-web using the docker environment.

A sample configuration for a Keycloak realm named SoftwareHeritage will be loaded when the new keycloak service will start.

In that realm, a client named swh-web-api has been created whose purpose is to protect the access to the Software Heritage Web API.

For that client, three roles have been created that can be associated to users:

  • normal-user
  • partner-user
  • staff-user

Users with role partner-user or staff-user have the permission throttling-exempted associated while users
with role normal-user do not have any permissions.

In order to test users authentication and permissions in swh-web, a Python script will be executed when the
associated docker service will start. That script takes care of creating the following users in the realm:

  • admin (password: admin): the realm administrator with role staff-user
  • johndoe (password: johndoe-swh): a user with role partner-user
  • janedoe (password: janedoe-swh): a user with role normal-user

In order to authenticate a user when making calls to the swh web api, proceed as follow:

  1. Get an access token by requesting the new endpoint /auth/token/access/ of the web api (let's test for the janedoe user)
$ curl -X POST http://localhost:5004/api/1/auth/token/access/ -d "username=janedoe" -d "password=janedoe-swh" | jq '.'
{
  "access_token": "eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJURWdfR1o5d1FCXzdDZWk5bUZiYXJYemVJQXlBQ2tTSkVnY0x5Uk1WcVlnIn0.eyJqdGkiOiI0Y2U4NjRjNS00ZTI4LTQ3YjAtODdhYy1hNDUwZWUxYTdmYmIiLCJleHAiOjE1NzA4MDc0OTcsIm5iZiI6MCwiaWF0IjoxNTcwODA3MTk3LCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoic3doLXdlYi1hcGkiLCJzdWIiOiI2ZjExMWEwMy0zNTRiLTQ0NTctYmQyNS01MjAxZGUyMzlkMmIiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6ImY0ZmUxMzRkLWZiNWQtNDc1NC04ZGE4LWU2NjZhYzU4ZGZiMyIsImFjciI6IjEiLCJyZXNvdXJjZV9hY2Nlc3MiOnsic3doLXdlYi1hcGkiOnsicm9sZXMiOlsiZGVmYXVsdCIsIm5vcm1hbC11c2VyIl19fSwic2NvcGUiOiJzd2gtc2VydmljZXMgZW1haWwgcHJvZmlsZSIsImVtYWlsX3ZlcmlmaWVkIjpmYWxzZSwibmFtZSI6IkphbmUgRG9lIiwicHJlZmVycmVkX3VzZXJuYW1lIjoiamFuZWRvZSIsImdpdmVuX25hbWUiOiJKYW5lIiwiZmFtaWx5X25hbWUiOiJEb2UiLCJlbWFpbCI6ImphbmUuZG9lQGV4YW1wbGUub3JnIn0.fZCwPauPdNZeWrKxSgC9D8Chcxl2HSN_WaIrysQ7XH_ipePX04Skbscfd-uZsEKPYhR585Td9-5Eek-JkMxlbYTvEZi1wd1VuFk9xEH_dErs9Lh0paTztsCbhK9tOowl8TdcSWNpCG0E4p8eL9oXRHA07kb2P23cG0PAbXsg87B7f7NHB9ttKZCMAK4SERfh926UH2zTdSHAfIFqLyUqJ59i8mvdrB3W7yiDiMYYlmn7vsZ2uDtObYCtp35QZZlOuGp7ynRxbvSVHPOiJTjBOXoN74R4XEzSBbl3DGgtdWFRnmXv9VfEmn3VR9tl3rQMIq1IMRQ0q8Dl1IJK-iGw0A",
  "expires_in": 300,
  "refresh_expires_in": 1800,
  "refresh_token": "eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIyMTI2NzU3ZC1jYTVhLTQwODMtOGIzOS0xMjlmZDFiODNmNGYifQ.eyJqdGkiOiJhYjdiYjEzNi04NTg2LTQwOWUtYWUyNi02OWY4OTllMjRkNWIiLCJleHAiOjE1NzA4MDg5OTcsIm5iZiI6MCwiaWF0IjoxNTcwODA3MTk3LCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoiaHR0cDovL2tleWNsb2FrOjgwODAvYXV0aC9yZWFsbXMvU29mdHdhcmVIZXJpdGFnZSIsInN1YiI6IjZmMTExYTAzLTM1NGItNDQ1Ny1iZDI1LTUyMDFkZTIzOWQyYiIsInR5cCI6IlJlZnJlc2giLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6ImY0ZmUxMzRkLWZiNWQtNDc1NC04ZGE4LWU2NjZhYzU4ZGZiMyIsInJlc291cmNlX2FjY2VzcyI6eyJzd2gtd2ViLWFwaSI6eyJyb2xlcyI6WyJkZWZhdWx0Iiwibm9ybWFsLXVzZXIiXX19LCJzY29wZSI6InN3aC1zZXJ2aWNlcyBlbWFpbCBwcm9maWxlIn0.xLSHCv2zCkDd3u_AYtYX6tZbvzEkB7N2dK82tTnS9kE"
}
  1. Then use the access token to perform authenticated calls to the web api
$ export TOKEN=<access_token>
$ curl -i -H 'Accept: application/json' -H "Authorization: Bearer ${TOKEN}" http://localhost:5004/api/1/stat/counters/
HTTP/1.1 200 OK
Server: gunicorn/19.9.0
Date: Fri, 11 Oct 2019 15:22:15 GMT
Connection: keep-alive
Content-Type: application/json
Vary: Accept, Cookie
Allow: OPTIONS, GET, HEAD, OPTIONS
X-RateLimit-Limit: 120
X-RateLimit-Remaining: 119
X-RateLimit-Reset: 1570807365
X-Frame-Options: SAMEORIGIN
Content-Length: 2

{}

We can see that the rate limiting headers are present in the api response as janedoe does not have any specific permission.

Let's perform the same operations with the johndoe user.

$ curl -X POST http://localhost:5004/api/1/auth/token/access/ -d "username=johndoe" -d "password=johndoe-swh" | jq '.'  
{
  "access_token": "eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJURWdfR1o5d1FCXzdDZWk5bUZiYXJYemVJQXlBQ2tTSkVnY0x5Uk1WcVlnIn0.eyJqdGkiOiIwOTc3N2Q3MS0xMzdmLTQyZjktYmJiZi1iYTkzNmQ5M2I1YTUiLCJleHAiOjE1NzA4MDc3NTQsIm5iZiI6MCwiaWF0IjoxNTcwODA3NDU0LCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoic3doLXdlYi1hcGkiLCJzdWIiOiIyNWUyNzNjMC02ZGY4LTQ1NjYtYmFkYS03ODE3MjM0ZTc3ZDgiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6IjJlNjgwNzUwLWZlYjQtNDcwZC1hNTg0LWVjOGMwYTNhMWIzMyIsImFjciI6IjEiLCJyZXNvdXJjZV9hY2Nlc3MiOnsic3doLXdlYi1hcGkiOnsicm9sZXMiOlsiZGVmYXVsdCIsInBhcnRuZXItdXNlciIsIm5vcm1hbC11c2VyIl19fSwic2NvcGUiOiJzd2gtc2VydmljZXMgZW1haWwgcHJvZmlsZSIsImVtYWlsX3ZlcmlmaWVkIjpmYWxzZSwibmFtZSI6IkpvaG4gRG9lIiwicHJlZmVycmVkX3VzZXJuYW1lIjoiam9obmRvZSIsImdpdmVuX25hbWUiOiJKb2huIiwiZmFtaWx5X25hbWUiOiJEb2UiLCJlbWFpbCI6ImpvaG4uZG9lQGV4YW1wbGUub3JnIn0.kbPLtlf2j6ZteG2xuoBXOs1kMQqYgK4eynIL3J3A31h5mAGYkVP7-ad_fwiTElRL7T_RLI-9Tu_MhOPTb5kjP5_sPdM1iFHwiXbA6rcVBCp4qTACFHDwAi6FQCzVXNFQvb2y3GJLrpFNbUUE9EPR6082rZpu-8q9iuuhe91k82jaB5UApBKv7AFU8Uf07lIUsXIda3mGTusaltCBs2B5Tu5roToR7PvlOebWVP5ufUb6UwYAr6cTDcURDwLV3wg1E2OUs6bW3LfECzmyVFlTUJ9lEm3StIYsK8kqslHroTvXBmUgZhg21BlI9uJfYLZZ69gMvk36spDwPArkSXNWqg",
  "expires_in": 300,
  "refresh_expires_in": 1800,
  "refresh_token": "eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIyMTI2NzU3ZC1jYTVhLTQwODMtOGIzOS0xMjlmZDFiODNmNGYifQ.eyJqdGkiOiIyMDY1ZWM5MC00MTkwLTRlNzYtOGIxNi1mNzBlNjE3MGY5NTQiLCJleHAiOjE1NzA4MDkyNTQsIm5iZiI6MCwiaWF0IjoxNTcwODA3NDU0LCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoiaHR0cDovL2tleWNsb2FrOjgwODAvYXV0aC9yZWFsbXMvU29mdHdhcmVIZXJpdGFnZSIsInN1YiI6IjI1ZTI3M2MwLTZkZjgtNDU2Ni1iYWRhLTc4MTcyMzRlNzdkOCIsInR5cCI6IlJlZnJlc2giLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6IjJlNjgwNzUwLWZlYjQtNDcwZC1hNTg0LWVjOGMwYTNhMWIzMyIsInJlc291cmNlX2FjY2VzcyI6eyJzd2gtd2ViLWFwaSI6eyJyb2xlcyI6WyJkZWZhdWx0IiwicGFydG5lci11c2VyIiwibm9ybWFsLXVzZXIiXX19LCJzY29wZSI6InN3aC1zZXJ2aWNlcyBlbWFpbCBwcm9maWxlIn0.gVN2wbq3tnGbBIn_T1Lg_LZpzu_nP0j1BV-ApS1mENs"
}
$ export TOKEN=<access_token>
$ curl -i -H 'Accept: application/json' -H "Authorization: Bearer ${TOKEN}" http://localhost:5004/api/1/stat/counters/HTTP/1.1 200 OK
Server: gunicorn/19.9.0
Date: Fri, 11 Oct 2019 15:25:42 GMT
Connection: keep-alive
Content-Type: application/json
Vary: Accept, Cookie
Allow: OPTIONS, GET, HEAD, OPTIONS
X-Frame-Options: SAMEORIGIN
Content-Length: 2

{}

We can now see the rate limiting headers are no more present as the johndoe user has the throttling-exempted permission.

To play with Keycloak realm configuration, you can log in as the admin user in the administration console reachable from http://localhost:5080/keycloak/auth/admin/SoftwareHeritage/console/index.html

The possibility to authenticate with an existing GithHub or GitLab account has also been added for testing purposes, either:

curl -X POST http://localhost:5004/api/1/auth/token/exchange/ -d "issuer=gitlab" -d "token=<gitlab_acces_token>"{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJURWdfR1o5d1FCXzdDZWk5bUZiYXJYemVJQXlBQ2tTSkVnY0x5Uk1WcVlnIn0.eyJqdGkiOiJmNzhlM2ViYy0xMjc2LTRjYzQtYjZkNi04YjdkN2Q2MDY3ZWMiLCJleHAiOjE1NzA4MDg4MjIsIm5iZiI6MCwiaWF0IjoxNTcwODA4NTIyLCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoic3doLXdlYi1hcGkiLCJzdWIiOiI1YzBhNjJiZS04OWEyLTQxMmQtYjExYy0zZDNjODMxMWRkZTIiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6IjNhMDY5NTFkLTg1ZTktNGNhNS1hMzI2LTc5NjM4ZWRjMjhiNCIsImFjciI6IjEiLCJyZXNvdXJjZV9hY2Nlc3MiOnsic3doLXdlYi1hcGkiOnsicm9sZXMiOlsiZGVmYXVsdCIsIm5vcm1hbC11c2VyIl19fSwic2NvcGUiOiJzd2gtc2VydmljZXMgZW1haWwgcHJvZmlsZSIsImVtYWlsX3ZlcmlmaWVkIjpmYWxzZSwibmFtZSI6IkFudG9pbmUgTGFtYmVydCIsInByZWZlcnJlZF91c2VybmFtZSI6ImFubGFtYmVydCIsImdpdmVuX25hbWUiOiJBbnRvaW5lIiwiZmFtaWx5X25hbWUiOiJMYW1iZXJ0IiwiZW1haWwiOiJhbnRvaW5lLmxhbWJlcnQzM0BnbWFpbC5jb20ifQ.QTv8yMgbScN2GsHwtsdbZjuQa7sg-dSsyL7oehQMGVsKRFBa9f9CskDczvfVPcIfgBr7LcGCfNxsYrO6yc83_OkiyEB9xed9vS5Kxnk9YY-iqVi11zlqEkj08o1hz0h-jUVmUHYugg3cK-XaFTlT2MywzCCZoNdf6DEtUl7f7u2BTLqWyWzpylnYvAI9UE86fNZmdZb233k7_cFe9eJ22nPkT5Muc7lys9-SZyNbF76349QKaoZTtfJ1FM4H9T0nDT-q0NIv8FcYVF43dXykiyUdMwZLD5G8-ARkMY8dfX5CAG5KmaaIX_CAZ__n3dJz80oGQDkWpnFSS4n0Sd-MeA","expires_in":300,"refresh_expires_in":1800,"refresh_token":"eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIyMTI2NzU3ZC1jYTVhLTQwODMtOGIzOS0xMjlmZDFiODNmNGYifQ.eyJqdGkiOiJjNjlmMDI2OC1iYmM1LTQ0ODgtYTY2OS1kOGMzY2Q4OGQ0OWQiLCJleHAiOjE1NzA4MTAzMjIsIm5iZiI6MCwiaWF0IjoxNTcwODA4NTIyLCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoiaHR0cDovL2tleWNsb2FrOjgwODAvYXV0aC9yZWFsbXMvU29mdHdhcmVIZXJpdGFnZSIsInN1YiI6IjVjMGE2MmJlLTg5YTItNDEyZC1iMTFjLTNkM2M4MzExZGRlMiIsInR5cCI6IlJlZnJlc2giLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6IjNhMDY5NTFkLTg1ZTktNGNhNS1hMzI2LTc5NjM4ZWRjMjhiNCIsInJlc291cmNlX2FjY2VzcyI6eyJzd2gtd2ViLWFwaSI6eyJyb2xlcyI6WyJkZWZhdWx0Iiwibm9ybWFsLXVzZXIiXX19LCJzY29wZSI6InN3aC1zZXJ2aWNlcyBlbWFpbCBwcm9maWxlIn0.4d5EkQoPMGGudLgCzfxg37VkhnlrgTTQ5_8v7kQCHK0"}

Depends on D2130

Related T2020

Diff Detail

Repository
rCDFD Dockerfiles for developers
Branch
keycloak
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 8474
Build 12295: arc lint + arc unit

Event Timeline

anlambert edited the summary of this revision. (Show Details)
anlambert retitled this revision from [POC] Add keycloack service and sample SoftwareHeritage realm to [POC] Add keycloak service and sample SoftwareHeritage realm.Oct 11 2019, 5:45 PM
olasd added inline comments.
docker-compose.yml
91

If that's really necessary, please map that to a 50xx port or, if possible, rather add a configuration to the nginx reverse proxy to give access to keycloak instead. Port 8080 will often be taken by other things on the host.

docker-compose.yml
91

Ack, I will try to improve that

Update: Do not expose 8080 port to host and configure nginx reverse proxy to expose keycloak as http://localhost:5080/keycloak