That diff enables to test Keycloak integration in swh-web using the docker environment.
A sample configuration for a Keycloak realm named SoftwareHeritage will be loaded when the new keycloak service will start.
In that realm, a client named swh-web-api has been created whose purpose is to protect the access to the Software Heritage Web API.
For that client, three roles have been created that can be associated to users:
- normal-user
- partner-user
- staff-user
Users with role partner-user or staff-user have the permission throttling-exempted associated while users
with role normal-user do not have any permissions.
In order to test users authentication and permissions in swh-web, a Python script will be executed when the
associated docker service will start. That script takes care of creating the following users in the realm:
- admin (password: admin): the realm administrator with role staff-user
- johndoe (password: johndoe-swh): a user with role partner-user
- janedoe (password: janedoe-swh): a user with role normal-user
In order to authenticate a user when making calls to the swh web api, proceed as follow:
- Get an access token by requesting the new endpoint /auth/token/access/ of the web api (let's test for the janedoe user)
$ curl -X POST http://localhost:5004/api/1/auth/token/access/ -d "username=janedoe" -d "password=janedoe-swh" | jq '.' { "access_token": "eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJURWdfR1o5d1FCXzdDZWk5bUZiYXJYemVJQXlBQ2tTSkVnY0x5Uk1WcVlnIn0.eyJqdGkiOiI0Y2U4NjRjNS00ZTI4LTQ3YjAtODdhYy1hNDUwZWUxYTdmYmIiLCJleHAiOjE1NzA4MDc0OTcsIm5iZiI6MCwiaWF0IjoxNTcwODA3MTk3LCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoic3doLXdlYi1hcGkiLCJzdWIiOiI2ZjExMWEwMy0zNTRiLTQ0NTctYmQyNS01MjAxZGUyMzlkMmIiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6ImY0ZmUxMzRkLWZiNWQtNDc1NC04ZGE4LWU2NjZhYzU4ZGZiMyIsImFjciI6IjEiLCJyZXNvdXJjZV9hY2Nlc3MiOnsic3doLXdlYi1hcGkiOnsicm9sZXMiOlsiZGVmYXVsdCIsIm5vcm1hbC11c2VyIl19fSwic2NvcGUiOiJzd2gtc2VydmljZXMgZW1haWwgcHJvZmlsZSIsImVtYWlsX3ZlcmlmaWVkIjpmYWxzZSwibmFtZSI6IkphbmUgRG9lIiwicHJlZmVycmVkX3VzZXJuYW1lIjoiamFuZWRvZSIsImdpdmVuX25hbWUiOiJKYW5lIiwiZmFtaWx5X25hbWUiOiJEb2UiLCJlbWFpbCI6ImphbmUuZG9lQGV4YW1wbGUub3JnIn0.fZCwPauPdNZeWrKxSgC9D8Chcxl2HSN_WaIrysQ7XH_ipePX04Skbscfd-uZsEKPYhR585Td9-5Eek-JkMxlbYTvEZi1wd1VuFk9xEH_dErs9Lh0paTztsCbhK9tOowl8TdcSWNpCG0E4p8eL9oXRHA07kb2P23cG0PAbXsg87B7f7NHB9ttKZCMAK4SERfh926UH2zTdSHAfIFqLyUqJ59i8mvdrB3W7yiDiMYYlmn7vsZ2uDtObYCtp35QZZlOuGp7ynRxbvSVHPOiJTjBOXoN74R4XEzSBbl3DGgtdWFRnmXv9VfEmn3VR9tl3rQMIq1IMRQ0q8Dl1IJK-iGw0A", "expires_in": 300, "refresh_expires_in": 1800, "refresh_token": "eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIyMTI2NzU3ZC1jYTVhLTQwODMtOGIzOS0xMjlmZDFiODNmNGYifQ.eyJqdGkiOiJhYjdiYjEzNi04NTg2LTQwOWUtYWUyNi02OWY4OTllMjRkNWIiLCJleHAiOjE1NzA4MDg5OTcsIm5iZiI6MCwiaWF0IjoxNTcwODA3MTk3LCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoiaHR0cDovL2tleWNsb2FrOjgwODAvYXV0aC9yZWFsbXMvU29mdHdhcmVIZXJpdGFnZSIsInN1YiI6IjZmMTExYTAzLTM1NGItNDQ1Ny1iZDI1LTUyMDFkZTIzOWQyYiIsInR5cCI6IlJlZnJlc2giLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6ImY0ZmUxMzRkLWZiNWQtNDc1NC04ZGE4LWU2NjZhYzU4ZGZiMyIsInJlc291cmNlX2FjY2VzcyI6eyJzd2gtd2ViLWFwaSI6eyJyb2xlcyI6WyJkZWZhdWx0Iiwibm9ybWFsLXVzZXIiXX19LCJzY29wZSI6InN3aC1zZXJ2aWNlcyBlbWFpbCBwcm9maWxlIn0.xLSHCv2zCkDd3u_AYtYX6tZbvzEkB7N2dK82tTnS9kE" }
- Then use the access token to perform authenticated calls to the web api
$ export TOKEN=<access_token> $ curl -i -H 'Accept: application/json' -H "Authorization: Bearer ${TOKEN}" http://localhost:5004/api/1/stat/counters/ HTTP/1.1 200 OK Server: gunicorn/19.9.0 Date: Fri, 11 Oct 2019 15:22:15 GMT Connection: keep-alive Content-Type: application/json Vary: Accept, Cookie Allow: OPTIONS, GET, HEAD, OPTIONS X-RateLimit-Limit: 120 X-RateLimit-Remaining: 119 X-RateLimit-Reset: 1570807365 X-Frame-Options: SAMEORIGIN Content-Length: 2 {}
We can see that the rate limiting headers are present in the api response as janedoe does not have any specific permission.
Let's perform the same operations with the johndoe user.
$ curl -X POST http://localhost:5004/api/1/auth/token/access/ -d "username=johndoe" -d "password=johndoe-swh" | jq '.' { "access_token": "eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJURWdfR1o5d1FCXzdDZWk5bUZiYXJYemVJQXlBQ2tTSkVnY0x5Uk1WcVlnIn0.eyJqdGkiOiIwOTc3N2Q3MS0xMzdmLTQyZjktYmJiZi1iYTkzNmQ5M2I1YTUiLCJleHAiOjE1NzA4MDc3NTQsIm5iZiI6MCwiaWF0IjoxNTcwODA3NDU0LCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoic3doLXdlYi1hcGkiLCJzdWIiOiIyNWUyNzNjMC02ZGY4LTQ1NjYtYmFkYS03ODE3MjM0ZTc3ZDgiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6IjJlNjgwNzUwLWZlYjQtNDcwZC1hNTg0LWVjOGMwYTNhMWIzMyIsImFjciI6IjEiLCJyZXNvdXJjZV9hY2Nlc3MiOnsic3doLXdlYi1hcGkiOnsicm9sZXMiOlsiZGVmYXVsdCIsInBhcnRuZXItdXNlciIsIm5vcm1hbC11c2VyIl19fSwic2NvcGUiOiJzd2gtc2VydmljZXMgZW1haWwgcHJvZmlsZSIsImVtYWlsX3ZlcmlmaWVkIjpmYWxzZSwibmFtZSI6IkpvaG4gRG9lIiwicHJlZmVycmVkX3VzZXJuYW1lIjoiam9obmRvZSIsImdpdmVuX25hbWUiOiJKb2huIiwiZmFtaWx5X25hbWUiOiJEb2UiLCJlbWFpbCI6ImpvaG4uZG9lQGV4YW1wbGUub3JnIn0.kbPLtlf2j6ZteG2xuoBXOs1kMQqYgK4eynIL3J3A31h5mAGYkVP7-ad_fwiTElRL7T_RLI-9Tu_MhOPTb5kjP5_sPdM1iFHwiXbA6rcVBCp4qTACFHDwAi6FQCzVXNFQvb2y3GJLrpFNbUUE9EPR6082rZpu-8q9iuuhe91k82jaB5UApBKv7AFU8Uf07lIUsXIda3mGTusaltCBs2B5Tu5roToR7PvlOebWVP5ufUb6UwYAr6cTDcURDwLV3wg1E2OUs6bW3LfECzmyVFlTUJ9lEm3StIYsK8kqslHroTvXBmUgZhg21BlI9uJfYLZZ69gMvk36spDwPArkSXNWqg", "expires_in": 300, "refresh_expires_in": 1800, "refresh_token": "eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIyMTI2NzU3ZC1jYTVhLTQwODMtOGIzOS0xMjlmZDFiODNmNGYifQ.eyJqdGkiOiIyMDY1ZWM5MC00MTkwLTRlNzYtOGIxNi1mNzBlNjE3MGY5NTQiLCJleHAiOjE1NzA4MDkyNTQsIm5iZiI6MCwiaWF0IjoxNTcwODA3NDU0LCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoiaHR0cDovL2tleWNsb2FrOjgwODAvYXV0aC9yZWFsbXMvU29mdHdhcmVIZXJpdGFnZSIsInN1YiI6IjI1ZTI3M2MwLTZkZjgtNDU2Ni1iYWRhLTc4MTcyMzRlNzdkOCIsInR5cCI6IlJlZnJlc2giLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6IjJlNjgwNzUwLWZlYjQtNDcwZC1hNTg0LWVjOGMwYTNhMWIzMyIsInJlc291cmNlX2FjY2VzcyI6eyJzd2gtd2ViLWFwaSI6eyJyb2xlcyI6WyJkZWZhdWx0IiwicGFydG5lci11c2VyIiwibm9ybWFsLXVzZXIiXX19LCJzY29wZSI6InN3aC1zZXJ2aWNlcyBlbWFpbCBwcm9maWxlIn0.gVN2wbq3tnGbBIn_T1Lg_LZpzu_nP0j1BV-ApS1mENs" } $ export TOKEN=<access_token> $ curl -i -H 'Accept: application/json' -H "Authorization: Bearer ${TOKEN}" http://localhost:5004/api/1/stat/counters/HTTP/1.1 200 OK Server: gunicorn/19.9.0 Date: Fri, 11 Oct 2019 15:25:42 GMT Connection: keep-alive Content-Type: application/json Vary: Accept, Cookie Allow: OPTIONS, GET, HEAD, OPTIONS X-Frame-Options: SAMEORIGIN Content-Length: 2 {}
We can now see the rate limiting headers are no more present as the johndoe user has the throttling-exempted permission.
To play with Keycloak realm configuration, you can log in as the admin user in the administration console reachable from http://localhost:5080/keycloak/auth/admin/SoftwareHeritage/console/index.html
The possibility to authenticate with an existing GithHub or GitLab account has also been added for testing purposes, either:
- through the browser by reaching that login page: http://localhost:5080/keycloak/auth/realms/SoftwareHeritage/account
- through the web api by exchanging a Github/Gitlab access token to a Keycloak one, for instance
curl -X POST http://localhost:5004/api/1/auth/token/exchange/ -d "issuer=gitlab" -d "token=<gitlab_acces_token>"{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJURWdfR1o5d1FCXzdDZWk5bUZiYXJYemVJQXlBQ2tTSkVnY0x5Uk1WcVlnIn0.eyJqdGkiOiJmNzhlM2ViYy0xMjc2LTRjYzQtYjZkNi04YjdkN2Q2MDY3ZWMiLCJleHAiOjE1NzA4MDg4MjIsIm5iZiI6MCwiaWF0IjoxNTcwODA4NTIyLCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoic3doLXdlYi1hcGkiLCJzdWIiOiI1YzBhNjJiZS04OWEyLTQxMmQtYjExYy0zZDNjODMxMWRkZTIiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6IjNhMDY5NTFkLTg1ZTktNGNhNS1hMzI2LTc5NjM4ZWRjMjhiNCIsImFjciI6IjEiLCJyZXNvdXJjZV9hY2Nlc3MiOnsic3doLXdlYi1hcGkiOnsicm9sZXMiOlsiZGVmYXVsdCIsIm5vcm1hbC11c2VyIl19fSwic2NvcGUiOiJzd2gtc2VydmljZXMgZW1haWwgcHJvZmlsZSIsImVtYWlsX3ZlcmlmaWVkIjpmYWxzZSwibmFtZSI6IkFudG9pbmUgTGFtYmVydCIsInByZWZlcnJlZF91c2VybmFtZSI6ImFubGFtYmVydCIsImdpdmVuX25hbWUiOiJBbnRvaW5lIiwiZmFtaWx5X25hbWUiOiJMYW1iZXJ0IiwiZW1haWwiOiJhbnRvaW5lLmxhbWJlcnQzM0BnbWFpbC5jb20ifQ.QTv8yMgbScN2GsHwtsdbZjuQa7sg-dSsyL7oehQMGVsKRFBa9f9CskDczvfVPcIfgBr7LcGCfNxsYrO6yc83_OkiyEB9xed9vS5Kxnk9YY-iqVi11zlqEkj08o1hz0h-jUVmUHYugg3cK-XaFTlT2MywzCCZoNdf6DEtUl7f7u2BTLqWyWzpylnYvAI9UE86fNZmdZb233k7_cFe9eJ22nPkT5Muc7lys9-SZyNbF76349QKaoZTtfJ1FM4H9T0nDT-q0NIv8FcYVF43dXykiyUdMwZLD5G8-ARkMY8dfX5CAG5KmaaIX_CAZ__n3dJz80oGQDkWpnFSS4n0Sd-MeA","expires_in":300,"refresh_expires_in":1800,"refresh_token":"eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIyMTI2NzU3ZC1jYTVhLTQwODMtOGIzOS0xMjlmZDFiODNmNGYifQ.eyJqdGkiOiJjNjlmMDI2OC1iYmM1LTQ0ODgtYTY2OS1kOGMzY2Q4OGQ0OWQiLCJleHAiOjE1NzA4MTAzMjIsIm5iZiI6MCwiaWF0IjoxNTcwODA4NTIyLCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoiaHR0cDovL2tleWNsb2FrOjgwODAvYXV0aC9yZWFsbXMvU29mdHdhcmVIZXJpdGFnZSIsInN1YiI6IjVjMGE2MmJlLTg5YTItNDEyZC1iMTFjLTNkM2M4MzExZGRlMiIsInR5cCI6IlJlZnJlc2giLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6IjNhMDY5NTFkLTg1ZTktNGNhNS1hMzI2LTc5NjM4ZWRjMjhiNCIsInJlc291cmNlX2FjY2VzcyI6eyJzd2gtd2ViLWFwaSI6eyJyb2xlcyI6WyJkZWZhdWx0Iiwibm9ybWFsLXVzZXIiXX19LCJzY29wZSI6InN3aC1zZXJ2aWNlcyBlbWFpbCBwcm9maWxlIn0.4d5EkQoPMGGudLgCzfxg37VkhnlrgTTQ5_8v7kQCHK0"}Depends on D2130
Related T2020