Page MenuHomeSoftware Heritage

cli: Add auth command group
ClosedPublic

Authored by anlambert on Mar 20 2020, 5:34 PM.

Details

Summary

Add a CLI tool to easily retrieve bearer tokens for Web API authentication.

$ swh auth
Usage: swh auth [OPTIONS] COMMAND [ARGS]...

  Authenticate Software Heritage users with OpenID Connect.

  This CLI tool eases the retrieval of bearer tokens to authenticate a user
  querying the Software Heritage Web API.

Options:
  --oidc-server-url TEXT  URL of OpenID Connect server (default to
                          "https://auth.softwareheritage.org/auth/")
  --realm-name TEXT       Name of the OpenID Connect authentication realm
                          (default to "SoftwareHeritage")
  --client-id TEXT        OpenID Connect client identifier in the realm
                          (default to "swh-web")
  -h, --help              Show this message and exit.

Commands:
  login    Login and create new offline OpenID Connect session.
  logout   Logout from an offline OpenID Connect session.
  refresh  Refresh an offline OpenID Connect session.

The proposed workflow to authenticate API calls is the following.

When a user has an account on the swh Identity Provider server (Keycloak), he can
login using this tool and get its authentication tokens to send in HTTP headers
when querying the Web API.

$ swh auth login johndoe | jq
Password: 
{
  "access_token": "eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJPSnhVQ0p0TmJQT0NOUGFNNmc3ZU1zY2pqTXhoem9vNGxZaFhsa1c2TWhBIn0.eyJqdGkiOiI3NjMzNTAzZi04NGFjLTRmZTMtOGRmNS01ZTQ4MTAxZjAwMmIiLCJleHAiOjE1ODQ3MjE3NTAsIm5iZiI6MCwiaWF0IjoxNTg0NzIxMTUwLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvYXV0aC9yZWFsbXMvU29mdHdhcmVIZXJpdGFnZSIsImF1ZCI6WyJzd2gtd2ViIiwiYWNjb3VudCJdLCJzdWIiOiJmZWFjZDM0NC1iNDY4LTRhNjUtYTIzNi0xNGY2MWU2YjcyMDAiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJzd2gtd2ViIiwiYXV0aF90aW1lIjowLCJzZXNzaW9uX3N0YXRlIjoiYzE0ZTFiN2ItODI2My00ODUyLWJkMWMtYWRjN2JjMTJhMTM2IiwiYWNyIjoiMSIsImFsbG93ZWQtb3JpZ2lucyI6WyIqIl0sInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJvZmZsaW5lX2FjY2VzcyIsInVtYV9hdXRob3JpemF0aW9uIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsic3doLXdlYiI6eyJyb2xlcyI6WyJ0aHJvdHRsaW5nLWV4ZW1wdGVkIl19LCJhY2NvdW50Ijp7InJvbGVzIjpbIm1hbmFnZS1hY2NvdW50IiwibWFuYWdlLWFjY291bnQtbGlua3MiLCJ2aWV3LXByb2ZpbGUiXX19LCJzY29wZSI6Im9wZW5pZCBlbWFpbCBwcm9maWxlIG9mZmxpbmVfYWNjZXNzIiwiZW1haWxfdmVyaWZpZWQiOmZhbHNlLCJuYW1lIjoiSm9obiBEb2UiLCJncm91cHMiOlsiL3BhcnRuZXJzIl0sInByZWZlcnJlZF91c2VybmFtZSI6ImpvaG5kb2UiLCJnaXZlbl9uYW1lIjoiSm9obiIsImZhbWlseV9uYW1lIjoiRG9lIiwiZW1haWwiOiJqb2huLmRvZUBleGFtcGxlLmNvbSJ9.hFeLjlxkj3OzotJDwbuIZrSePmJdxRnxMTAla_nhgTnKdmY4TcMxyA2gaAG3ig7sSKbO3s86XCsn4B943TygLgS_YXYkpulQdk3bwUTu4zISE2zBtB2NFUyFie05Zxq3Z6zrlAw9TAApq4Ig0VnS5axpYdiQaXiTif4bLPZzxVRJ5NTHZJjjXjv7XUXhbH62Su0xd7FdUaxWMvCFrEY3ONiFxZ9v8n_jZNYt_1A1qPuR-LvPhytu86xOq7IImKfyrRcfAfYHUDmA4S5XduE2viIDNA_0p2bUuYwQwhjfHbv7IASAGquW6DqA6hAJp4dafnPyiRUDdlwtWVypo-ttMw",
  "expires_in": 600,
  "refresh_expires_in": 0,
  "refresh_token": "eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJmNjMzMDE5MS01YTU4LTQxMDAtOGIzYS00ZDdlM2U1NjA3MTgifQ.eyJqdGkiOiI2ZjU5ZWFkNS05OWIzLTRiMWUtYWM3My0xNTRmYzc2NmNjYWIiLCJleHAiOjAsIm5iZiI6MCwiaWF0IjoxNTg0NzIxMTUwLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvYXV0aC9yZWFsbXMvU29mdHdhcmVIZXJpdGFnZSIsImF1ZCI6Imh0dHA6Ly9sb2NhbGhvc3Q6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwic3ViIjoiZmVhY2QzNDQtYjQ2OC00YTY1LWEyMzYtMTRmNjFlNmI3MjAwIiwidHlwIjoiT2ZmbGluZSIsImF6cCI6InN3aC13ZWIiLCJhdXRoX3RpbWUiOjAsInNlc3Npb25fc3RhdGUiOiJjMTRlMWI3Yi04MjYzLTQ4NTItYmQxYy1hZGM3YmMxMmExMzYiLCJyZWFsbV9hY2Nlc3MiOnsicm9sZXMiOlsib2ZmbGluZV9hY2Nlc3MiLCJ1bWFfYXV0aG9yaXphdGlvbiJdfSwicmVzb3VyY2VfYWNjZXNzIjp7InN3aC13ZWIiOnsicm9sZXMiOlsidGhyb3R0bGluZy1leGVtcHRlZCJdfSwiYWNjb3VudCI6eyJyb2xlcyI6WyJtYW5hZ2UtYWNjb3VudCIsIm1hbmFnZS1hY2NvdW50LWxpbmtzIiwidmlldy1wcm9maWxlIl19fSwic2NvcGUiOiJvcGVuaWQgZW1haWwgcHJvZmlsZSBvZmZsaW5lX2FjY2VzcyJ9.h8AxJU1_ouNrwnz6NKKx8AKBu2UbtOu1riP4Jil7XWU",
  "token_type": "bearer",
  "id_token": "eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJPSnhVQ0p0TmJQT0NOUGFNNmc3ZU1zY2pqTXhoem9vNGxZaFhsa1c2TWhBIn0.eyJqdGkiOiIwZGIwMmM5Zi0yZGJlLTRiMjgtOTYzNS1lMGVhNjliMWM3NjEiLCJleHAiOjE1ODQ3MjE3NTAsIm5iZiI6MCwiaWF0IjoxNTg0NzIxMTUwLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvYXV0aC9yZWFsbXMvU29mdHdhcmVIZXJpdGFnZSIsImF1ZCI6InN3aC13ZWIiLCJzdWIiOiJmZWFjZDM0NC1iNDY4LTRhNjUtYTIzNi0xNGY2MWU2YjcyMDAiLCJ0eXAiOiJJRCIsImF6cCI6InN3aC13ZWIiLCJhdXRoX3RpbWUiOjAsInNlc3Npb25fc3RhdGUiOiJjMTRlMWI3Yi04MjYzLTQ4NTItYmQxYy1hZGM3YmMxMmExMzYiLCJhY3IiOiIxIiwiZW1haWxfdmVyaWZpZWQiOmZhbHNlLCJuYW1lIjoiSm9obiBEb2UiLCJncm91cHMiOlsiL3BhcnRuZXJzIl0sInByZWZlcnJlZF91c2VybmFtZSI6ImpvaG5kb2UiLCJnaXZlbl9uYW1lIjoiSm9obiIsImZhbWlseV9uYW1lIjoiRG9lIiwiZW1haWwiOiJqb2huLmRvZUBleGFtcGxlLmNvbSJ9.i5QcmvmAYv4m26mrMQZi_FUuKNqg3I9yN8IdJ2nGKS-1q7jjJkjfVtOut6cdQ_V4UY7X9U7p4bXN3TRReOKn8dHsSlPwQpfzP9r6BimPAOFMS5yw-QAgOiLWFPtD8jlFwNCKNC5mT5iftTv-e9nEAGGgrFbgmsieIhVJzYb4sn8UvjX2YikHn2emYnzobrv_7hTHYOr6JXouzNKiziNOdEB1K_H4zAngfidXexZoQa872x_z0XpEYwZmWYc1PScGsAwc1qSQLz014StytnNiKcOjbUX8fQ64xT7xkf3QPiaye0usVzCNmh3kURfF3aW1Kmhax-AUKDItzO6fvHWtPQ",
  "not-before-policy": 1584551170,
  "session_state": "c14e1b7b-8263-4852-bd1c-adc7bc12a136",
  "scope": "openid email profile offline_access"
}

The access token must be sent in HTTP headers to authenticate requests to Web API.
It has a short living period (usually 10 minutes) and must be renewed regularly.

To renew an access token, the refresh token has to be used by sending it to the Keycloak
server in a refresh request. This token has a longer living period and can be
used at anytime to get a new access token without the need to login again.

The refresh operation will be automatically handled if you use the swh.web.client.WebAPIClient
class (diff incoming for that feature).

For users not working with Python, they can get a new access token through this command:

$ swh auth refresh $TOKEN | jq
"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJPSnhVQ0p0TmJQT0NOUGFNNmc3ZU1zY2pqTXhoem9vNGxZaFhsa1c2TWhBIn0.eyJqdGkiOiI1ZTJiZWJmMS1hNDhhLTRlYjAtODBjNi01MWM3MTlhOWNjZjUiLCJleHAiOjE1ODQ3MjIyMDksIm5iZiI6MCwiaWF0IjoxNTg0NzIxNjA5LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvYXV0aC9yZWFsbXMvU29mdHdhcmVIZXJpdGFnZSIsImF1ZCI6WyJzd2gtd2ViIiwiYWNjb3VudCJdLCJzdWIiOiJmZWFjZDM0NC1iNDY4LTRhNjUtYTIzNi0xNGY2MWU2YjcyMDAiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJzd2gtd2ViIiwiYXV0aF90aW1lIjowLCJzZXNzaW9uX3N0YXRlIjoiYzE0ZTFiN2ItODI2My00ODUyLWJkMWMtYWRjN2JjMTJhMTM2IiwiYWNyIjoiMSIsImFsbG93ZWQtb3JpZ2lucyI6WyIqIl0sInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJvZmZsaW5lX2FjY2VzcyIsInVtYV9hdXRob3JpemF0aW9uIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsic3doLXdlYiI6eyJyb2xlcyI6WyJ0aHJvdHRsaW5nLWV4ZW1wdGVkIl19LCJhY2NvdW50Ijp7InJvbGVzIjpbIm1hbmFnZS1hY2NvdW50IiwibWFuYWdlLWFjY291bnQtbGlua3MiLCJ2aWV3LXByb2ZpbGUiXX19LCJzY29wZSI6Im9wZW5pZCBlbWFpbCBwcm9maWxlIG9mZmxpbmVfYWNjZXNzIiwiZW1haWxfdmVyaWZpZWQiOmZhbHNlLCJuYW1lIjoiSm9obiBEb2UiLCJncm91cHMiOlsiL3BhcnRuZXJzIl0sInByZWZlcnJlZF91c2VybmFtZSI6ImpvaG5kb2UiLCJnaXZlbl9uYW1lIjoiSm9obiIsImZhbWlseV9uYW1lIjoiRG9lIiwiZW1haWwiOiJqb2huLmRvZUBleGFtcGxlLmNvbSJ9.J8BTceH3VrfDLw677CLys7cNZgiyaRC0_ewtegn_thdFzv1CwDULKnTIZHb-o_nBXhpA-hEaMUnu-tsE3yleCqMuKLjBq6c9kt2bVGvyR3-aIcbBb5gX6KzJPxPSBfvz-mQzjpuQMfswPMrYkDm--fQa3KgwuHc5CDs15rPSINKH7YZLkVbi4rIQ--7DhbqM3F4NPIUrTIAb7pSAIe2N9XzSZHc0ZbWbq7-NkxjOL-fOeI8oqPbQkFDIq7vzTcGYiiI9VJoIIxTHtkTYJHv6RdU3Yj2u0-im7P-_8em0waaHy1zAq2SBzXSKpadlQ22k6yiBrcQ6gNov0w896nVcPA"

It is also possible to logout from the authenticated session which invalidates all previously
emitted tokens.

$ swh auth logout $TOKEN
Successfully logged out from OpenID Connect session

Diff Detail

Repository
rDWCLI Web client
Branch
add-api-auth-feature
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 11260
Build 17015: tox-on-jenkinsJenkins
Build 17014: arc lint + arc unit

Event Timeline

Update: add missing click depdendency and update commit message

Update: add missing pytest-mock dependency

vlorentz added a subscriber: vlorentz.

Could you write this Diff's description in the README?

swh/web/client/auth.py
75

why doesn't this one request offline_access?

92

same

swh/web/client/cli.py
17

The group should be named "authentication", as it's not a single action

39

You should first use ctx.ensure_object()

44

Why the session- prefix?

47

if you want the session- prefix, then the function name should be session_login

69

print(json.dumps(resp_json, indent=4, sort_keys=True)) to be developer-readable

71

Hiding the traceback makes debugging harder :/

And we should exit with non-0 in case of error.

88

same

91

same

93

same

111

same

swh/web/client/tests/test_cli.py
26–27

You can use the input= argument of runner.invoke instead

56

needs a test of failure for each command

This revision now requires changes to proceed.Mar 20 2020, 6:08 PM

Thanks for the review !

Could you write this Diff's description in the README?

I will create a dedicated page in the module documentation.

swh/web/client/auth.py
75

The offline_access is only needed when logging in. It informs Keycloak that an offline token should be generated for the session. This is a special type of refresh token with longer living period.

swh/web/client/cli.py
17

ack

39

ack

44

To explicit the fact that it opens an authenticated session on OIDC server.

But I agree shorter is better and the doc give the details, I will remove the prefix.

69

ack

71

Can I just remove the try/except block then ?

swh/web/client/tests/test_cli.py
26–27

nice, thanks

56

ack

swh/web/client/cli.py
71

I guess. Or catch specific expected errors, and let everything raise to the toplevel

anlambert added inline comments.
swh/web/client/cli.py
71

Ok will remove the blocks then.

swh/web/client/tests/test_cli.py
26–27

Unfortunately this does not seem to play well with getpass, providing input='password\n' still makes the prompt appear. Let's keep this mock then.

Update: Rebase and address @vlorentz comments, still the documentation to write

anlambert edited the summary of this revision. (Show Details)

Update: restore default oidc-server-url value

anlambert retitled this revision from cli: Add authenticate command groups to cli: Add authentication command groups.Mar 20 2020, 7:40 PM
vlorentz added inline comments.
swh/web/client/tests/test_cli.py
26–27

indeed, getpass, reads from the tty

40

can remove this.

This revision is now accepted and ready to land.Mar 20 2020, 7:55 PM

Update: Rebase and add CLI tool documentation in a new Authentication section located in docs/index.rst

nice.

swh/web/client/cli.py
42

you can remove that pass ;)

you've got a free pass for that one :D

89

maybe add a output_json function (or something) to avoid the repetition?

I saw this at least thrice already.

swh/web/client/cli.py
42

Whoopsie, thanks for spotting !

89

Agreed !

the doc LGTM, but swh authentication feels like a mouthful, would it be possible to use swh auth instead? (which will also work for both "authorization", that I suspect is something we will also put behind this at some point)

In D2861#69178, @zack wrote:

the doc LGTM, but swh authentication feels like a mouthful, would it be possible to use swh auth instead? (which will also work for both "authorization", that I suspect is something we will also put behind this at some point)

Ack, I will update accordingly.

Update: Rename command group from authentication to auth

anlambert retitled this revision from cli: Add authentication command groups to cli: Add auth command group.Mar 24 2020, 11:57 AM
anlambert edited the summary of this revision. (Show Details)
This revision was automatically updated to reflect the committed changes.