I am struggling finding which authentication method to pass in my API calls headers.
Basic authentication is failing and passing the /jupyterhub or /gitlab Bearer token doesn’t seem to do the trick.
The first method gives me a -32601 error while the other gives me a simpler 401 unauthorized
Furthermore, I can’t find the proper authentication API endpoint that may provide a proper token.
For reference, I am trying to use /api/renku/datasets.import on a privately deployed renku server.
What is the right way to authenticate my API calls ?
Hi @erwan.beguin if you are using the swagger page to test the API you need to authorize it with the PKCE flow:
This will fetch a JWT token from keycloak (assuming your deployment is using keycloak) for you using the swagger client and insert it into the API call headers. Any backend API calls sent to /api with the JWT in the header will get forwarded to the appropriate service with the appropriate credentials. This client (and the swagger page) were added relatively recently - which renku chart version are you using?
For some reason this isn’t working right now on renkulab.io/swagger (looking into it) but you can try it on dev.renku.ch/swagger (our development cluster). Note that you need to be logged in to renku normally in order for the swagger client to be able to get a token for you.
If you can’t get the PKCE grant to work, you can get a token from keycloak directly, but you need to have the client credentials. Do you have admin access to the keycloak that your renku instance is using?
I am building a custom connector between a Dataverse server and our renku platform to load datasets.
The ideal scenario would be for me to either have an admin token to manage all the API calls server side, or to let the user login with his renku credentials on my connector and get his token via another API endpoint.
Testing a token with swagger is not very usefull for me as I am using a production application.
I am using the current production version of Renku, and I can have admin access (at least my college, who deployed the platform, can).
Is there a keycloak endpoint to retrieve easily a Token with credentials?
Yes you can do that; you can find the endpoints on a URL similar to this one: https://renkulab.io/auth/realms/Renku/.well-known/openid-configuration - to get the tokens with a specific client you would use the token_endpoint and pass a request body that includes
grant_type: "password"
username: "username"
password: "password"
scope: "openid"
client_id: "renku"
client_secret: "secret"
You will also need to enable the “Direct access grant” for that client in Keycloak in order for this to work. However, it’s not ideal because users will have to trust you with their credentials. You should instead set up an additional client in keycloak under the Renku realm and have your application use this client to perform an oauth2 flow to obtain the JWT token.
PS: it’s really cool to learn that you are working on this! Please keep us informed of your progress and let us know what we can do to help move it along. There was also quite a bit of discussion on this topic already in Import from Dataverse · Issue #536 · SwissDataScienceCenter/renku-python · GitHub and Dataset Explore · Issue #5028 · IQSS/dataverse · GitHub - not sure if any of it is relevant for your case and the information there is probably pretty outdated by this point, but maybe it’s worth having a look.
Thank you. I will try setting up this flow shortly
I indeed read this GitHub discussion and based my preliminary tests on this.
I will keep you posted, this project takes place in the french public institution INRAE and the code will be open (before the end of the month)
I am confused about how to obtain the client_secret, is that a global config secret ? Or is it user owned ?
The client secret is set in keycloak at setup time, if you create the keycloak instance with our chart - see here. Once it’s installed, you can go to the admin panel in keycloak and go to Clients -> Renku -> Credentials to find it. If you prefer, you can also extract the renku secret from kubernetes and look under clients - there you will see the renku client defined with the client secret. Hope that makes sense… so many secrets! 
Auth0 has excellent resources on OAuth2, in particular their diagram of the authorization code flow might be helpful. Authorization Code Flow
Compared to their diagram, the “Auth0 tenant” would be the Renku Keycloak instance and “your API” is the actual Renku API.
I managed to get the API token endpoint working for a given user and retrieve a working access token.
However I am struggling passing that token to my next API calls. A authentication Bearer on headers yield :
{“error”: “authentication”, “message”: “unknown”, “target”: “renku”}
While any other form of headers authentication fail with a -32601 error
which API endpoint are you trying to use?
and how are you forming the header?
I am using the datasets.import endpoint and I am passing Authorization: Bearer on the headers.
So just to be sure - you are using POST /api/renku/datasets.import and passing Authorization: Bearer <JWT-token> as one of the headers - correct?
Has the user logged in to Renku itself? I’ve sometimes gotten errors like that when the gateway service (which swaps out the JWT for whatever other tokens are needed) doesn’t have the information on the user. Try logging the user in and repeating the request.
Also, can you try a simpler endpoint like GET /api/renku/cache.files_list?
Yes this is the exact header I am using, as well as the endpoint.
The user was indeed logged in renku. I tried logged in and logged out with the same result.
I get the same error with the files_list endpoint
Ok it could be that the token retrieved in that way is somehow different from the one we obtain automatically. I will try it out.
I just tried this again with a token I retrieved from the renku client using the Direct Access grant (i.e. using client id/secret and username/password). This gives me a JWT token which is then passed in the Authorization header and it works. Are you also trying it with the renku client or did you make a new client for this?
Two more things to look at: 1) check the JWT token at jwt.io 2) check the renku-gateway-auth service logs for errors.
Does a request to /api/user with the same authorization header work?
I apologize, there was something I didn’t understand.
I tried the token endpoint with ‘admin_cli’ as client_id to prevent using the client_secret. I believe it’s the reason it gave me the wrong access token.
But after I tried with renku as client_id with the right client secret, i get :
{‘error’: ‘unauthorized_client’, ‘error_description’: ‘Client not allowed for direct access grants’}
I believe it is referring to what you said here :
“You will also need to enable the “Direct access grant” for that client in Keycloak in order for this to work. However, it’s not ideal because users will have to trust you with their credentials. You should instead set up an additional client in keycloak under the Renku realm and have your application use this client to perform an oauth2 flow to obtain the JWT token.”
Could you elaborate on this, I am unsure of the way to go from here
No worries - by default the direct access grant (which allows retrieving JWTs based on username/password) is disabled for the renku client. But you can enable it in keycloak if you go to Clients --> renku --> Settings like here:
Here you can turn on the direct access grant. If you want to use your admin client, you can set it up in the same way and leave the default renku client untouched. However, in that case, you need to make sure to add the renku audience to the client as well - you can check the “Mappers” tab in the keycloak client settings for the renku or swagger clients to see how the renku audience is set up. That’s probably the reason the token was rejected (though we should arguably give a more informative error in that case).
While you can set up the client in this way to obtain user JWT tokens to authenticate the API, I wouldn’t recommend this as the final solution because you will need to be responsible for user’s renku credentials. Instead you should implement the standard Authorization Code flow that @andreas linked to above. This will involve you setting up a confidential client in keycloak under the Renku realm, adding to it the renku audience, and using it from your application to authenticate users with Renku. Users will get taken to the renku login screen and then return to your app, having logged in and authorized your app to make requests to the renku API on their behalf.
Thank you very much for your help !
I managed to have a working token and I will implement the Authorization code flow shortly
2 Likes
