lyzadanger/auth.md

## auth.md

      
    Raw
  

              auth.md
            
          
    LMS-related endpoints: Authority, auth_clients, authentication, authorization, oh, my!

AuthClient and Auth’n Background

For most of our API endpoints, authentication is handled via token authentication. Every user has a unique API token (hint: you can find yours in the Settings (gear) -> Developer section from your home/activity page on h), so by sending a given API token in an Authorization: Bearer… HTTP header, an API request may be authenticated as the user who that token belongs to. Thus, an API client may retrieve groups for a given user or post an annotation as a given user. One specific user at a time.
But we also have a small set of endpoints that are only accessible with a different kind of authentication mechanism, which we call auth_client. auth_clients are special and more powerful and different: instead of being tied to a user, they are tied to an authority. This allows us to grant special privileges to known entities such that they can take actions on the set of users and resources within their assigned authorities—auth_clients are considerably more privileged than a single user/API token. A request with valid auth_client credentials may, for example, create a new user (within its authority) or add a user to a private group (both user and group must be within their associated authority).
(NB: Slightly confusingly, one registers  auth_client credentials in the Admin -> OAuth clients section of h, but it may be helpful to know that the auth  mechanism is not OAuth at all, but merely HTTPBasicAuth).
To authenticate users with API tokens, we have been using a Pyramid AuthenticationPolicy. In short, this is the “normal way to do things” in Pyramid and keeps authentication in one area of the app.
For auth_client -protected endpoints, however, there is logic invoked from within the views themselves to validate the auth_client credentials. That means it’s not true “authentication” per se as it happens at a different layer of the app than the “rest” of the authentication. Also, the pre-existing logic contains elements of authorization as well as authentication.
Why This Matters/Problems

OK, so why should we care? During the work to add features to support LMS users, we ran into a few limitations with the current setup:

Because the API token auth is “true” auth (AuthenticationPolicy) while the auth_client “auth” happened inside of views, there’s no way to combine the two if we want an endpoint that can be auth’d through either token auth or auth_client auth. We needed to do just that with the create-group endpoint. (Why? e.g. So that the existing create-group endpoint may continue to be used by individual users and also support the ability for auth_client requests to create private groups within their authorities).
We needed to extend the logic of the auth_client authentication to be able to take into account an X-Forwarded-User header, which allows auth clients to authenticate “on behalf of” users within their authority. (Why? So that we could invoke the create-group endpoint on behalf of an instructor user such that the instructor becomes the owner/creator of the group).

To address these issues, we created a new set of Authentication Policies for pyramid that moved the auth_client validation into the real authentication layer of the application. We also extended the AuthClientPolicy to allow for forwarded users. This new auth policy combination has been applied to the create-group endpoint and allows that endpoint to be used both by an API-token-user and an auth-client-with-forwarded-user.
Cool! Now we have a true pyramid Authentication Policy combination that we want to apply to more auth-client -allowed endpoints so that we can factor out and eliminate the redundant view-level logic.
Moving Along to Authorization now

But! The pre-existing view-level auth-client verification logic also contained some elements of authorization, not just authentication.  (And I’m being sloppy with terminology, because views really should be concerned with authorization, not authentication.)
Moving away from the view-level authn/authz code to the new AuthPolicy for the create-group endpoint didn’t raise any major issues because the authorization logic for that endpoint is very simple: any authenticated user may create a group (this approach is, in a kosher sense, incorrect, but I won’t elaborate right now as this treatise is already plenty long). And the group created therein is owned by the currently-authenticated user (and the group is created within the currently-authenticated user’s authority). So, indirectly, if inelegantly, we can be assured of an authority match. It works.
To use the new AuthPolicy and factor out the old view-level logic for the remaining endpoints for which auth_clients are applicable, we have some work to do, because there are some problems with the way our app currently handles authorization—some of which we know about but are now becoming blockers to progress.

Our traversal code ain’t quite right. And we know it. h.traversal.roots is heavily commented to point out that, to use Pyramid auth’z correctly, those need to return context objects, not models as they do now in most cases. To assign permissions correctly to be checked against in these views, we need to clean up the traversal for the affected resource(s) — most notably groups, which has an __acl__ (Access Control List, where permissions get assigned) in the model, which ain’t good. More pressingly, though, it makes it hard or maybe impossible to assign permissions at the root level (versus the context level). See h.traversal.roots.AnnotationRoot and h.traversal.contexts.AnnotationContext for how this is supposed to be constructed.
Our views are often checking against effective_principals instead of permission. This seems valid enough—but Pyramid considers effective_principals a “view predicate”—that is, if the check fails, the view won’t match the request at all, resulting in a 404 when you might well have wanted it to raise a 403 in that case. (A permission match failure will raise a 403). In the API views, these response codes are more important than in our web views, and they’re a bit borken at present, which is compounded by:
We conflate our 404 and 403 view handlers in our API. 403s are force-changed to 404s. You’ll never get a 403 out of the API. In some cases, this is good, and I imagine that this conflation was implemented to avoid leaking details about the existence of resources to un-authorized parties (i.e. if GET /annotation/foo returns a 403, you know it exists, which is bad). On the other hand, POST /groups returning a 404 is sort of nonsensical.

The above are existing code bits that need refactoring over time to support proper authorization and response codes.
tl;dr: Summary of what I propose

All of this long-windedness because I wanted to accomplish:

Apply new APIAuthPolicy (TokenAuth with fallback to AuthClient) to add-member-to-group  and create-user endpoints (auth’n). This is easy enough (a few lines of code), but needs to be orchestrated along with:
Make sure only the right folks can access those endpoints (auth’z). We’ll need a combination of roles and permissions that will require some remediation of the Groups and Users contexts. This is the biggest unknown.
Make sure that the resources involved in the API requests are contained within the associated authority of the requester (probably also auth’z). The “problem” is that the pre-existing view-level logic would ultimately return—to the view—an AuthClient model object, and the view could then check the authority on that model to make sure it matched the resource(s) being operated upon. Now all of the AuthClient checking is handled during the Authentication part of the request cycle and the view doesn’t have an immediate way to get its hands on the client’s authority. Propose for now adding a function to h.auth.util to return the current auth client’s authority. I can do this independently of step 2 and with minimal code change—I’ve already got some code ready. I imagine we’ll want to get more sophisticated (i.e. get this out of the view’s concern and into the proper auth’z level) but it may be too big of a chunk to bite off just yet.

After this is accomplished, I want to make POST /groups and other relevant API endpoints properly return a 403 instead of a 404 when authorization fails. This will require addressing some more permissions and refactoring the h.views.api.exceptions error handlers.