Skip to content

Instantly share code, notes, and snippets.

@jijivishu
Last active April 2, 2024 18:15
Show Gist options
  • Save jijivishu/9b2d8e6426abd1cc005e3f61052523bf to your computer and use it in GitHub Desktop.
Save jijivishu/9b2d8e6426abd1cc005e3f61052523bf to your computer and use it in GitHub Desktop.
"Security: Bring CORS and CSP into core" proposal to Django Software Foundation for Google Summer of Code 2024.

GSOC Proposal: Security - Bring CORS and CSP into Django Core

Table of Contents

Abstract

Cross-Origin Resource Sharing (CORS) is a critical security feature implemented in web browsers to mitigate the risks associated with cross-origin HTTP requests. When a web page hosted on one origin makes a request to access resources (such as data, images, or APIs) hosted on a different origin, it triggers a cross-origin request. The primary purpose of CORS is to mitigate the risks associated with cross-origin requests, particularly those initiated by client-side scripts running in web browsers.

As of the current version, Django does not inherently include built-in support for Cross-Origin Resource Sharing (CORS). Consequently, if a client application hosted on a different domain attempts to access resources from a Django server, the browser may restrict the request due to CORS policy violations. While developers can implement CORS support manually using middleware or by customizing HTTP headers in views or middleware layers, it necessitates a deeper understanding of HTTP protocols and Django's middleware architecture to ensure secure and efficient CORS handling. Third party libraries like django-cors-headers do offer a range of features and functionalities aimed at simplifying the implementation and management of CORS policies, but they could increase complexity, bring in dependency management, reduce customisation and also increase performance overhead at times.

Content Security Policy (CSP) on the other hand, is a security mechanism that web developers use to mitigate data injection and cross-site scripting(XSS) attacks. CSP enables web administrators to define and enforce a set of rules specifying which resources, such as scripts, stylesheets, and images, are allowed to be loaded and executed on a web page. By controlling the origins from which content can be fetched and executed, CSP helps prevent unauthorized access to sensitive data and protects against malicious code injection.

As of current version of Django, CSP support is not offered built-in with Django. Thus, the servers that are developed using just the raw Django find themselves vulnerable to these attacks. There are no restrictions on where they can load their content from and thus can fall prey to injection attacks. The most reliable option to bring CSP into Django at the moment is django-csp. The third-party library maintained by mozilla offers dynamic policy generation, nonce support and per-view customisations. Still, it would be better to include the middleware in Django to reduce complexity of integration and performance overhead.

This proposal aims to implement CORS and CSP support within Django, following the design principles of existing security middlewares like CSRF protection. This would offer better flexibility, increase security of default Django-based applications and reduce dependency and performance concerns. By providing per-path customization through decorators, this enhancement will significantly improve the posture of Django applications.

Current Scenario

CORS Middleware

The go-to library for including CORS in a django project at the moment is django-cors-headers. The library is brought into any project as an app which contains a custom middleware CorsMiddleware. CorsMiddleware needs to be added before Django's in-built CommonMiddleware in MIDDLEWARE in settings of the project, in order to avoid overwriting of CORS headers in the response pipeline.

django-cors-headers uses three major configurations that need to be set in settings of the project. They would also serve as the basis of our configuration while bringing CORS into core. They are:

CORS_ALLOWED_ORIGINS : Sequence[str]

List of origins authorized to make HTTP-CORS requests, where null and file:// are considered a valid origin.

CORS_ALLOWED_ORIGIN_REGEXES : Sequence[str | Pattern[str]]

List of regex strings that match origins that are authorized to make HTTP-CORS request to the server.

CORS_ALLOW_ALL_ORIGIN : bool

This is similar to the wildcard CORS_ALLOWED_ORIGINS = [*] and overrides the other two settings. Some other configurations are used to offer customisations regarding resource sharing. Though not necessary to include, these settings come in handy to observe in-depth working of CORS. I will utilize these when writing the code for thd CORS middleware.

CORS_URLS_REGEX : str | Pattern[str]

List of regex strings that define URLs for which CORS headers would be sent. This setting comes in handy in case only a part of your site needs to utilize CORS.

CORS_ALLOW_METHODS : Sequence[str]

Tuple of the name of methods that are allowed to make CORS request to the server.

CORS_ALLOW_HEADERS : Sequence[str]

Tuple of non-standard HTTP headers that are allowed to be sent with the request.

CORS_EXPOSE_HEADERS : Sequence[str]

List of extra HTTP headers that are sent with response, and need to be exposed to the browser.

CORS_PREFLIGHT_MAX_AGE : int

The number of seconds browsers are allowed to cache the preflight response.

CORS_ALLOW_CREDENTIALS : bool

Credentials would be allowed to be sent via HTTP CORS requests, if set True.

CORS_ALLOW_PRIVATE_NETWORK : bool

Allow a site hosted on public IP to make CORS request to this server hosted on private IP, if set True.

Due to presence of Same-origin policy(SOP), bowsers utilize CORS to allow cross-origin access. What it drops down to, is sending the request with certain extra headers that realize whether the request is genuine or not. The requests are categorised as simple(GET, HEAD & POST) and non-simple.

For simple requests, browsers send a HTTP request with origin in header, and if that origin is allowed for CORS on the server, the server responds with 200-OK response along with access-control-allow-origin header set to either origin or * depending on the backend configuration. If the origin is not valid, no access-control-allow-origin would be present in the response. This is how django-cors-headers tackles it

        patch_vary_headers(response, ("origin",))

        origin = request.headers.get("origin")
        if not origin:
            return response

        try:
            url = urlsplit(origin)
        except ValueError:
            return response

        if (
            not conf.CORS_ALLOW_ALL_ORIGINS
            and not self.origin_found_in_white_lists(origin, url)
            and not self.check_signal(request)
        ):
            return response
            
        if conf.CORS_ALLOW_ALL_ORIGINS and not conf.CORS_ALLOW_CREDENTIALS:
            response[ACCESS_CONTROL_ALLOW_ORIGIN] = "*"
        else:
            response[ACCESS_CONTROL_ALLOW_ORIGIN] = origin

        if conf.CORS_ALLOW_CREDENTIALS:
            response[ACCESS_CONTROL_ALLOW_CREDENTIALS] = "true"

        if len(conf.CORS_EXPOSE_HEADERS):
            response[ACCESS_CONTROL_EXPOSE_HEADERS] = ", ".join(
                conf.CORS_EXPOSE_HEADERS
            )

For non-simple requests, a preflight OPTIONS HTTP request is first sent by the browser to the server. It contains origin, access-control-request-method, access-control-request-headers and other headers. A 204-NoContent response is returned by the server in case when the origin is allowed to send CORS request to the server. Suitable headers are attached with the response. This acts like a handshake before transfer of data. Through request methods specified in the header access-control-allow-methods. In case of invalid/ineligible origin, access-control-allow-origin is absent in the received response. This is how django-cors-headers tackels non-simple requests:

        if conf.CORS_ALLOW_ALL_ORIGINS and not conf.CORS_ALLOW_CREDENTIALS:
            response[ACCESS_CONTROL_ALLOW_ORIGIN] = "*"
        else:
            response[ACCESS_CONTROL_ALLOW_ORIGIN] = origin

        if conf.CORS_ALLOW_CREDENTIALS:
            response[ACCESS_CONTROL_ALLOW_CREDENTIALS] = "true"

        if len(conf.CORS_EXPOSE_HEADERS):
            response[ACCESS_CONTROL_EXPOSE_HEADERS] = ", ".join(
                conf.CORS_EXPOSE_HEADERS
            )
            
        if request.method == "OPTIONS":
            response[ACCESS_CONTROL_ALLOW_HEADERS] = ", ".join(conf.CORS_ALLOW_HEADERS)
            response[ACCESS_CONTROL_ALLOW_METHODS] = ", ".join(conf.CORS_ALLOW_METHODS)
            if conf.CORS_PREFLIGHT_MAX_AGE:
                response[ACCESS_CONTROL_MAX_AGE] = str(conf.CORS_PREFLIGHT_MAX_AGE)

        if (
            conf.CORS_ALLOW_PRIVATE_NETWORK
            and request.headers.get(ACCESS_CONTROL_REQUEST_PRIVATE_NETWORK) == "true"
        ):
            response[ACCESS_CONTROL_ALLOW_PRIVATE_NETWORK] = "true"

        return response

django-cors-headers also offers async-support and utilizes signals to offer more customization while allowing access. However, it no longer utilizes MiddlewareMixin since async support is enabled, which I will be utilizing in order to maintain the code consistency with other Django's built-in middlewares.

CSP Middleware

The go-to library to include Content Security Policy in a Django project is django-csp. It offers two middlewares which are CSPMiddleware and RateLimitedCSPMiddleware that needs to be added to settings of a project in order to utilize content-security-policy headers. With regards to hierarchy of middlewares in project settings, CSP is vey liberal and wherever it is placed doesn't make a difference.

Since CSP involves varieties of directives that could be sent over a http response via Content-Security-Policy header, django-csp offers ample attributes that could be configured in the settings of the project. It also offers four decorators for per-view customisations which are:

@csp_exempt()

Disables CSP header on the given view.

@csp_update()

Appends values to the source list specified in settings i.e. appends to policy.

@csp_replace()

Replaces value of directive mentioned in the argument from settings.

@csp

Overwrites the CSP configuration i.e. source list defined in settings completely.

Whatever security policy builds up after combining the configuration with the decorator output, is dynamically converted to a valid <directive> <value> string that is attached to the Content-Security-Policy header of the response. There is another configuration offered by name CSP_REPORT_ONLY in settings that allows only CSP reports to be sent and not the actual CSP to be activated on the site, which comes in handy when testing CSP for an old website. In that case, Content-Security-Policy header is replaced with Content-Security-Policy-Report-Only header in the response. RateLimitedCSPMiddleware comes in handy here to throttle number of reports.

Based on a similar methodology, I will be implementing Content Security Policy in core with exact same decorators, but with an extra option to mail the json blob report to project admins in case of vulnerability.

Approach

Following the design of Django's CSRF and clickjacking protection mechanisms, I propose to implement CORS and CSP support as middleware, ensuring compatibility with Django's HTTP request-response cycle. The middleware will offer safe defaults while allowing developers to customize settings on a per-path basis using decorators, maintaining Django's philosophy of flexibility and ease of use.

CORS Middleware Implementation

Write configuration settings at django/conf/global_settings.py

Global configuuration variables like CORS_ALLOW_SELECTED_ORIGINS would be added here to make them available throughout the project.

- Write CORSMiddleware at django/middleware/cors.py

Keeping in mind the standard design style of Django's in-built middlewares, I will be writing a CORSMiddleware class that inherits from the Django's in-built MiddlewareMixin class. This will inherit methods like process_request which are handy in reading request headers and adding headers to the response accordingly. I would also write code to allow import of configuration values like CORS_ALLOWED_ORIGINS directly from settings of the project. I would include every configuration provided by django-cors-headers that I mentioned above in Current Scenario - CORS. The middleware would allow both sync and async support and the implementation and designs would draw inspiration from CSRFMiddleware, GZIPMiddleware and django-cors-headers. The pseudo code for which looks something like this:

from django.utils.deprecation import MiddlewareMixin

# Define header names as global strings for handy access, for example:
ACCESS_CONTROL_REQUEST_METHOD = "access-control-request-method"

def CORSMiddleware(MiddlewareMixin):
        def process_request(self, request):
                ### Only comes into play for non-simple requests i.e. if the request is HTTP OPTIONS
                # Check whether the origin matches any origin defined in project 
                  settings(both regex and allowed origins) 
                # [omit previous step for wildcard i.e. when 
                  CORS_ALLOW_ALL_ORIGIN is True in project settings] 
                # Previous step could be omitted if True is returned by a 
                  signal handler when the current request is sent via signals
                # Check if the request method is preflight OPTIONS and
                # Check if ACCESS_CONTROL_REQUEST_METHOD header is present in the request
                # Confirm that related decorators do not offer hinderance to the origin in request 
                
                # When all the above three conditions are met, prepare an empty HTTPResponse 
                  with `content-length` header set to "0" and return it
                # If any condition fails, return None
                
         
        def process_response(self, response):
                ### Check validity of request based on decarators used, and in case of 
                    invalidity, return empty response from here only
                    instead of entering process_response.
                
        def process_response(self, response):
                ### Set the headers on response object depending on the request and settings of the project
                
                # Check whether the origin matches any origin defined in project settings(both regex and allowed origins)
                # [omit previous step for wildcard i.e. when CORS_ALLOW_ALL_ORIGIN is True in project settings]
                # Previous step could be omitted if True is returned by a signal 
                  handler when the current request is sent via signals
                # If negative, return None
                # If positive, add `vary: origin` to response
                
                # Try splitting origin and perform other header related operations and 
                  add response headers depending on the request. 
                # If splitting fails, return Response.
                # If at the end, no header causes issues, set access-control-allow-origin to origin in the response 
                # Note: I have option to set access-control-allow-origin to * in response 
                  whenever wildcard is activated in settings,
                # but I am avoiding it as it causes issues if access-control-allow-credentials 
                  header is set to True in response
                
       # Note that every utility function in question will have two occurence, both sync and async.             

- Write decorators at django/views/decorators/cors.py

The decorators are handy when our end user aims to customise middleware on a per-view or per-path basis. This was one major offer missing in the django-cors-headers. At the moment, I am looking forward to add four decorators which would be:

@cors_exempt()

This decorator would be used to exempt specific views from CORS checks.

@cors_allow_all_origins()

This decorator would allow all origins to make CORS request to a specific view.

@cors_restrict_credentials()

This decorator would restrict cookies for the specific view.

@cors_custom(allow_origin='https://example.com', allow_credentials=True, allow_methods='GET, POST')

This decorator is the most versatile one, which will offer the developer the flexibility to specify origins, allow credentials and allow methods for a specific view. Whichever argument isn't mentioned explicitly by the user at the time of decoration, would source it's default value from the settings of the project.

These decorators would be written in django/views/decorators/cors.py and would utilize Django's built-in decorator_from_middleware, view_func and _view_wrapper functions for abstraction. The file would draw it's inspiration from django/views/decorators/csrf.py. This being quoted, I am open to any suggestions to add/edit/remove decorators throughout the GSOC'24 tenure.

- Write CORSMiddlewareTest at tests/middleware/tests.py

Adhering to Djnango's philosophy of writing extensive tests, I will be writing comperhensive tests inspired from django-cors-headers which would cover desired and edge conditions for every header and configuration that I would be including in the CORSMiddleware. I will write class CORSMiddlewareTest in tests/middleware/tests.py which would inherit from built-in SimpleTestCase class. A simple pseudo code demostrating one of those tests looks like:

from django.middleware.cors import CORSMiddleware, ACCESS_CONTROL_ALLOW_ORIGIN
from django.test import RequestFactory, SimpleTestCase, override_settings

@override_settings(ROOT_URLCONF="middleware.urls")
class CORSMiddlewareTest(SimpleTestCase):
        client = RequestFactory()
        
        @override_settings(CORS_ALLOWED_ORIGINS=["https://example.org"])
        def test_get_not_in_allowed_origins_due_to_wrong_scheme(self):
                ### Two origins with similar netloc but different schemes need to be considered different origins.
                resp = self.client.get("/", HTTP_ORIGIN="http://example.org")
                assert ACCESS_CONTROL_ALLOW_ORIGIN not in resp

- Write decorator-specific tests at tests/decorators/test_cors.py

Corresponding to the four decorators I proposed above, I would be writing four separate test-classes at tests/decorators/test_cors.py to comprehensively cover the working of decorators. These classes would inherit from CORSTestMixin(to be written by me) and SimpleTestCase(django's in-built). The structure of them would look like:

from django.http import HttpRequest, HttpResponse
from django.test import SimpleTestCase
from django.middleware.cors import CORSMiddleware, ACCESS_CONTROL_ALLOW_ORIGIN, ACCESS_CONTROL_ALLOW_CREDENTIALS, ACCESS_CONTROL_ALLOW_METHODS

DEFAULT_CORS_SETTINGS = {
    'allow_origin': 'http://example.com',
    'allow_credentials': False,
    'allow_methods': 'GET, POST',
}

class CorsHeadersTests(SimpleTestCase):
    def test_cors_headers_decorator(self):
        @cors_headers(**DEFAULT_CORS_SETTINGS)
        def view(request):
            return HttpResponse()

        middleware = CORSMiddleware()
        request = HttpRequest()
        request.method = "GET"
        middleware.process_request(request)

        response = view(request)
        
        # Test that CORS headers are set correctly
        self.assertEqual(response[ACCESS_CONTROL_ALLOW_ORIGIN], DEFAULT_CORS_SETTINGS['allow_origin'])
        
        if DEFAULT_CORS_SETTINGS['allow_credentials']:
                self.assertEqual(response[ACCESS_CONTROL_ALLOW_CREDENTIALS], 'true')
        else:
                self.assertNotIn(ACCESS_CONTROL_ALLOW_CREDENTIALS, response)
                
        if request.method in DEFAULT_CORS_SETTINGS['allow_methods']:   
                allowed_methods = DEFAULT_CORS_SETTINGS['allow_methods'].split(', ')
                self.assertEqual(response[ACCESS_CONTROL_ALLOW_METHODS], ', '.join(allowed_methods))
        else:
                self.assertNotIn(ACCESS_CONTROL_ALLOW_METHODS, response)

    # More tests like no credentials, no methods, etc. will be added here, in their corresponding decorator classes

CSP Middleware Implementation

Write configuration settings at django/conf/global_settings.py

Global configuration variables including all directive settings like CSP_DEFAULT_SRC would be located here.

Write CSPMiddleware at django/middleware/csp.py

Keeping in mind the standard design style of Django's in-built middlewares, I will be writing a CSPMiddleware class that inherits from the Django's in-built MiddlewareMixin class. This inherits methods like process_request which are handy in reading request headers and adding headers to the response accordingly. The pseudo code for the middleware would be close to:

def CSPMiddleware(MiddlewareMixin):
  # Check for @cors_exempt decorator, which if True, return from here only
  # Check for ignored path prefix i.e. settings.CSP_EXCLUDE_URL_PREFIXES, 
    which if matches request origin, return from here only
  # Check for internal server error or client not found through response code, 
    and if DEBUG is activated, return response from here only
  # Add "Content-Security-Policy" or "Content-Security-Policy-Report-Only" header 
    to the response depending on the configuration.
  # Set it's value to policy which is built dynamically based on configurration and the decorators.
  # Return the response 

Write decorators at django/views/decorators/cors.py

Exact decorators as used in django-csp will be written here, since they serve the functionality well.

Write context processors at django/template/context_processors.py

Custom context processor written here will allow sharing CSP_NONCE throughout the request/response pipeline.

Write CSPMiddlewareTest at tests/middlewares/tests.py

Comprehensive tests covering configuration combinations will be written in CSPMiddlewareTest class here.

Write decorator-specific tests at tests/decorators/test_csp.py

Decorator specific tests would be written for all four decorators, similar to those for CORS written above

Schedule

Midterm Evaluation (CORS Middleware)

I will implement CORS middleware section in the first half of GSOC which is from May 27 to July 8. The rough schedule for it is:
NOTE: Only large classes and functions are mentioned here, utility classes and functions will be added alongside when writing the code.

Week 1 (May 27 - June 2)

Create new file django/middleware/cors.py and write the code for CORSMiddleware in it.

Week 2 (June 3 - June 9)

Create new file django/views/decorators/cors.py and write the code for decorators proposed above.

Week 3 & 4 (June 10 - June 23)

Create new file tests/middleware/tests.py and write code for CORSMiddlewareTest class which covers the middleware written in Week1 in depth.

Week 5 (June 24 - June 30)

Create new file tests/decorators/test_cors.py and write decorator specific test-classes for each decorator proposed.

Week 6 (July 1 - July 7)

Buffer week to complete the implementation of CORS middleware, write documentations and accomplish any changes proposed by the mentor/community.

Midterm Evaluation submission for CORS (July 8 - July 12)

Submit the evaluation for CORS middleware.

Final Evaluation (CSP Middleware)

Week 7 (July 15 - July 21)

Create new file django/middleware/csp.py and write the code for CSPMiddleware in it.

Week 8 (July 22 - July 28)

Create new file django/views/decorators/csp.py and write the code for the decorators proposed above.

Week 9 (July 29 - August 4)

Write context processor in django/template/context_processors.py and facilitate availability of CSP_NONCE throughout the application.

Week 10 (August 5 - August 11)

Create new file tests/middleware/tests.py and write code for CSPMiddlewareTest class which covers the middleware written in Week7 in depth.

Week 11 (August 12 - August 18)

Create new file tests/decorators/test_csp.py and write decorator specific test-classes for each decorator proposed.

August 19

Final submission i.e. merger of two PRs into django

About Me

I am Kunwar Kuldeep Srivastava (Kuldeep), a 2023 graduate in Electronics and Communication engineering from University of Allahabad(India). I took Harvard's infamous CS50W: Web Programming with Python and Javascript last summer, and since then I have been working on Django projects and Django Rest Framework. What was initially supposed to be a utility framework casually started capturing my interst into it's core, to observe how the framework works under the hood. And with that interest, I write this proposal to give my 2 cents in further enhancing the capability of Django.

I have read about the focus on choosing a known-individual for this project, and I agree that I am anything but a known-individual to the Django community, well evident from the the fact that I am yet to make my first PR. However, I aim to change that asap and start contributing to the community from this week itself, while the proposals are being reviewed. I have done in-depth analysis of how middlewares work in Django, how the request/response pipeline works, and how custom middlewares like django-cors-headers and django-csp hook themselves to the pipeline. Thus, I feel confident to execute the proposed project under the given timeline and look forward to discuss about it with the community/mentor.

I go by the nickname jijivishu on the internet, and am open to any communications through email or discord with the preferred language being English. My timezone is UTC + 5:30(Asia/Kolkata).
Cheers!

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment