Model: claude-3-5-sonnet-20240620
./pyproject.toml
---
[project]
name = "django-http-debug"
version = "0.2"
description = "Django app for creating database-backed HTTP debug endpoints"
readme = "README.md"
requires-python = ">=3.8"
authors = [{name = "Simon Willison"}]
license = {text = "Apache-2.0"}
classifiers = [
"License :: OSI Approved :: Apache Software License"
]
dependencies = [
"filetype",
"django"
]
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project.urls]
Homepage = "https://github.com/simonw/django-http-debug"
Changelog = "https://github.com/simonw/django-http-debug/releases"
Issues = "https://github.com/simonw/django-http-debug/issues"
CI = "https://github.com/simonw/django-http-debug/actions"
[project.optional-dependencies]
test = ["pytest"]
---
./README.md
---
# django-http-debug
[](https://pypi.org/project/django-http-debug/)
[](https://github.com/simonw/django-http-debug/actions/workflows/test.yml)
[](https://github.com/simonw/django-http-debug/releases)
[](https://github.com/simonw/django-http-debug/blob/main/LICENSE)
Django app for creating database-backed HTTP debug endpoints
## Installation
Install this library using `pip`:
```bash
pip install django-http-debug
```
## Configuration
Once installed in the same environment as your Django application, add the following to `INSTALLED_APPS` in your Django settings:
```python
INSTALLED_APPS = [
# ...
'django_http_debug',
# ...
]
```
And add this to `MIDDLEWARE`:
```python
MIDDLEWARE = [
# ...
"django_http_debug.middleware.DebugMiddleware",
# ...
]
```
Then run `./manage.py migrate` to create the necessary database tables.
## Usage
You can configure new endpoints in the Django admin. These will only work if they are for URLs that are not yet being served by the rest of your application.
Give an endpoint a path (starting without a `/`) such as:
webhooks/debug/
You can optionally configure the returned body or HTTP headers here too.
If you want to return a binary body - a GIF for example - you can set that endpoint to use Base64 encoding and then paste a base64-encoded string into the body field.
On macOS you can create base64 strings like this:
```bash
base64 -i pixel.gif -o -
```
Any HTTP requests made to `/webhooks/debug/` will be logged in the database. You can view these requests in the Django admin.
You can turn off the "Logging enabled" option on an endpoint to stop logging requests to it to the database.
## Development
To contribute to this library, first checkout the code. Then create a new virtual environment:
```bash
cd django-http-debug
python -m venv venv
source venv/bin/activate
```
Now install the dependencies and test dependencies:
```bash
pip install -e '.[test]'
```
To run the tests:
```bash
pytest
```
---
./django_http_debug/models.py
---
from django.db import models
import base64
class DebugEndpoint(models.Model):
path = models.CharField(max_length=255, unique=True)
status_code = models.IntegerField(default=200)
content_type = models.CharField(max_length=64, default="text/plain; charset=utf-8")
headers = models.JSONField(default=dict, blank=True)
content = models.TextField(blank=True)
is_base64 = models.BooleanField(default=False)
logging_enabled = models.BooleanField(default=True)
def __str__(self):
return self.path
def get_absolute_url(self):
return f"/{self.path}"
class RequestLog(models.Model):
endpoint = models.ForeignKey(DebugEndpoint, on_delete=models.CASCADE)
method = models.CharField(max_length=10)
query_string = models.CharField(max_length=255, blank=True)
headers = models.JSONField()
body = models.TextField(blank=True)
is_base64 = models.BooleanField(default=False)
timestamp = models.DateTimeField(auto_now_add=True)
def __str__(self):
return f"{self.method} {self.endpoint.path} at {self.timestamp}"
def set_body(self, body):
try:
# Try to decode as UTF-8
self.body = body.decode("utf-8")
self.is_base64 = False
except UnicodeDecodeError:
# If that fails, store as base64
self.body = base64.b64encode(body).decode("ascii")
self.is_base64 = True
def get_body(self):
if self.is_base64:
return base64.b64decode(self.body.encode("ascii"))
return self.body
---
./django_http_debug/__init__.py
---
def example_function():
return 1 + 1
---
./django_http_debug/admin.py
---
import re
import filetype
from django.contrib import admin
from django.utils.html import format_html
from django.utils.safestring import mark_safe
from .models import DebugEndpoint, RequestLog
sequence_re = re.compile(r"((?:\\x[0-9a-f]{2})+)")
octet_re = re.compile(r"(\\x[0-9a-f]{2})")
QUERY_STRING_TRUNCATE = 16
@admin.register(DebugEndpoint)
class DebugEndpointAdmin(admin.ModelAdmin):
list_display = ("path", "status_code", "logging_enabled")
list_filter = ("logging_enabled",)
@admin.register(RequestLog)
class RequestLogAdmin(admin.ModelAdmin):
list_display = (
"timestamp",
"endpoint",
"method",
"query_string_truncated",
"body_preview",
"is_base64",
)
list_filter = ("endpoint", "method", "is_base64")
readonly_fields = (
"endpoint",
"query_string",
"method",
"headers",
"body",
"body_display",
"is_base64",
"timestamp",
)
def has_add_permission(self, request):
return False
def has_change_permission(self, request, obj=None):
return False
def query_string_truncated(self, obj):
return obj.query_string[:QUERY_STRING_TRUNCATE] + (
"…" if len(obj.query_string) > QUERY_STRING_TRUNCATE else ""
)
query_string_truncated.short_description = "Query string"
def body_preview(self, obj):
body = obj.get_body()
if isinstance(body, bytes):
return f"Binary data ({len(body)} bytes)"
return body[:50] + ("..." if len(body) > 50 else "")
body_preview.short_description = "Body preview"
def body_display(self, obj):
body = obj.get_body()
if not isinstance(body, bytes):
return format_html("<pre>{}</pre>", body)
# Attempt to guess filetype
suggestion = None
match = filetype.guess(body[:1000])
if match:
suggestion = "{} ({})".format(match.extension, match.mime)
encoded = repr(body)
# Ditch the b' and trailing '
if encoded.startswith("b'") and encoded.endswith("'"):
encoded = encoded[2:-1]
# Split it into sequences of octets and characters
chunks = sequence_re.split(encoded)
html = []
if suggestion:
html.append(
'<p style="margin-top: 0; font-family: monospace; font-size: 0.8em;">Suggestion: {}</p>'.format(
suggestion
)
)
for chunk in chunks:
if sequence_re.match(chunk):
octets = octet_re.findall(chunk)
octets = [o[2:] for o in octets]
html.append(
'<code style="color: #999; font-family: monospace">{}</code>'.format(
" ".join(octets).upper()
)
)
else:
html.append(chunk.replace("\\\\", "\\"))
return mark_safe(" ".join(html).strip().replace("\\r\\n", "<br>"))
---
./django_http_debug/middleware.py
---
from django.http import Http404
from .views import debug_view
class DebugMiddleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
response = self.get_response(request)
if response.status_code == 404:
path = request.path.lstrip("/")
debug_response = debug_view(request, path)
if debug_response:
return debug_response
return response
---
./django_http_debug/views.py
---
import base64
from django.http import HttpResponse
from django.views.decorators.csrf import csrf_exempt
from .models import DebugEndpoint, RequestLog
@csrf_exempt
def debug_view(request, path):
try:
endpoint = DebugEndpoint.objects.get(path=path)
except DebugEndpoint.DoesNotExist:
return None # Allow normal 404 handling to continue
if endpoint.logging_enabled:
log_entry = RequestLog(
endpoint=endpoint,
method=request.method,
query_string=request.META.get("QUERY_STRING", ""),
headers=dict(request.headers),
)
log_entry.set_body(request.body)
log_entry.save()
content = endpoint.content
if endpoint.is_base64:
content = base64.b64decode(content)
response = HttpResponse(
content=content,
status=endpoint.status_code,
content_type=endpoint.content_type,
)
for key, value in endpoint.headers.items():
response[key] = value
return response
---
./django_http_debug/migrations/0003_requestlog_query_string.py
---
# Generated by Django 5.0.3 on 2024-08-07 20:05
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
(
"django_http_debug",
"0002_debugendpoint_content_type_debugendpoint_is_base64",
),
]
operations = [
migrations.AddField(
model_name="requestlog",
name="query_string",
field=models.CharField(blank=True, max_length=255),
),
]
---
./django_http_debug/migrations/0002_debugendpoint_content_type_debugendpoint_is_base64.py
---
# Generated by Django 5.0.3 on 2024-08-07 19:46
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("django_http_debug", "0001_initial"),
]
operations = [
migrations.AddField(
model_name="debugendpoint",
name="content_type",
field=models.CharField(default="text/plain; charset=utf-8", max_length=64),
),
migrations.AddField(
model_name="debugendpoint",
name="is_base64",
field=models.BooleanField(default=False),
),
]
---
./django_http_debug/migrations/__init__.py
---
---
./django_http_debug/migrations/0004_debugendpoint_logging_enabled.py
---
# Generated by Django 5.0.3 on 2024-08-07 20:11
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("django_http_debug", "0003_requestlog_query_string"),
]
operations = [
migrations.AddField(
model_name="debugendpoint",
name="logging_enabled",
field=models.BooleanField(default=True),
),
]
---
./django_http_debug/migrations/0001_initial.py
---
# Generated by Django 5.0.2 on 2024-08-07 18:12
import django.db.models.deletion
from django.db import migrations, models
class Migration(migrations.Migration):
initial = True
dependencies = []
operations = [
migrations.CreateModel(
name="DebugEndpoint",
fields=[
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("path", models.CharField(max_length=255, unique=True)),
("status_code", models.IntegerField(default=200)),
("headers", models.JSONField(blank=True, default=dict)),
("content", models.TextField(blank=True)),
],
),
migrations.CreateModel(
name="RequestLog",
fields=[
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("method", models.CharField(max_length=10)),
("headers", models.JSONField()),
("body", models.TextField(blank=True)),
("is_base64", models.BooleanField(default=False)),
("timestamp", models.DateTimeField(auto_now_add=True)),
(
"endpoint",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
to="django_http_debug.debugendpoint",
),
),
],
),
]
---
./tests/test_django_http_debug.py
---
from django_http_debug import example_function
def test_example_function():
assert example_function() == 2
---
step by step advice on how to implement automated tests for this, which is hard because the tests need to work within a temporary Django project that lives in the tests/ directory somehow. Provide all code at the end.
To implement automated tests for the django-http-debug package, we'll need to create a temporary Django project within the tests directory. Here's a step-by-step guide on how to set up and implement the tests:
- Create a new directory structure for the tests:
tests/
__init__.py
test_django_http_debug.py
test_project/
__init__.py
settings.py
urls.py
conftest.py
- Update the
pyproject.tomlfile to include pytest-django as a test dependency:
[project.optional-dependencies]
test = ["pytest", "pytest-django"]- Create a
pytest.inifile in the root directory:
[pytest]
DJANGO_SETTINGS_MODULE = tests.test_project.settings
python_files = tests.py test_*.py *_tests.py- Update the
tests/test_project/settings.pyfile:
SECRET_KEY = "django-insecure-test-key"
DEBUG = True
ALLOWED_HOSTS = ["*"]
INSTALLED_APPS = [
"django.contrib.admin",
"django.contrib.auth",
"django.contrib.contenttypes",
"django.contrib.sessions",
"django.contrib.messages",
"django.contrib.staticfiles",
"django_http_debug",
]
MIDDLEWARE = [
"django.middleware.security.SecurityMiddleware",
"django.contrib.sessions.middleware.SessionMiddleware",
"django.middleware.common.CommonMiddleware",
"django.middleware.csrf.CsrfViewMiddleware",
"django.contrib.auth.middleware.AuthenticationMiddleware",
"django.contrib.messages.middleware.MessageMiddleware",
"django.middleware.clickjacking.XFrameOptionsMiddleware",
"django_http_debug.middleware.DebugMiddleware",
]
ROOT_URLCONF = "tests.test_project.urls"
DATABASES = {
"default": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": ":memory:",
}
}
USE_TZ = True- Update the
tests/test_project/urls.pyfile:
from django.contrib import admin
from django.urls import path
urlpatterns = [
path("admin/", admin.site.urls),
]- Create a
tests/conftest.pyfile:
import pytest
from django.core.management import call_command
@pytest.fixture(scope="session")
def django_db_setup(django_db_setup, django_db_blocker):
with django_db_blocker.unblock():
call_command("migrate")
@pytest.fixture
def client():
from django.test.client import Client
return Client()- Update the
tests/test_django_http_debug.pyfile with the following tests:
import pytest
from django.urls import reverse
from django_http_debug.models import DebugEndpoint, RequestLog
@pytest.mark.django_db
def test_debug_endpoint_creation():
endpoint = DebugEndpoint.objects.create(
path="test/endpoint",
status_code=200,
content="Test content",
content_type="text/plain",
)
assert endpoint.path == "test/endpoint"
assert endpoint.status_code == 200
assert endpoint.content == "Test content"
assert endpoint.content_type == "text/plain"
@pytest.mark.django_db
def test_debug_view(client):
DebugEndpoint.objects.create(
path="test/endpoint",
status_code=200,
content="Test content",
content_type="text/plain",
)
response = client.get("/test/endpoint")
assert response.status_code == 200
assert response.content == b"Test content"
assert response["Content-Type"] == "text/plain"
@pytest.mark.django_db
def test_request_logging(client):
endpoint = DebugEndpoint.objects.create(
path="test/log",
status_code=200,
content="Log test",
)
client.get("/test/log?param=value")
log = RequestLog.objects.filter(endpoint=endpoint).first()
assert log is not None
assert log.method == "GET"
assert log.query_string == "param=value"
@pytest.mark.django_db
def test_logging_disabled(client):
DebugEndpoint.objects.create(
path="test/nolog",
status_code=200,
content="No log test",
logging_enabled=False,
)
client.get("/test/nolog")
assert RequestLog.objects.count() == 0
@pytest.mark.django_db
def test_base64_content(client):
import base64
content = base64.b64encode(b"Binary content").decode()
DebugEndpoint.objects.create(
path="test/binary",
status_code=200,
content=content,
is_base64=True,
content_type="application/octet-stream",
)
response = client.get("/test/binary")
assert response.status_code == 200
assert response.content == b"Binary content"
assert response["Content-Type"] == "application/octet-stream"
@pytest.mark.django_db
def test_custom_headers(client):
DebugEndpoint.objects.create(
path="test/headers",
status_code=200,
content="Custom headers test",
headers={"X-Custom-Header": "Test Value"},
)
response = client.get("/test/headers")
assert response.status_code == 200
assert response["X-Custom-Header"] == "Test Value"These tests cover the main functionality of the django-http-debug package, including endpoint creation, request handling, logging, base64 content, and custom headers.
To run the tests, you can now use the following command:
pytestThis setup creates a temporary Django project within the tests directory, allowing you to test the django-http-debug package in isolation. The tests cover various aspects of the package's functionality, ensuring that it works as expected.
give me bash commands I can run to create the directory structure
Certainly! Here are the bash commands to create the directory structure for the tests:
# Ensure you're in the root directory of the project
cd /path/to/django-http-debug
# Create the test directory structure
mkdir -p tests/test_project
# Create necessary files
touch tests/__init__.py
touch tests/test_django_http_debug.py
touch tests/test_project/__init__.py
touch tests/test_project/settings.py
touch tests/test_project/urls.py
touch tests/conftest.py
# Create pytest.ini in the root directory
touch pytest.ini
# If you haven't already created these files, create them as well
touch pyproject.toml
touch README.mdAfter running these commands, you'll have the following structure:
django-http-debug/
├── pyproject.toml
├── README.md
├── pytest.ini
└── tests/
├── __init__.py
├── conftest.py
├── test_django_http_debug.py
└── test_project/
├── __init__.py
├── settings.py
└── urls.py
You can then use your preferred text editor to add the content to each of these files as described in the previous response. For example:
# Edit the files (replace 'nano' with your preferred text editor)
nano pytest.ini
nano tests/test_project/settings.py
nano tests/test_project/urls.py
nano tests/conftest.py
nano tests/test_django_http_debug.pyRemember to also update your pyproject.toml file to include the test dependencies as mentioned earlier.