Skip to content

Instantly share code, notes, and snippets.

@vytas7

vytas7/multipart.rst

Last active Feb 16, 2020
Embed
What would you like to do?
Multipart form handling in Falcon proposal

multipart/form-data handling in Falcon proposal

Design philosophy

Multipart form handling should be performant, straightforward, and leave full control of the parsing process to the user, i.e. no surprising magic such as automatic creation of large files and saving anything there by default (although tools/helpers may exist to assist in that too).

The interface has also been discussed (and refined) at PyCon US '19 sprints between me and kgriffs.

Interface

Multipart forms would be treated as yet another type of request media. A new handler, MultipartFormHandler is introduced for these. However, in contrast to existing JSON, Msgpack etc handlers, it would not consume all the stream by default.

Instead, one would iterate the multipart form object (now found in req.media):

for part in req.media:
    # Do something with the body part
    print("* {} ({})".format(part.name, part.content_type))

    # ...

The BodyPart class, but currently exposes the following properties:

  • stream -- stream wrapper just for the current body part
  • data -- body part content bytes
  • content_type, would default to text/plain if not specified, as per RFC
  • text -- the current body part decoded as text string (only provided it is of type text/plain, None otherwise)
  • media -- automatically parsed by media handlers in the same way as req.media
  • name, filename -- relevant parts from the Content-Disposition header
  • secure_filename -- sanitized filename that could safely be used on the server filesystem, a-la Werkzeug

Show me the code!

The code has now been merged to Falcon master as part of this PR.

Warning

The code is currently slated for an alpha release.

As such, the code is not production ready. There may be bugs lurking in the code, and the interface may change based on community feedback.

Try also the test.py below.

Example: direct upload to AWS S3

(from (hopefully) the future FAQ)

The stream of a body part is a file-like object implementing the read() method that may be used with boto3's upload_fileobj:

import boto3
s3 = boto3.client('s3')

# ...

for part in req.media:
    if part.name == 'myfile':
        s3.upload_fileobj(part.stream, 'mybucket', 'mykey')

Note

Falcon is not endorsing any particular cloud service provider, and AWS S3 and boto3 are referenced here just as a popular example. The same principles hopefully apply to other cloud storage APIs implementing upload of arbitrary file-like objects.

See also the unit test case in my extended WiP test suite illustrating the concepts presented above: https://github.com/vytas7/falcon-multipart-tests/blob/master/tests/test_cloud_upload.py

"""
Falcon multipart form handling demo.
Install the code in your environment from the dev branch::
pip install git+https://github.com/vytas7/falcon@multipart-form-handler
Run with ``gunicorn`` or any other WSGI server of choice::
# pip install gunicorn
gunicorn test:api
Then, just ``POST`` your form of choice, or just visit http://127.0.0.1:8000.
"""
import hashlib
import io
import falcon
from falcon import media
class Demo:
def on_get(self, req, resp):
resp.body = (
'<!doctype html>\n'
'<html>\n'
'<head><title>Multipart demo</title></head>\n'
'<body>\n'
'<form action="/" method="post" enctype="multipart/form-data">'
'Name: <input type="text" name="name"><br>'
'File: <input type="file" name="upload"><br>'
'<input type="submit" value="Submit">'
'</form>\n'
'</body>\n'
'</html>\n'
)
resp.content_type = falcon.MEDIA_HTML
def on_post(self, req, resp):
output = []
for part in req.media:
data = {
'name': part.name,
'content_type': part.content_type,
'filename': part.filename,
}
if part.filename:
length = 0
sha256 = hashlib.sha256()
while True:
chunk = part.stream.read(io.DEFAULT_BUFFER_SIZE * 4)
if not chunk:
break
length += len(chunk)
sha256.update(chunk)
data.update(length=length, sha256=sha256.hexdigest())
output.append(data)
resp.media = output
handlers = media.Handlers({
falcon.MEDIA_JSON: media.JSONHandler(),
falcon.MEDIA_MULTIPART: media.MultipartFormHandler(),
})
api = falcon.API()
api.req_options.media_handlers = handlers
api.add_route('/', Demo())
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment