Skip to content

Instantly share code, notes, and snippets.

@codekitchen
Last active April 15, 2017 01:16
Show Gist options
  • Save codekitchen/8290940 to your computer and use it in GitHub Desktop.
Save codekitchen/8290940 to your computer and use it in GitHub Desktop.
diff of Rack::Multipart::Parser for streaming uploads
1c1
< require 'rack/utils'
---
> require 'streaming_upload'
3,4c3,18
< module Rack
< module Multipart
---
> class StreamingMultipartRequest < ActionDispatch::Request
> def parse_multipart(env)
> Parser.new(env).parse
> end
>
> # This code is mostly taken directly from Rack's Multipart::Parser class. It
> # has been adapted to upload the file directly to S3 using a temporary
> # filename, for later renaming, while we are reading off the socket. The file
> # is also stored in a local tempfile as normal, since we need that for FFMPEG
> # processing.
> #
> # I've copied the entire class rather than sub-classing because of the
> # invasiveness of the change -- monkey patching in this functionality would
> # be error prone as we update Rack versions.
> #
> # The diff from Rack::Multipart::Parser is available at https://gist.github.com/codekitchen/8290940
6c20,32
< BUFSIZE = 16384
---
> BUFSIZE = 65536
>
> EOL = "\r\n"
> MULTIPART = %r|\Amultipart/.*boundary=\"?([^\";,]+)\"?|n
> TOKEN = /[^\s()<>,;:\\"\/\[\]?=]+/
> CONDISP = /Content-Disposition:\s*#{TOKEN}\s*/i
> DISPPARM = /;\s*(#{TOKEN})=("(?:\\"|[^"])*"|#{TOKEN})/
> RFC2183 = /^#{CONDISP}(#{DISPPARM})+$/i
> BROKEN_QUOTED = /^#{CONDISP}.*;\sfilename="(.*?)"(?:\s*$|\s*;\s*#{TOKEN}=)/i
> BROKEN_UNQUOTED = /^#{CONDISP}.*;\sfilename=(#{TOKEN})/i
> MULTIPART_CONTENT_TYPE = /Content-Type: (.*)#{EOL}/ni
> MULTIPART_CONTENT_DISPOSITION = /Content-Disposition:.*\s+name="?([^\";]*)"?/ni
> MULTIPART_CONTENT_ID = /Content-ID:\s*([^#{EOL}]*)/ni
31c57
< Utils.normalize_params(@params, name, data) unless data.nil?
---
> ::Rack::Utils.normalize_params(@params, name, data) unless data.nil?
49c75
< @params = Utils::KeySpaceConstrainedParams.new
---
> @params = ::Rack::Utils::KeySpaceConstrainedParams.new
54c80
< @boundary_size = Utils.bytesize(@boundary) + EOL.size
---
> @boundary_size = ::Rack::Utils.bytesize(@boundary) + EOL.size
82c108
< raise EOFError, "bad content body" if Utils.bytesize(@buf) >= BUFSIZE
---
> raise EOFError, "bad content body" if ::Rack::Utils.bytesize(@buf) >= BUFSIZE
104c130
< body = Tempfile.new("RackMultipart")
---
> body = StreamingUpload.new("RackMultipart")
138c164
< filename = Utils.unescape(filename)
---
> filename = ::Rack::Utils.unescape(filename)
176d200
< end
@codekitchen
Copy link
Author

wow my WIP diff was huge. In the end it came down to only one significant line changed? Crazy. Maybe Rack just needs DI for the Tempfile class.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment