Skip to content

Instantly share code, notes, and snippets.

@vinipsmaker

vinipsmaker/proposal.md Secret

Last active March 29, 2016 14:10
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save vinipsmaker/4998ccfacb971a0dc1bd to your computer and use it in GitHub Desktop.
Save vinipsmaker/4998ccfacb971a0dc1bd to your computer and use it in GitHub Desktop.
Download ZIP
Raw
proposal.md

Boost Summer of Code 2016

We propose to design and deliver a high-quality implementation of a versatile HTTP parser and generator framework for Boost.Http.

We anticipate to submit this library separately for formal Boost review so it can be used independently on the existing Boost.Http project.

The student has demonstrated both the competencies and endurance needed for a success completion of the proposed project.

Background

Boost.Http was initiated as part of GSoC 2014. Boost.Http and Boost.Hana were the GSoC 2014 projects that gathered more interest from the community. You can still check that Boost.Http is the most starred/watched repo in the BoostGSoC14 org and even surpass older GSoC projects that had more time to be known.

Boost.Http is currently used to serve HTTP requests on various ARM and MIPS boards for the Dynastrom multi-room audio solution.

Boost.Http went through a formal Boost review in August 2015, which concluded that:

"There was a general consensus that Boost ought have a HTTP library, but that Boost.Http was too incomplete in its present state."

The reviewed Boost.Http only provided a server-side API, but there was a strong interest in a client-side API as well. The HTTP generator part of this proposal is a prerequisite step towards a client-side API.

The HTTP parser part also addresses the explicit requests for a HTTP parser, as well as the the request for a header-only version of Boost.Http (which is not possible with the current HTTP parser).

You can check how the HTTP parser project relates to the previous review in detail by looking at the list of issues created in response of the past review: https://github.com/boostgsoc14/boost.http/issues?utf8=%E2%9C%93&q=is%3Aissue+label%3Aboost-review+.

Extensive research of current approaches was done before implementing Boost.Http. Boost.Http is unique. It has a foundation strong enough to expose the whole HTTP power abstracting which backend you're using without compromising performance. It easily allows you to reuse its implementation in various environments. It fully follows Asio patterns and you can use it with whatever async model you think it fits better in your application while integrating with other network traffic in the same thread.

Having such library within Boost would give Boost a special place among competitors such as POCO, cpp-netlib, pion and others.

Proposal

We propose to re-think HTTP parsers with ideas from parser combinators and incremental parsing.

Parser combinators means that the parser is put together by mini-parsers for the individual BNF rules in the grammar.

Incremental parsing means that the input is parsed piece by piece, give much user flexibility to control the parsing and reducing memory footprint.

These characteristics are important for HTTP Upgrade requests to WebSockets, which starts off using the normal HTTP grammar and then changes on-the-fly to a binary protocol.

The HTTP generator will also be incremental, allowing the user to construct a HTTP message piece by piece.

Timeline

  • 22 April to 20 May: Slow and careful iterations on the API design. This is extra research phase.
  • 23 May to 27 May: Mock parser API should be ready at this moment. Also start implementing HTTP parsing.
  • 30 May to 3 June: Continue to develop the HTTP parser.
  • 6 June to 10 June: Finish the parser algorithm and start writing the documentation.
  • 13 June to 17 June: Start adding tests.

The HTTP parser should be done by now. Except it is not fully tested nor integrated into Boost.Http. But it should be usable by midterm.

  • 20 June to 24 June: Initial integration into Boost.Http. This means creating a separate branch where the Ryan Dahl's HTTP parser is replaced.
  • 27 June to 1 July: Static imutable messages and initial generators implementation.
  • 4 July to 8 July: "Resumable generators" (i.e. make sure it's possible to continue the generation so if you buffer gets fully used, you can just send it and use it again later when it is free to generate the rest of the message).
  • 11 July to 15 July: Finish the integration into Boost.Http and add parser options/traits to configure the Boost.Http socket.
  • 18 July to 22 July: Start a benchmark application.
  • 25 July to 29 July: Finish the benchmark application. Run some careful benchmarks.
  • 1 August to 5 August: Start working on some client API. The main purpose of the client API will be to test the parser/generator that should be already in place.
  • 8 August to 12 August: Continue to work on the client API.
  • 15 August to 19 August: Integration tests between client and server, parser and generator. Or it could be extra time in case I delay any previous task.

Bonus items in case I finish earlier:

  • Replace remaining items in Boost.Http that require separate compilation. Turn the library into header-only.
  • An URL parser.

I believe this is a realistic timeline focusing on deliverables. There are other factors that make me believe this is even more achievable/realistic like previous experience developing Boost.Http and alternative designs that are close (they are close, but they don't relate directly to HTTP and I'll need to write new shiny code anyway) to what I want to achieve.

The most feared task here may be the HTTP client code, but I also have reserved some ideas to prepare myself to the challenge.

After the Summer

Continue to be one of the core Boost.Http library maintainers. Continue to evolve the library until it is accepted into Boost.

Student Personal Details

Background Information

C++ - Git - HTTP - Qt - async - MongoDB - Emacs

Qualifications summary

  • I'm passionate about network-facing problems and it is what I do since I entered on my university.
  • I'm a free software fan and I use ArchLinux as my main system.
  • I love performance and I'm used to favor languages built on the zero-overhead principle.

GitHub Projects

  • vinipsmaker/tufao : Asynchronous web framework for C++ built on top of Qt.
    • Project started in 2011 and still maintained.
    • 170 stars on GitHub.
    • 19 users on the mailing list and usually 4 users online on IRC.
    • Inspired by Node.js.
    • C++, but cross-platform.
    • Great documentation (reference, tutorial, examples, QtCreator plugin to speedup...).
    • Very flexible.
    • I made a talk on Tufão for one of the largest free software events in Brazil -- FISL.
    • It gave me great experience on how to coordinate collaboration with possible contributors, how to encourage positive feedback, run a free software project...
  • vinipsmaker/tamanduang : a compiler class' assignment
    • My first Rust code.
    • A lexer for a shitty language that I'm designing.
  • vinipsmaker/tufao-qtcreatorplugin : plugin for QtCreator IDE to easily create new Tufão projects
    • It gave me experience on how to work with mysterious APIs that are half-baked, underdocumented and break API every release.
    • Fair enough, it's a very small project.
  • vinipsmaker/logica-whopper : Just a logic class' assignment.
  • vinipsmaker/drunken-octo-sansa : Random code to solve random problems
    • Code created while I was training to compete on the ACM-ICPC.
  • vinipsmaker/c-except : An exception system for C in a single header file
    • Just an experiment to see how I'd implement an exception system for the C language.
  • vinipsmaker/dotfiles : Config files for my ArchLinux system

Technical Skills

  • Programming Languages: Rust C++ Python JavaScript Java
  • Frameworks: STL Node.js + ConnectJS Flask Qt GStreamer v8 Boost.Asio
  • Databases: MongoDB redis
  • Tools: Git CMake Emacs bash + some regex

Professional Experience

  • C++ developer, Expertise Solutions, Brazil (September, 2014 – today)
    • Implement JavaScript bindings for the EFL toolkit.
    • Technologies got to work on: C++ v8 EFL JavaScript Git autotools
  • Rust developer, MaidSafe (June, 2015 – today)
    • Implement connection manager library capable of abstracting over multiple protocols, NAT traversal and peer discovery.
    • Technologies got to work on: Rust, uTP, cross-platform quirks, NAT traversal

Other Personal Projects

Undergraduate Courses Taken

Extra Curricular Awards

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment