Skip to content

Instantly share code, notes, and snippets.

@rampage644
Created August 24, 2014 09:59
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rampage644/4977c599b4808af5cff9 to your computer and use it in GitHub Desktop.
Save rampage644/4977c599b4808af5cff9 to your computer and use it in GitHub Desktop.
Impalad summary
## Git repo
Find modified impala [here](https://github.com/rampage644/impala-cut). First, have a look at [this](https://github.com/rampage644/impala-cut/blob/executor/README.md) *README* file.
## Task description
Original task was to prune impalad to some sort of *executor* binary which only executes part of query. Two approaches were suggested: top-down and bottom-up. I used bottom-up approach.
My intention was to write unittest that whill actually test the behavior we need. So, look at `be/src/runtime/plan-fragment-executior-test.cc`. It contains all possible tests (that is, actual code snippets) to run part of query with or without data. Doing so helped me a lot to understand impalad codebase relative to query execution.
## Achievements ?
First of all, I actually wrote unittest that executes part of query given the query itself and data. As side effect, i did some impalad codebase research (everything relating to query execution). There are a lot code left aside, though.
### Impalad peculiarities
1. MPPI-oriented. Uses lots of threads and even more. Vanilla impalad when doing nothing has ~183 threads created.
2. Some of functionality is done in JAVA via JNI.
3. Closely coupled code, a lot of *Singletones*.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment