title: A Rainforest in Machine Learning author: Arjoonn Sharma @ CLAP Research patat: incrementalLists: true ...
When all you have is a hammer, everything is a nail. That's why we fevicol expands your carpentry.
So talk is all about decision trees?
- We discuss everything BUT neural networks.
- Lots of good talks about neural networks, 3Blue1Brown for example.
- Running neural networks in production is hard. Quora for example uses a version of decision trees.
- Everybody needs ❤.
What was overthrown with the current wave of neural networks?
- Now an empirically dominated field, machine learning is. -Yoda
- Some algorithms which quietly do the work while remaining in the shadows
- Core Vector Machines are cheaper alternatives to SVMs
- Genetic search and programming
- Random embeddings
- Adversarial Genetic Search
- Meta genetic search
There were supposed to be decision trees in this talk. Where are they?
- Everyone has their own world
- Human crafted trees
- I3, CART
- Bagging
- Boosting
How do you solve the bombing problem?
- A fundamental change is needed. Here are some:
- Decision Stream combat tree depth.
- Soft Decision Trees combat non linear boundaries.
- Rotation Forests combat random forest saturation.
- Cascade forests combat the yellow duck problem.
Decision trees are certainly diverse. So what?
- Substitute 'conditions' by massive use of a simpler kind of compute "+-".
- With decision trees we directly compute "conditions".
- Better software provides direct benefit to usage.
I think I've seen some of these concepts before...
| The old | The new |
|---|---|
| Decision Trees | MLP |
| CART | Backprop |
| Bagging, Boosting | Dropout, Distillation |
| Rotation Forest | Convolution |
| Cascade Forest | Deep Neural Networks |
| Directed Acyclic Graph | Residual Connections |
| Meta GS | Auto ML |
| Adversarial GS | GAN |
No talk is complete without questions. Please fire away! This talk can be found online
More references.
Conditional Random Fields Supervised forest kernel Born again trees Catboost Core Vector Machine Manifold learning Gini coefficient World model nn