deep-censored-quantile-regression.Rmd

---
title: "Deep Censored Quantile Regression"
author: "Wush Wu"
date: "September 7, 2018"
output: ioslides_presentation
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = FALSE)
```

## Remarks from KDD

- 可以開始研究如何把我們在Second Price Auction的應用拓展到First Price
    - Challenge: Where is the data of first price auction?
- 我認為在 KDD 2018 Deep Censored Learning 之中，我們測試三種 parametric distribution 不是對的方向
    - 我希望可以有更有彈性的，對於 distribution 的描述

## Problem Definition

- $y$ 是 winning price
- $X$ 是 features
- 給定一個數列$0 < tau_1 < \tau_2 < ... < \tau_k < 1$
- 如果用$q_{\tau}$代表$y$的$\tau$-quantile，我希望找到$q_{\tau_1} \leq q_{\tau_2} \leq ... \leq q_{\tau_k} | X$ 的估計
- 只要$\tau$ 夠密，這就是一個描述CDF的方式，可能會比現有的parametric way 更有彈性

## Possible Solution: Deep Quantile Regression

- [Deep Quantile Regression](https://towardsdatascience.com/deep-quantile-regression-c85481548b5a), A blog posted at Mar. 25.
- $y$ is an observed winning price, $x$ is the features, $f(\tau, x)$表示基於$x$預測的$\tau$-quantile
- Loss:

$$\mathbb{L}(\delta = y - f(\tau, x)) = \left\{\begin{array}{lc}
\tau \delta & \text{ if } \delta \geq 0, \\
(\tau - 1) \delta & \text{ if } \delta < 0.
\end{array}\right.$$

## Loss Function

```{r}
get.f <- function(tau) {
  force(tau)
  function(z) {
    ifelse(z >= 0, z * tau, z * (tau - 1))
  }
}
f <- get.f(.25)
curve(f, -5, 5, main = "tau = .25")
```

## Loss Function

- It is trivial to implement $\tau$-quantile regression in Keras
- However, can we estimate a series of quantile at the same time?
    - Yes, multi-task learning
    - Challenge: cross-over $\hat{q}_{\tau_1} > \hat{q}_{\tau_2}$
- Existed Implementation: [Keras Quantile Model](https://github.com/sachinruk/KerasQuantileModel/blob/master/Keras%20Quantile%20Model.ipynb)
- [Quantile Loss in Tensorflow](https://github.com/strongio/quantile-regression-tensorflow/blob/master/Quantile%20Loss.ipynb)

## Censored Quantile Regression

- [Yang et al. A New Approach to Censored Quantile Regression Estimation.Journal of Computational and Graphical Statistics 2018.](https://amstat.tandfonline.com/doi/abs/10.1080/10618600.2017.1385469?journalCode=ucgs20#.W5F2JxhLeCg)

## Challenge

- How to extend existed censored quantile regression model to deep learning models?
- How to make inference based on the estimated quantiles? How to compare them with our existed works?