# qeML Example: Nonparametric Quantile Regression

This article is originally published at https://matloff.wordpress.com

In this post, I will first introduce the concept of quantile regression (QR), a powerful technique that is rarely taught in stat courses. I’ll give an example from the **quantreg **package, and then will show how **qeML** can be used to do model-free QR estimation. Along the way, I will also illustrate the use of *closures* in R.

Notation: We are predicting a scalar Y (including the case of dummy/one-hot variables) from a feature vector X.

In its simplest form, QR estimates the conditional median of Y given X, as opposed to the usual conditional mean, using a linear model. As we all know, the median is less affected by outliers than is the mean, so QR is giving us outlier robustness. As a bonus, we dispense with the homoskedasticity assumption, i.e. constant Var(Y|X).

But it’s more than that. We can model any conditional quantile, e.g. estimate the 80th percentile weight for each human height. Quantile analysis has a variety of applications.

One can conduct QR in R with the **quantreg** package, written by Prof. Roger Koenker, one of the major names in the QR field. Here is an example, using the **qeML** dataset **mlb**:

`> data(mlb)`

> library(quantreg)

> z <- rq(Weight ~ Height,data=mlb,tau=0.80)

> summary(z)

Call: rq(formula = Weight ~ Height, tau = 0.8, data = mlb)

tau: [1] 0.8

Coefficients:

Value Std. Error t value Pr(>|t|)

(Intercept) -201.66667 17.58797 -11.46617 0.00000

Height 5.66667 0.23856 23.75376 0.00000

As you can see, the call form here is like that of the R linear model function **lm**, and we could have had multiple predictors, e.g. age in addition to height.

But what if we don’t believe a linear model is appropriate? Of course, as usual we may consider adding polynomial terms, and there is also a package **quantreg.nonpar**. But we obtain model-free estimates easily using **qeKNN** in **qeML**.

Standard k-Nearest Neighbors estimation is simple. Say to predict the weight of someone 70 inches tall and 28 years ago, we find the k closest data points in our training data to the vector (70,28). We then compute the mean weight among those k people, and it’s then our predicted weight for the new person who has known height and age but unknown weight.

But **qeKNN **offers the user more flexibility, via an argument **smoothingFtn**. Instead of computing mean Y among the neighbors, we can specify the median, or even specify that a small linear model be fit to the neighboring data. The latter may be useful if the new person to be predicted is either very short or very tall, as things tend to be biased near the edges of a dataset. If the new person is 77 inches tall, most or all people in our neighboring data will be shorter than this, thus lighter, so our prediction based on the mean will be biased downward.

But we can also specify our own smoothingFtn, perfect for QR. We simply define a function that gives us the desired Y quantile among the neighbors.

The call form is

`smoothingFtn(nearIdxs,x,y,predpt)`

Here **x** and **y** are our X and Y training data, **predpt** is the new X value at which we wish to predict (redundant in most cases), and **nearIdxs** are the indices in **x **and **y** of the nearest neighbors to **predpt**. Note that at the time kNN calls **smoothingFtn**, the indices have already been computed.

Our code is then

`sftn <- function(nearIdxs,x,y,predpt)`

{

nearYs <- y[nearIdxs]

quantile(nearYs,0.80)

}

u <- mlb[c('Height','Age','Weight')]

set.seed(9999) # qeML ftns do random holdout

z <- qeKNN(u,'Weight',smoothingFtn=sftn)

predict(z,c(70,28)) # prints 200

It would be nice, though, to run this for a general quantile level **q**, rather than the special case 0.80. But we can’t do that directly, because the **smoothingFtn** argument to **qeKNN** must be a function object, no provision there for an argument to **smoothingFtn**. But we can accomplish what we want via R *closures*.

`makeSmFtn <- function(q) function(newIdxs,x,y,predpt) quantile(y[newIdxs],q)`

To understand this, one must first know more about the R reserved word **function**. Consider this simple example:

`f <- function(x) x^2`

Here we are saying, “R, please create a function for me. Its formal argument will be named **x**, and it will compute and return the square of that quantity. After you create that function–an object, just like other R entities–assign it to **f**.” In other words, **function** creates functions. As I like to tell my students,

The function of the function namedfunctionis to create functions!

Now, going back to **makeQFtn** above, it creates a function object (the call to quantile), and returns that object, just as with **f **above, but the key point is that here the value of **q** will be “baked in” to that object.

So our call to **qeKNN** for general **q** would be

z <- qeKNN(u,’Weight’,smoothingFtn=makeSmFtn(q))

Thanks for visiting r-craft.org

This article is originally published at https://matloff.wordpress.com

Please visit source website for post related comments.