quantile_score

typhon.retrieval.scores.quantile_score(y_tau, y_test, taus)

The quantile loss function.

Let $y_tau$ be an estimate of the $tautext{th}$ quantile of a random distribution $Y$. Then the quantile loss is a proper scoring rule that measures the quality of the estimate $y_tau$ given a sample $y$ from $Y$:

\[\begin{split}\mathcal{L}_\tau(y_\tau, y_{true}) = \begin{cases} (1 - \tau)|y_\tau - y_{true}| & \text{ if } y_\tau < y_\text{true} \\ \tau |y_\tau - y_\text{true}| & \text{ otherwise. }\end{cases}\end{split}\]

Parameters:

• y_tau (numpy.array) – Numpy array with shape (n, k) containing one row of k estimated quantiles for each of the n test cases.

• y_test (numpy.array) – Numpy array with the n observed test values of the conditional distributions whose quantiles are estimated by the elements in y_tau

• taus (numpy.array) – Numpy array containing the k quantile fractions \(\tau\) that are estimated by the columns in y_tau.

Returns:

Array of shape (n, k) containing the quantile scores for each quantile estimate.

Raises:

ValueError – If the shapes of y_tau, y_test and taus are inconsistent.