Schedule for: 18w5112 - Shape-Constrained Methods: Inference, Applications, and Practice

A buffet dinner is served daily between 5:30pm and 7:30pm in the Vistas Dining Room, the top floor of the Sally Borden Building.

Breakfast is served daily between 7 and 9am in the Vistas Dining Room, the top floor of the Sally Borden Building.

We give a direct derivation of the distribution of the maximum and the location of the maximum of one-sided and two-sided Brownian motion with a negative parabolic drift. The argument uses a relation between integrals of special functions, in particular involving integrals with respect to functions which can be called `incomplete Scorer functions'. The relation is proved by showing that both integrals, as a function of two parameters, satisfy the same extended heat equation, and the maximum principle is used to show that these solutions must therefore have the stated relation. Once this relation is established, a direct derivation of the distribution of the maximum and location of the maximum of Brownian motion minus a parabola is possible, leading to a considerable shortening of the original proofs.

We study the least squares regression function estimator over the class of real-valued functions on $[0, 1]^d$ that are increasing in each coordinate. For uniformly bounded signals and with a fixed, cubic lattice design, we establish that the estimator achieves the minimax rate of order $n^{\min\{2/(d+2),1/d\}}$ in the empirical $L_2$-loss, up to poly-logarithmic factors. Further, we prove a sharp oracle inequality, which reveals in particular that when the true regression function is piecewise constant on $k$ hyperrectangles, the least squares estimator enjoys a faster, adaptive rate of convergence of $(k/n) \min(1,2/d)$, again up to poly-logarithmic factors. Previous results are confined to the case $d = 2$. Finally, we establish corresponding bounds (which are new even in the case $d = 2$) in the more challenging random design setting. There are two surprising features of these results: first, they demonstrate that it is possible for a global empirical risk minimisation procedure to be rate optimal up to poly-logarithmic factors even when the corresponding entropy integral for the function class diverges rapidly; second, they indicate that the adaptation rate for shape-constrained estimators can be strictly worse than the parametric rate.

Unobserved heterogeneity is an common feature of statistical survival analysis where it is usually referred to as frailty. Parametric mixture models are frequently used to capture these effects, but it is desirable to consider nonparametric mixture models as well. We illustrate the latter approach with a reanalysis of a large scale mortality study of Mediterranean fruit flies employing the Kiefer-Wolfowitz nonparametric maximum likelihood estimator of the mixing distribution for a nonparametric Weibull frailty model. Empirical Bayes methods are employed to design actuarially fair life insurance contracts to contrast the approach with parametric methods. Some associated problems of profile likelihood will also be addressed.

Meet in foyer of TCPL to participate in the BIRS group photo. The photograph will be taken outdoors, so dress appropriately for the weather. Please don't be late, or you might not be in the official group photo!

Lunch is served daily between 11:30am and 1:30pm in the Vistas Dining Room, the top floor of the Sally Borden Building.

Meet in the Corbett Hall Lounge for a guided tour of The Banff Centre campus.

The log-concave maximum likelihood estimator of a density on $\mathbb R^d$ on a sample of size n is known to attain the minimax optimal rate of convergence up to a log factor when $d=2$ and $d=3$. In this talk, I will present new results on adaptation properties in this multivariate setting. This is based on joint work with Oliver Feng, Aditya Guntuboyina and Richard Samworth.

We consider extended optimization formulations that unify tasks classically done sequentially or hierarchically. In particular, we develop self-tuning robust penalties, that require optimizing over both model parameters and shape parameters (rather than using cross-validation or black-box optimization). If time permits, we will also show extended formulations that simultaneously learn a model and detect outliers in the input data.

Many modern applications rely on convex optimization, which offers a rich modeling paradigm as well as strong theoretical guarantees and computational advantages. Convex duality often plays a central role. I will describe the geometry behind a simple form of convex duality based on polarity of convex cones, its relationship to lifting, and show how it leads to a computationally efficient algorithm for the phase-retrieval problem in X-ray crystallography.

A buffet dinner is served daily between 5:30pm and 7:30pm in the Vistas Dining Room, the top floor of the Sally Borden Building.

This talk focuses on adaptation properties of a certain class of shape constrained density estimators. It is well-known that the Nonparametric Maximum Likelihood Estimator (NPMLE) for non-increasing densities on the nonnegative real line (also known as the Grenander estimator) adapts to histogram densities. I will first review these results and then present analogous (but weaker) results for the NPMLE for estimating Gaussian location mixture densities. An application to Gaussian denoising will also be discussed. The results are based on the paper https://arxiv.org/pdf/1712.02009.pdf which I co-authored with Sujayam Saha at the University of California, Berkeley.

I describe some recent research examining shape constraints in option price data. Financial theory indicates that the ratio between the option-implied and physical return densities for a market index should be monotonically decreasing, but the data suggest otherwise. Beare and Schmidt (Journal of Applied Econometrics, 2016) assess the statistical significance of departures from monotonicity, while Beare and Dossani (Quantitative Finance, 2018) examine whether departures from monotonicity may be used to improve probabilistic forecasts of market return indices. I discuss the empirical results of these papers, and also related methodological results of Beare and Moon (Econometric Theory, 2015) and Beare and Fang (Electronic Journal of Statistics, 2017) on the application of the functional delta method using the least concave majorant operator.

Varying coefficient models are quite appealing for describing complex data structures. In this talk we consider varying coefficient models in the context of longitudinal data. The aim is to study possible qualitative properties of the unknown coefficient functions in the model. Among the qualitative properties that are of interest are: whether a coefficient function is constant or not; and whether it is a monotone function or not, or a convex function or not. Also simultaneous testing for qualitative properties of several coefficient functions is an important issue for applications. This problem of shape testing appears in different settings, such as mean regression and quantile regression, and in homoscedastic or heteroscedastic settings. We discuss shape constraint testing procedures under various settings, illustrate the performances of the procedures with some small simulation study, and demonstrate their use in real data analysis.

We consider nonparametric kernel estimation of an instrumental regression function φ defined by conditional moment restrictions that stem from a structural econometric model E(Y − φ(Z)|W ) = 0, and involve endogenous variables Y and Z and instruments W . The function φ is the solution to an ill-posed inverse problem. Our primary focus lies in shape constrained estimation, and we present a simple and robust approach towards shape constrained local nonparametric instrumental regression. The constraints can be imposed on the estimated function $\hat \varphi$, its derivatives, or combinations thereof. Our approach facilitates imposing, say, axioms of consumer/producer theory on an otherwise unrestricted but smooth estimate. Theoretical underpinnings are provided and applications are considered.

We consider bandwidth matrix selection for kernel density estimators (KDEs) of density level sets in $\mathbb R^d$, $d \ge 2$. We also consider estimation of highest density regions, which differs from estimating level sets in that one specifies the probability content of the set rather than specifying the level directly; this complicates the problem. Bandwidth selection for KDEs is well-studied, but the goal of most methods is to minimize a global loss function for the density or its derivatives. The loss we consider here is instead the measure of the symmetric difference of the true set and estimated set. We derive an asymptotic approximation to the corresponding risk. The approximation depends on unknown quantities which can be estimated, and the approximation can then be minimized to yield a choice of bandwidth, which we show in simulations performs well.

We construct adaptive confidence sets in isotonic and convex regression. In univariate isotonic regression, if the true parameter is piecewise constant with $k$ pieces, then the Least-Squares estimator achieves a parametric rate of order $k/n$ up to logarithmic factors. We construct honest confidence sets that adapt to the unknown number of pieces of the true parameter. The proposed confidence set enjoys uniform coverage over all non-decreasing functions. Furthermore, the squared diameter of the confidence set is of order $k/n$ up to logarithmic factors, which is optimal in a minimax sense. In univariate convex regression, we construct a confidence set that enjoys uniform coverage and such that its diameter is of order $q/n$ up to logarithmic factors, where $q-1$ is the number of changes of slope of the true regression function. We will also discuss application of the presented techniques to sparse linear regression.

In this talk we discuss nonparametric estimation of a log-concave density and nonparametric estimation of a tail inflation function (McCullagh and Polson, 2012) in a common framework. A variation of the latter model assumes data coming from a density $f_0(x) \exp(\rho(x))$ with a given density $f_0$ and $\rho$ being an unknown convex function, a so-called tail inflation function. We discuss and illustrate a variation of the active set approach by Duembgen, Huesler and Rufibach (2007/2011) which works in both settings. This is joint work with Peter McCullagh, Alexandre Moesching and Christof Straehl.

I investigate the asymptotic distribution of linear quantile regression coefficient estimates when the parameter may lie on the boundary of the parameter space, and related inference procedures when the null hypothesis asserts that the parameters lie on a boundary of this set. Particular attention is paid to parameter spaces defined by sets of linear inequalities. I provide a uniform characterization of the constrained quantile regression process over an interval of quantile levels. This asymptotic theory is used to derive asymptotic inference methods for three related processes based on the constrained quantile regression process.

During the production of steel, there are various process parameters one can tune to influence the quality of the resulting product. The holy grail is to have a program that determines the needed values of these parameters based on the required mechanical properties of the steel. But,..... there is a long way to go still. In this presentation I will sketch approaches taken to solve some statistical sub-problems within this project. Based on joint work with Kimberly McGarrity, Martina Vittorietti, Piet Kok and Jilt Sietsma.

Nonparametric estimation methods avoid functional form misspecification which is caused by parametric assumptions. However, the flexibility of nonparametric methods often cause difficulties in interpreting results in many applications of productivity analysis. Fortunately, microeconomic theory provides additional structure for modeling a production or cost function which can be interpreted as shape constraints. Several nonparametric shape constrained estimators have been proposed that combine the advantage of avoiding functional misspecification with improving the interpretability of estimation results relative to unconstrained nonparametric methods. However, existing methods only allow simple shape constraints such as concavity/convexity and monotonicity to be imposed. These structure exclude economic phenomena such as increasing returns to scale due to specialization, fixed costs, or learning. Thus more general functional structures are desirable such as the Regular Ultra Passum (RUP) law. In this paper, we propose an algorithm to estimate general production function which satisfies the RUP law and input isoquant convexity. We do not impose homotheticity of input isoquants in our most general algorithm, thus the input isoquants can have different shapes at different output levels. Our estimation algorithm is composed of two steps: 1) Estimate convex isoquants at a set of output levels, 2) Estimate S-shape functions on a set of rays from the origin. We estimate the production function for the corrugated cardboard industry from the Japanese census of manufacturing data. The estimation results provide a description of the supply-side of Japanese cardboard manufacturing industry as we report marginal productivity, marginal rate of substitution and most productive scale size.

We develop a framework for analyzing M-estimators subject to general classes of constraints for example about the shape, support, continuity, slope, location of modes, and other conditions that, individually or in combination, restrict the family of functions under consideration. We establish existence of estimators and their cluster points, strong consistency under mild assumptions, large-deviation global rates of convergence, and robustness in the presence of model misspecification. The framework views the functions to be estimated (densities, classifiers, regression functions, distribution functions etc.) as elements of spaces of semicontinuous functions equipped with the hypo-distance metric. This metric emerges as natural and convenient when considering broad classes of side conditions. For distribution functions, the hypo-distance metrizes weak convergence. In this metric, convergence of densities implies convergence of modes, near-modes, height of modes, and high-likelihood events. Likewise, convergence of regression functions guarantees convergence of maxima and maximum values, a central property in response surface modelling. Thus, the framework automatically leads to strong consistency of a rich class of plug-in estimators for modes, maxima, and related quantities. We avoid the full assumptions associated with uniform laws of large numbers and instead leverage one-sided conditions that suffice for almost sure epi-convergence. The framework facilitates the development of a large variety of upper bounds on global rates of convergence through new covering numbers for semicontinuous functions; we give large deviation results for maximum likelihood estimation of densities and least-squares regression. The richness of possible constraints is illustrated in the context of density estimation by simultaneously considering bounds on density values and subgradients, restriction to concavity, penalization that encourages lower modes, and imprecise information about the expected value.

Population domain means are frequently expected to respect qualitative (shape) constraints that arise naturally on the survey data. For example, mean salaries for STEM-related jobs might be higher than those for Humanities-related jobs. A constrained estimator of domain means that merges the most common survey estimators and shape constraints into the estimation stage is proposed. Inequality constraints that can be expressed with irreducible matrices are considered, as these cover a broad class of restrictions. The constrained estimator is shown to be consistent and asymptotically normal distributed under mild conditions, given that the assumed shape is true. Simulation experiments indicate that both estimation and variability of domain means is improved by the constrained estimator in comparison with usual unconstrained estimators, especially for small domains. An application of the proposed estimator to the 2015 US National Survey of College Graduates is shown.

We tackle the problem of high-dimensional nonparametric density estimation by taking the class of log-concave densities on $\mathbb{R}^p$ and incorporating within it a symmetry assumption. We say that a log-concave density is $K$-symmetrical for a convex body $K$ if all the level sets of the density are scaled versions of $K$. This extends the notion of spherical symmetry and leads naturally to a two-stage estimation algorithm where we first estimate the location and the contour $K$ of the density and then estimate the slope profile of the density. We prove that the risk (in Hellinger loss) of this estimator can be bounded by a sum of two terms the first of which accounts for the errors in estimating the location and $K$ and the second of which is an adaptive rate that is independent of $p$ and identical to the rate of estimating a univariate log-concave density. This immediately yields a bound on the rate of convergence for estimating an elliptically-symmetric log-concave density. Our future work focuses on developing estimators for the contour when $K$ is a general convex body.

n the talk, I will return to our joint work with Roger Koenker, originating more than 10 years ago, on the use of Renyi entropies/divergencies in the estimation of unimodal densities, and also will review some recent developments in this direction. Alluding to the workshop format of the meeting, the talk aims at exposing and opening to potential discussion possibly controversial aspects of the methodology, rather than giving an overview of undisputable achievements. In particular, the focus will be on how far it is reasonable to go from the statistical point of view - how much and when the theory still underpins the progress on the numerical front. If time permits, some attention will be also given to other recent, both theoretical and numerical, successes.

5-day workshop participants are welcome to use BIRS facilities (BIRS Coffee Lounge, TCPL and Reading Room) until 3 pm on Friday, although participants are still required to checkout of the guest rooms by 12 noon.