Optimal bounds on approximation of submodular and XOS functions by juntas

Abstract

We investigate the approximability of several classes of real-valued functions by functions of a small number of variables (juntas). Our main results are tight bounds on the number of variables required to approximate a function f: {0, 1}n → [0, 1] within ℓ2-error ϵ over the uniform distribution: (a) If f is submodular, then it is ϵ-close to a function of O(1/2ϵ2 log 1/ϵ) variables. This is an exponential improvement over previously known results [V. Feldman, P. Kothari, and J. Vondrák, JMLR Workshop Conf. Proc., 35 (2013), pp. 711-740]. We note that Ω(1/2ϵ) variables are necessary even for linear functions. (b) If f is fractionally subadditive (XOS) it is ϵ-close to a function of 2Ω(1/ϵ2) variables. This result holds for all functions with low total ℓ1-influence and is a real-valued generalization of Friedgut's theorem for Boolean functions. We show that 2Ω(1/ϵ) variables are necessary even for XOS functions. As applications of these results, we provide learning algorithms over the uniform distribution. For XOS functions, we give a probably approximately correct learning algorithm that runs in time 21/poly(γ ϵ)poly(n). For submodular functions we give an algorithm in the more demanding probably mostly approximately correct (PMAC) learning model [M. F. Balcan and N. Harvey, preprint, arXiv:1008.2159, 2012] which requires a multiplicative (1+γ) factor approximation with probability at least 1 - ϵ over the target distribution. Our uniform distribution algorithm runs in time 21/poly(γ ϵ)poly(n). This is the first algorithm in the PMAC model that can achieve a constant approximation factor arbitrarily close to 1 for all submodular functions (even over the uniform distribution). It relies crucially on our bounds for approximation by juntas. As follows from the lower bounds in the above paper by Feldman, Kothari, and Vondrák both of these algorithms are close to optimal. We also give applications for proper learning, testing, and agnostic learning of these classes.