{"id":595,"date":"2024-12-16T07:04:12","date_gmt":"2024-12-16T07:04:12","guid":{"rendered":"https:\/\/mailitics.com\/index.php\/2024\/12\/16\/2412-09779\/"},"modified":"2024-12-16T07:04:12","modified_gmt":"2024-12-16T07:04:12","slug":"2412-09779","status":"publish","type":"post","link":"https:\/\/mailitics.com\/index.php\/2024\/12\/16\/2412-09779\/","title":{"rendered":"A Statistical Analysis for Supervised Deep Learning with Exponential Families for Intrinsically Low-dimensional Data"},"content":{"rendered":"

A Statistical Analysis for Supervised Deep Learning with Exponential Families for Intrinsically Low-dimensional Data
\n \t

\n
<\/BR>
\n
\n \t

\n
<\/BR><\/p>\n

arXiv:2412.09779v1 Announce Type: new
\nAbstract: Recent advances have revealed that the rate of convergence of the expected test error in deep supervised learning decays as a function of the intrinsic dimension and not the dimension $d$ of the input space. Existing literature defines this intrinsic dimension as the Minkowski dimension or the manifold dimension of the support of the underlying probability measures, which often results in sub-optimal rates and unrealistic assumptions. In this paper, we consider supervised deep learning when the response given the explanatory variable is distributed according to an exponential family with a $beta$-H”older smooth mean function. We consider an entropic notion of the intrinsic data-dimension and demonstrate that with $n$ independent and identically distributed samples, the test error scales as $tilde{mathcal{O}}left(n^{-frac{2beta}{2beta + bar{d}_{2beta}(lambda)}}right)$, where $bar{d}_{2beta}(lambda)$ is the $2beta$-entropic dimension of $lambda$, the distribution of the explanatory variables. This improves on the best-known rates. Furthermore, under the assumption of an upper-bounded density of the explanatory variables, we characterize the rate of convergence as $tilde{mathcal{O}}left( d^{frac{2lfloorbetarfloor(beta + d)}{2beta + d}}n^{-frac{2beta}{2beta + d}}right)$, establishing that the dependence on $d$ is not exponential but at most polynomial. We also demonstrate that when the explanatory variable has a lower bounded density, this rate in terms of the number of data samples, is nearly optimal for learning the dependence structure for exponential families.<\/div>\n

\t

\n
<\/BR>
\n Saptarshi Chakraborty, Peter L. Bartlett
\n \t

\n
<\/BR>
\nGo to original source<\/a>
\n \t

\n
<\/BR><\/p>\n","protected":false},"excerpt":{"rendered":"

A Statistical Analysis for Supervised Deep Learning with Exponential Families for Intrinsically Low-dimensional Data arXiv:2412.09779v1 Announce Type: new Abstract: Recent advances have revealed that the rate of convergence of the expected test error in deep supervised learning decays as a function of the intrinsic dimension and not the dimension $d$ of the input space. Existing […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[62,113,190,112,191],"tags":[460,656,199],"class_list":["post-595","post","type-post","status-publish","format-standard","hentry","category-aimldsaimlds","category-cs-lg","category-math-st","category-stat-ml","category-stat-th","tag-beta","tag-dimension","tag-learning"],"_links":{"self":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts\/595"}],"collection":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/comments?post=595"}],"version-history":[{"count":0,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts\/595\/revisions"}],"wp:attachment":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/media?parent=595"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/categories?post=595"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/tags?post=595"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}