{"id":6736,"date":"2025-09-10T07:03:44","date_gmt":"2025-09-10T07:03:44","guid":{"rendered":"https:\/\/mailitics.com\/index.php\/2025\/09\/10\/2509-07289\/"},"modified":"2025-09-10T07:03:44","modified_gmt":"2025-09-10T07:03:44","slug":"2509-07289","status":"publish","type":"post","link":"https:\/\/mailitics.com\/index.php\/2025\/09\/10\/2509-07289\/","title":{"rendered":"Kernel VICReg for Self-Supervised Learning in Reproducing Kernel Hilbert Space"},"content":{"rendered":"<p>    Kernel VICReg for Self-Supervised Learning in Reproducing Kernel Hilbert Space<br \/>\n \t<BR><br \/>\n<BR><\/BR><br \/>\n    <!-- no image --><br \/>\n \t<BR><br \/>\n<BR><\/BR><\/p>\n<div>arXiv:2509.07289v1 Announce Type: new<br \/>\nAbstract: Self-supervised learning (SSL) has emerged as a powerful paradigm for representation learning by optimizing geometric objectives&#8211;such as invariance to augmentations, variance preservation, and feature decorrelation&#8211;without requiring labels. However, most existing methods operate in Euclidean space, limiting their ability to capture nonlinear dependencies and geometric structures. In this work, we propose Kernel VICReg, a novel self-supervised learning framework that lifts the VICReg objective into a Reproducing Kernel Hilbert Space (RKHS). By kernelizing each term of the loss-variance, invariance, and covariance&#8211;we obtain a general formulation that operates on double-centered kernel matrices and Hilbert-Schmidt norms, enabling nonlinear feature learning without explicit mappings.<br \/>\n  We demonstrate that Kernel VICReg not only avoids representational collapse but also improves performance on tasks with complex or small-scale data. Empirical evaluations across MNIST, CIFAR-10, STL-10, TinyImageNet, and ImageNet100 show consistent gains over Euclidean VICReg, with particularly strong improvements on datasets where nonlinear structures are prominent. UMAP visualizations further confirm that kernel-based embeddings exhibit better isometry and class separation. Our results suggest that kernelizing SSL objectives is a promising direction for bridging classical kernel methods with modern representation learning.<\/div>\n<p> \t<BR><br \/>\n <BR><\/BR><br \/>\n    M. Hadi Sepanj, Benyamin Ghojogh, Paul Fieguth<br \/>\n \t<BR><br \/>\n<BR><\/BR><br \/>\n<a href=\"https:\/\/arxiv.org\/abs\/2509.07289\">Go to original source<\/a><br \/>\n \t<BR><br \/>\n <BR><\/BR><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Kernel VICReg for Self-Supervised Learning in Reproducing Kernel Hilbert Space arXiv:2509.07289v1 Announce Type: new Abstract: Self-supervised learning (SSL) has emerged as a powerful paradigm for representation learning by optimizing geometric objectives&#8211;such as invariance to augmentations, variance preservation, and feature decorrelation&#8211;without requiring labels. However, most existing methods operate in Euclidean space, limiting their ability to capture [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[62,611,113,112],"tags":[1135,199,3749],"class_list":["post-6736","post","type-post","status-publish","format-standard","hentry","category-aimldsaimlds","category-cs-cv","category-cs-lg","category-stat-ml","tag-kernel","tag-learning","tag-vicreg"],"_links":{"self":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts\/6736"}],"collection":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/comments?post=6736"}],"version-history":[{"count":0,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts\/6736\/revisions"}],"wp:attachment":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/media?parent=6736"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/categories?post=6736"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/tags?post=6736"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}