Primary tabs

A Spectral Analysis of Dot-product Kernels

Research thrust(s): Advanced Algorithms for Deep Learning

DPK Illustration

In [1], we present eigenvalue decay estimates of integral operators associated with compositional dot-product kernels. The estimates improve on previous ones established for power series kernels on spheres. This allows us to obtain the volumes of balls in the corresponding reproducing kernel Hilbert spaces. We discuss the consequences on statistical estimation with compositional dot product kernels and highlight interesting trade-offs between the approximation error and the statistical error depending on the number of compositions and the smoothness of the kernels.

Results

We present the comparison of the convergence rate of regularized least-squares with dot product kernels on the sphere for various Taylor expansions of the kernel. Namely, given standard assumptions on the data distribution, we present how close a regularized least squares estimator is to the conditional mean function as the number of samples n augment.

DPK Results

References

[1] Scetbon, M. and Harchaoui, Z. 2021. A Spectral Analysis of Dot-product Kernels. In Proccedings of the 24th International Conference on Artificial Intelligence and Statistics. (Bibtex)

Acknowledgments

We acknowledge support from NSF DMS 2023166, CCF 2019844, CIFAR “Learning in Machines and Brains” progam, and “Chaire d’excellence de l’IDEX Paris Saclay”. The authors would like to thank A. Rudi and L. Rosasco for pointing out relevant literature.