### abstract ###
%   <- trailing '%' for backward compatibility of
sty file Given a sample from a probability measure with support on a submanifold in Euclidean space one can construct a neighborhood graph which can be seen as an approximation of the submanifold
The graph Laplacian of such a graph is used in several machine learning methods like semi-supervised learning, dimensionality reduction and clustering
In this paper we determine the pointwise limit of three different graph Laplacians used in the literature as the sample size increases and the neighborhood size approaches zero
We show that for a uniform measure on the submanifold all graph Laplacians have the same limit up to constants
However in the case of a non-uniform measure on the submanifold only the so called random walk graph Laplacian converges to the weighted Laplace-Beltrami operator
### introduction ###
In recent years, methods based on graph Laplacians have become increasingly popular in machine learning
They have been used in semi-supervised learning  CITATION , spectral clustering  CITATION  and dimensionality reduction  CITATION
Their popularity is mainly due to the following properties of the Laplacian which will be discussed in more detail in a later section:   the Laplacian is the generator of the diffusion process (label propagation in semi-supervised learning),  the eigenvectors of the Laplacian have special geometric properties (motivation for spectral clustering),  the Laplacian induces an adaptive regularization functional, which adapts to the density and the geometric structure of the data (semi-supervised learning, classification)
If the data lies in  SYMBOL  the neighborhood graph built from the random sample can be seen as an approximation of the continuous structure
in particular,  if the data has support on a low-dimensional submanifold the neighborhood graph is a discrete approximation of the submanifold
In machine learning we are interested in the intrinsic properties and objects of this submanifold
The approximation of the Laplace-Beltrami operator via the graph Laplacian is a very important one since it has numerous applications as we will discuss later
Approximations of the Laplace-Beltrami operator or related objects have been studied for certain special deterministic graphs
The easiest case is a grid in  SYMBOL
In numerics it is standard to approximate the Laplacian with finite-differences schemes on the grid
These can be seen as a special instances of a graph Laplacian
There convergence for decreasing grid-size follows easily by an argument using Taylor expansions
Another more involved example is the work of  CITATION , where for a graph generated by an  SYMBOL -packing of a manifold, the equivalence of certain properties of random walks on the graph and Brownian motion on the manifold have been established
The connection between random walks and the graph Laplacian becomes obvious by noting that the graph Laplacian as well as the Laplace-Beltrami operator are the generators of the diffusion process on the graph and the manifold, respectively
In  CITATION  the convergence of a discrete approximation of the Laplace Beltrami operator for a triangulation of a 2D-surface in  SYMBOL  was shown
However,  it is unclear whether the approximation described there can be written as a graph Laplacian and whether this result can be generalized to higher dimensions
In the case where the graph is generated randomly, only first results have been proved so far
The first work on the large sample limit of graph Laplacians has been done by  CITATION
There the authors studied the convergence of the regularization functional induced by the graph Laplacian using the law of large numbers for  SYMBOL -statistics
In a second step taking the limit of the neighborhoodsize  SYMBOL , they got  SYMBOL  as the effective limit operator in  SYMBOL
Their result has recently been generalized to the submanifold case and uniform convergence over the space of H\"older-functions by  CITATION
In  CITATION , the neighborhoodsize  SYMBOL  was kept fixed while the large sample limit of the graph Laplacian was considered
In this setting, the authors showed strong convergence results of graph Laplacians to certain integral operators, which imply the convergence of the eigenvalues and eigenfunctions
Thereby showing the consistency of spectral clustering for a fixed neighborhood size
In contrast to the previous work  in this paper we will consider the large sample limit and the limit as the neighborhood size approaches zero simultaneously for a certain class of neighbhorhood graphs
The main emphasis lies on the case where the data generating measure has support on a submanifold of  SYMBOL
The bias term, that is the difference between the continuous counterpart of the graph Laplacian and the Laplacian itself has been studied first for compact submanifolds without boundary by  CITATION  and  CITATION  for the Gaussian kernel and a uniform data generating measure and was then generalized by  CITATION  to general isotropic weights and general probability measures
Additionally Lafon showed that the use of data-dependent weights for the graph allows to control the influence of the density
They all show that the bias term converges pointwise if the neighborhood size goes to zero
The convergence of the graph Laplacian towards these continuous averaging operators was left open
This part was first studied by  CITATION  and  CITATION
In  CITATION  the convergence was shown for the so called unnormalized graph Laplacian in the case of a uniform probability measure on a compact manifold without boundary and using the Gaussian kernel for the weights, whereas in  CITATION  the pointwise convergence was shown for the random walk graph Laplacian in the case of general probability measures on non-compact manifolds with boundary using general isotropic data-dependent weights
More recently  CITATION  have extended the pointwise convergence for the unnormalized graph Laplacian shown by  CITATION  to uniform convergence on compact submanifolds without boundary giving explicit rates
In  CITATION , see also  CITATION , the rate of convergence given by  CITATION  has been improved in the setting of the uniform measure
In this paper we will study the three most often used graph Laplacians in the machine learning literature and show their pointwise convergence in the general setting of  CITATION  and  CITATION , that is we will in particular consider the case where by using data-dependent weights for the graph we can control the influence of the density on the limit operator
In Section  we introduce the basic framework necessary to define graph Laplacians for general directed weighted graphs and then simplify the general case to undirected graphs
in particular,  we define the three graph Laplacians used in machine learning so far, which we call the normalized, the unnormalized and the random walk Laplacian
In Section  we introduce the neighborhood graphs studied in this paper, followed by an introduction to the so called weighted Laplace-Beltrami operator, which will turn out to be the limit operator in general
We also study properties of this limit operator and provide insights why and how this operator can be used for semi-supervised learning, clustering and regression
Then finally we present the main convergence result for all three graph Laplacians and give the conditions on the neighborhood size as a function of the sample size necessary for convergence
In Section  we illustrate the main result by studying the difference between the three graph Laplacians and the effects of different data-dependent weights on the limit operator
In Section  we prove the main result
We introduce a framework for studying non-compact manifolds with boundary and provide the necessary assumptions on the submanifold  SYMBOL , the data generating measure  SYMBOL  and the kernel  SYMBOL  used for defining the weights of the edges
We would like to note that the theorems given in Section  contain slightly stronger results than the ones presented in Section
The reader who is not familiar with differential geometry will find a brief introduction to the basic material used in this paper in Appendix
